Interactive basketball system
By integrating sensors and cameras in the basketball system, combining machine learning models, real-time monitoring and feedback of user shooting performance, the problem that the existing system cannot effectively improve basketball skills is solved, multi-user interactive games are realized, and training experience is improved.
Patent Information
- Application Number
- CN202411023538.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2020-08-19
- Filing Date
- 2021-08-18
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-08-18
AI Technical Summary
The existing basketball training system is difficult to monitor and feedback users' shooting performance in real time, and lacks interactivity, so it cannot effectively improve users' basketball skills.
A basketball system with multi-layer components is designed to monitor users' shooting attempts in real time using sensors, cameras and machine learning models, provide feedback and suggestions for improvement, and support multi-user interactive game modes.
Real-time monitoring and feedback on users' shooting performance is achieved, helping users improve their basketball skills, and providing a variety of interactive game modes to enhance the training experience.
Smart Images

Figure CN119056030A_ABST
Abstract
Description
[0001] This application is a divisional application of an application with an application date of August 18, 2021, application number 202180063418.5, and invention name “Interactive Basketball System”.
[0002] CROSS-REFERENCE TO RELATED APPLICATIONS
[0003] This application claims the benefit of U.S. Provisional Application No. 63 / 067,422, filed on August 19, 2020, which is incorporated herein by reference. Technical Field
[0004] The present specification relates generally to the sport of basketball, and one particular embodiment relates to systems and methods for basketball games and various drills. Background Art
[0005] The game of basketball is a sport that may involve two teams competing against each other on a court. The two teams may compete against each other with the goal of shooting a basketball into the defending team's hoop while preventing the opposing team from shooting into its own hoop. Players of varying skill levels may play basketball and often practice basketball skills to improve their chances of success in basketball games against opponents. Additionally, players may hire coaches, trainers, or others to assist in developing their basketball skills over time. Summary of the invention
[0006] This specification describes a basketball system that includes specific components for monitoring users playing basketball. Specifically, the basketball system may include a backboard, a basket, and poles that support the backboard. The backboard may include multiple layers, each layer housing different components. The components may be positioned in the backboard in specific ways and different layouts to enable the components to monitor one or more characteristics of one or more users interacting with the basketball system on the court within the proximity of the basketball system.
[0007] In some embodiments, the basketball system can be placed at a basketball court. The basketball court can include, for example, a driveway, a street, a court inside a gymnasium that can include professional, college-level or younger sizes, and other various locations. The court can include, for example, two halves, each with free throw lines, three-point lines, half-court lines, and other features. The basketball system can monitor the characteristics of a user playing basketball up to the half-court line or the full length of the court. In other examples, the basketball system can monitor the characteristics of a user playing basketball while playing in their driveway, street, or other location.
[0008] The basketball system can monitor the characteristics of a user playing basketball or multiple users playing basketball. In particular, a user can interact with the basketball system to indicate that he / she requests to be tracked by the basketball system. The user can play basketball using the basketball system, and the basketball system can generate characteristics describing the user's performance. For example, the characteristics can include the number of throws attempted by the user, the number of throws made by the user through attempts, the user's movement, posture during throw attempts, characteristics of each throw attempt of the user, and other characteristics. The basketball system can provide these characteristics to the user's client device for later review, or display these characteristics on the basketball system in real time as the user interacts with the basketball system.
[0009] In some embodiments, the backboard of the basketball system may include components for tracking one or more users playing basketball with the basketball system and the ability to provide feedback to the user. For example, the backboard of the basketball system may include multiple layers, each layer accommodating different components. The front layer of the backboard may include a transparent coating for protecting the components within the backboard. The second layer of the backboard may include one or more components for monitoring the user interacting with the basketball system. In some embodiments, the second layer may also include a display for providing feedback to the user playing basketball together with the sensor. In other embodiments, the display may be positioned in the third layer, and the components may be disposed in the second layer. In some examples, the layers of the backboard may be sorted from front to back as: the first layer, the second layer, and / or the third layer.
[0010] In some embodiments, the components in the backboard may be configured to monitor the user playing basketball and provide feedback. For example, the components may include a camera sensing system, one or more speakers, one or more microphones, multiple sensors, a control unit, a display screen, and a power unit. The control unit may receive data from each of these components, generate the characteristics of the user, and provide feedback to the user to help improve their basketball skills or performance. In some examples, the control unit may train a machine learning model for tracking and generating the characteristics of the user playing basketball. The machine learning model may be, for example, a convolutional neural network (CNN). These components and their functions will be described in further detail below.
[0011] In some embodiments, the basketball system can provide various games for users to participate. For example, the basketball system enables users to play with another user or other users at different locations geographically. In this example, another user can use his own basketball system to play basketball, and two basketball systems can communicate with each other in real time when two users use their corresponding basketball systems to play. For example, two backboards can display real-time video feeds of throwing statistics, another user's basketball game, and other information that each user can view when interacting with the basketball system. In another example, the basketball system enables users to play games, such as training course mode, local head-to-head confrontation, live broadcast mode, and global competition mode. These games will be further described below.
[0012] In one general aspect, a basketball board includes: a display screen; a plurality of sensors configured to generate sensor data regarding a user's throwing attempts; one or more imaging devices configured to generate image data of the throwing attempts; a speaker; and a control unit, wherein the control unit is configured to: receive (i) sensor data from one or more of the plurality of sensors and (ii) image data from the one or more imaging devices; determine whether the throwing attempt is successful based on the received sensor data; generate an analysis indicating (i) characteristics of the user, (ii) characteristics of the throwing attempt, and (iii) suggestions for improving the throwing attempts for subsequent throwing attempts, and (iv) game performance based on the received image data and whether the throwing attempt is successful; and provide output data representing the analysis to one or more of (i) the speaker, (ii) the display screen, and (iii) a client device of the user.
[0013] Other embodiments of these and other aspects of the present disclosure include corresponding systems, devices, and computer programs configured to perform the actions of the method encoded on a computer storage device. A system of one or more computers may be so configured by means of software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the actions in operation. One or more computer programs may be so configured by means of instructions that, when executed by a data processing device, cause the device to perform the actions.
[0014] The above and other embodiments may each optionally include one or more of the following features, either alone or in combination. For example, one embodiment includes all of the following features in combination.
[0015] In some embodiments, the backboard includes, wherein the plurality of sensors include one or more of a LIDAR sensor, a motion sensor, a travel sensor, and an accelerometer, and wherein the LIDAR sensor is configured to detect a user's throwing attempt and one or both of an angle and a height of the basketball from the throwing attempt; the motion sensor is configured to detect one or more users on the court within close range of the backboard; the travel sensor is configured to determine whether the throwing attempt is successful; and the accelerometer is configured to determine an indication of the position of the basketball relative to the backboard based on accelerometer data and a vibration pattern.
[0016] In some embodiments, the backboard includes, wherein the one or more imaging devices include one or more depth sensing cameras or one or more RGB cameras, wherein the one or more depth sensing cameras are configured to perform one or more of the following: (i) detecting a user on the court, (ii) tracking the movement of the user, (iii) detecting a basketball used by the user for a throwing attempt, (iv) tracking the movement of the basketball, (v) detecting the user's posture, and wherein the one or more RGB cameras are configured to (i) record image data of the field of view of the court and (ii) record image data of the area below the backboard to detect when the user's throwing attempt corresponds to a layup.
[0017] In some embodiments, the backboard includes a basket frame coupled to the backboard.
[0018] In some embodiments, a backboard includes wherein the plurality of sensors includes a travel sensor configured to determine whether a basketball passes through the basket via a throwing attempt.
[0019] In some embodiments, the backboard includes wherein the speaker is configured to provide an audible output in response to receiving output data representative of the analysis from the control unit.
[0020] In some embodiments, the backboard includes, wherein the display screen is configured to display one or more of: (i) image data from the one or more imaging devices, (ii) a heads-up display (HUD) that displays throw attempts and shots made by a user or any other data relevant to the user and / or the game / training session, and (iii) image data from a second control unit connected via a network.
[0021] In some embodiments, the backboard further includes a protective layer coupled to the display screen.
[0022] In some embodiments, the backboard includes wherein the protective layer comprises tempered glass.
[0023] In some embodiments, the backboard includes a rim coupled to a protective layer.
[0024] In some embodiments, a backboard includes, wherein the control unit is configured to provide the received image data to a trained machine learning model to generate (i) characteristics of a user and (ii) characteristics of a shot attempt, and (iii) a game performance, wherein the characteristics of the user include an identification of the user and a position of the user relative to the backboard, wherein the characteristics of the shot attempt include an angle of the trajectory of the basketball and an indication of whether the basketball passes through a basket, and wherein the game performance includes data associated with a game in which the user is participating; and the control unit is configured to store the characteristics of the user and the characteristics of the shot attempt in a profile of the user in a server external to the backboard.
[0025] In some embodiments, the backboard includes a trained machine learning model configured to simultaneously identify and track multiple users on the court, and the control unit is configured to: associate each of the multiple users identified by the trained machine learning model with a stored user profile; and update each of the stored user profiles with characteristics of each user and characteristics of each user's throwing attempts.
[0026] In some embodiments, the backboard includes wherein the server stores multiple profiles corresponding to different users.
[0027] In some embodiments, the backboard includes, wherein the control unit is configured to: generate suggestions for improving a throwing attempt for a subsequent throwing attempt, wherein the suggestions include one or more of (i) posture, (ii) arm angle, (iii) release point of the basketball, and (iv) trajectory of the basketball; display the generated suggestions on a display screen; and provide audible voice output to a speaker to convey the generated suggestions to a user.
[0028] In some embodiments, the backboard includes wherein the control unit is configured to provide the generated suggestions to a client device of the user via a network.
[0029] In some embodiments, the backboard includes, wherein the control unit is configured to: determine resulting data based on the received sensor data, the resulting data including one or more of the following: (i) whether the throwing attempt caused the basketball to pass through the basket frame, (ii) whether the throwing attempt caused the basketball to bounce off the front side of the backboard and not pass through the basket frame, (iii) whether the throwing attempt caused the basketball to bounce off the basket frame and not pass through the basket frame, (iv) the location of the user's throwing attempt, and (v) the arc of the basketball during the user's throwing attempt; combine the resulting data with (i) characteristics of the user and (ii) characteristics of the throwing attempt output from the trained machine learning model; and store the received sensor data, the received image data, the combined resulting data, the characteristics of the user, the characteristics of the throwing attempt, and the generated suggestions in a profile of the user on a server.
[0030] In some embodiments, a backboard includes, wherein the control unit is configured to: receive an indication from a user to participate in a competition with a second user; connect to a second control unit associated with a second backboard used by the second user via a network, wherein the second control unit is located at a geographically different location from the control unit; provide the received image data to the second control unit via the network; receive second image data from the second control unit via the network; provide the received second image data from the second control unit to a display screen; count the number of throwing attempts of the user based on the received sensor data and the received image data; receive a second number of throwing attempts of the second user from the second control unit; provide (i) the number of throwing attempts of the user and (ii) the second number of throwing attempts of the second user to a display screen, wherein the display screen overlays the number of throwing attempts and the second number of throwing attempts over the received second image data; and provide the number of throwing attempts of the user to the second control unit.
[0031] In some embodiments, a backboard includes, wherein the control unit is configured to: receive an indication from a user to participate in a local competition with a second user; provide the received image data to a display screen; count the number of throwing attempts by the user based on the received sensor data and the received image data; count the number of throwing attempts by the second user based on the received sensor data and the received image data; and provide (i) the number of throwing attempts by the user and (ii) the number of throwing attempts by the second user to the display screen, wherein the display screen overlays the number of throwing attempts and the second number of throwing attempts over the received image data.
[0032] In some embodiments, the backboard includes, wherein the control unit is configured to: determine whether the throwing attempt results in the basketball passing through the basket based on: generating an inner conical area and an outer conical area in the received image data, the inner conical area including a first cone having a base and a first height that conforms to the basket frame, the outer conical area including a second cone having a radius centered at the center of the basket frame and a second height, determining whether the basketball enters a first portion of the outer conical area and leaves a second portion of the outer conical area; in response to leaving the second portion of the outer conical area, determining whether the basketball enters a third portion of the inner conical area and leaves the base of the inner conical area; and in response to determining that the basketball leaves the base of the inner conical area, determining that the result of the throwing attempt is a make because the basketball passes through the basket frame.
[0033] In some embodiments, the backboard includes, wherein the control unit is configured to: in response to determining that the basketball entered the third portion of the inner cone and did not leave the base of the inner cone, determine that the result of the shot attempt was a miss because the basketball did not pass through the basket.
[0034] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1A is a block diagram illustrating an example of a system for monitoring a user playing basketball.
[0036] Figure 1B is a block diagram illustrating an example of a system for a backboard.
[0037] Figure 1C is another block diagram illustrating an example of a system for a backboard.
[0038] Figure 2A is a block diagram illustrating an example of video analysis for detecting a basketball passing through a basket.
[0039] Figure 2B is a block diagram illustrating an example of video analysis for detecting a basketball that does not pass through a basket.
[0040] Figure 3 is a block diagram illustrating an example of a system in which two users play basketball using a connected basketball system.
[0041] Figure 4 is a block diagram illustrating an example computing system of a backboard system.
[0042] Figure 5 is a flow chart illustrating an example of a process for generating characteristics of a user playing basketball.
[0043] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION
[0044] Figure 1A 1 is a block diagram illustrating an example of a system 100 for monitoring one or more users interacting with the system 100 through an activity, such as one or more users playing basketball. The system 100 includes a basketball system 107, a basketball court 108, one or more users 102, and a client device 104 associated with the one or more users 102. The system 100 may also include a network and a server external to the basketball system 107. In brief, the system 100 may monitor the one or more users 102 playing basketball 106 using the basketball system 107, generate data describing characteristics of the one or more users 102 playing basketball 106, and provide the data as feedback to the one or more users 102. The system 100 may provide the data as feedback to the client device 104 or to a display of the basketball system 107. Figure 1AVarious operations in exemplary stages (A) through (G) are shown that may be performed in the sequence indicated or in another sequence.
[0045] In some embodiments, the basketball system 107 may include a backboard 110, a pole 113, and a basket 112. The basketball system 107 may include various components and algorithms that enable tracking and monitoring of the one or more users 102 playing a basketball game. In addition, the various components of the basketball system 107 may generate suggestions for improving the basketball skills of the users 102. The suggestions may focus on improving, for example, the trajectory of the user's shot, the user's posture during the shot, the user's dribbling technique, and other basketball techniques. As will be further described below, the basketball system 107 may track multiple users 102, track the users' basketball characteristics, and store this data in user profiles located in a server external to the basketball system 107.
[0046] In addition, the basketball system 107 enables the one or more users 102 to play basketball games with each other at the same basketball system and with other users of their own corresponding basketball systems (including other basketball systems located in various places around the world). In some embodiments, the user 102 can request to play a basketball game or other activities such as basketball training courses by interacting with the basketball system 107. In other embodiments, the user 102 can request to play a basketball game using the basketball system 107 by interacting with a client device 104, which communicates with the basketball system 107 via a network, such as one or more of Bluetooth, Wi-Fi, the Internet, cloud access, and a cellular data network (e.g., a network with 4G and 5G capabilities). As will be further described below, the basketball game or other activities may include, for example, a training course mode, a head-to-head matchup mode, a global competition mode, and a live broadcast mode, to name a few.
[0047] In some embodiments, basketball system 107 may include a basket rim 112 and a backboard 110 that are vertically supported above a basketball court 108 or a playing surface such as a driveway, street, lawn, or other applicable surface. Basketball system 107 includes a pole 113 or support on which backboard 110 and basket rim 112 are supported. In some examples, pole 113 may be inserted or embedded into the ground at a specific depth to maintain the stability of basketball system 107. In other examples, pole 113 may be inserted into a base platform above the ground that maintains the stability of basketball system 107. In some cases, basketball system 107 may be a small basketball system that fits over a door frame in an office or bedroom, for example.
[0048] In some embodiments, backboard 110 may include multiple layers, each layer accommodating different components. Specifically, the front layer of backboard 110 may include a transparent coating for protecting the components within the backboard. For example, the front layer of backboard 110 may include tempered glass, which (i) protects the components within backboard 110 and (ii) allows the user to see the display screen behind the front layer. Backboard 110 may include a second layer, which is placed or positioned behind the front layer. The second layer may include one or more components, such as sensors and cameras, for monitoring and generating data associated with users on the court (e.g., sensor data and image data). In some embodiments, the second layer of the backboard may also include a display for providing feedback to the one or more users playing basketball on basketball court 108. In this case, the sensors and cameras may be connected to the display screen. In other embodiments, the backboard may include a third layer positioned behind the second layer. In this embodiment, the third layer may include a display for providing feedback to the one or more users playing basketball on basketball court 108.
[0049] In some embodiments, basketball system 107 may include a control unit. The control unit may include one or more central processing units (CPUs), one or more graphics processing units (GPUs), and memory components for executing software via the CPUs and GPUs. In some examples, the control unit may be located on the back side of backboard 110. In other examples, the control unit may be located within the second or third layer of backboard 110.
[0050] Typically, the control unit may receive sensor and image data from the one or more components within the backboard. Based on the received sensor and image data, the control unit may generate data about basketball characteristics of one or more users 102 playing basketball using the basketball system 107 on the basketball court 108. The control unit may, for example, identify a user profile associated with the user 102 and store the generated basketball characteristics of the user in the profile of the user. The control unit may store the user profile and associated data within the backboard 110. Alternatively, the control unit may access a server external to the backboard 110 via a network and store the generated basketball characteristics in the identified user profile in the server. The network may include, for example, one or more of Bluetooth, Wi-Fi, the Internet, cloud access, and cellular data networks (e.g., 4G and 5G capabilities).
[0051] In some embodiments, the control unit may generate recommendations for a specific user based on data about the user's basketball characteristics. The control unit may display the generated recommendations on a display screen of the backboard. In addition, the control unit may provide the generated recommendations to the client device 104 of the user 102. The control unit may also store the generated recommendations with the generated profile of the identified user (e.g., with the profile of the user 102).
[0052] In some embodiments, basketball system 107 may include a power source that powers the one or more components coupled to backboard 110. For example, the power source may power the one or more components within the second layer of backboard 110, a display within backboard 110, and a control unit. The power source may include, for example, a power plug that plugs into an outlet, a solar panel coupled to basketball system 107, or a rechargeable battery pack attached or coupled to basketball system 107.
[0053] In some embodiments, the backboard 110 may include one or more components that monitor the user 102 playing basketball 106 on the basketball court 108. In particular, the one or more components may include a plurality of sensors and a plurality of cameras positioned within the backboard 110. The plurality of sensors may include, for example, one or more vibration sensors, one or more travel sensors, one or more accelerometers, a light detection and ranging (LIDAR) sensor, one or more motion sensors, and one or more pressure sensors. The plurality of cameras may include a depth sensing camera (e.g., a true sensing depth camera), and a red, green, and blue (RGB) camera. The control data may receive data from each of the plurality of sensors and from each of the plurality of cameras to generate characteristics about the user 102 playing basketball.
[0054] Each of the sensors included in backboard 110 may be configured for a different purpose. For example, a vibration sensor may be configured to detect vibrations of basketball system 107 based on basketball 106 (i) bouncing off rim 112, (ii) bouncing off backboard 110, (iii) passing through rim 112, and (iv) dribbling by user 102 on basketball court 108. A travel sensor may be configured to determine whether a throw attempt by user 102 is successful. A successful throw attempt may indicate that basketball 106 passed through rim 112. An unsuccessful throw attempt may indicate that basketball 106 did not pass through rim 112. This will be further illustrated and described below.
[0055] The accelerometer may be configured to determine the position of the basketball relative to the backboard based on the accelerometer data and the vibration pattern. For example, when the user 102 throws the basketball 106 toward the basketball system in an unsuccessful throw, the basketball 106 may bounce off the basket 112 at a specific location. The location may be, for example, the front of the basket 112, the rear of the basket 112, or the side of the basket 112. In addition, the location may be, for example, a specific location on the front side of the backboard. Based on where the basketball 106 bounces off the backboard 110, the basketball system 107 may exhibit a specific vibration pattern. The specific vibration pattern may correspond to the speed or frequency of vibration of a component of the basketball system 107 (e.g., the backboard 110, the basket 112, the rod 113, or a combination of the above). The accelerometer may be configured to measure the speed or frequency of the vibration pattern and provide the detected vibration pattern to the control unit.
[0056] In some examples, the accelerometer may determine the location where the basketball 106 bounces off the basketball system 107 based on the determined vibration pattern. The accelerometer may compare the determined vibration pattern with one or more stored vibration patterns. Each stored vibration pattern may indicate the location where the basketball 106 hits the basketball system 107. In this case, the accelerometer may provide the vibration pattern and location of the basketball 106 bouncing off the basketball system 107 in a successful or unsuccessful throwing attempt to the control unit. In other examples, the control unit may use the accelerometer data in conjunction with a signal processing algorithm to determine a specific vibration pattern inherent in different impact locations of the basket. For example, the control unit may sample the accelerometer data, run the sampled data through one or more matched filters to try to identify the vibration pattern, and identify the vibration pattern that most closely matches the matched filter. In other examples, the control unit may apply other signal processing algorithms, such as low-pass filters, high-pass filters, sound modeling, waveform matching, fast Fourier transforms, acceleration signal matching, and matching between signals based on statistical properties.
[0057] In some embodiments, the LIDAR sensor may be configured to detect a throw attempt by the user 102 and to detect characteristics of the throw attempt of the basketball 106 performed by the user 102. For example, the LIDAR sensor may indicate the detection of the throw attempt based on the user 102's release point of the basketball 106 and the posture of the user 102 during the throw attempt. The LIDAR sensor may generate thousands of points per second at millimeter resolution for objects within its range, which may be, for example, 15 meters. The LIDAR sensor may detect when the basketball 106 is separated from the user 102 during the throw attempt. In addition, the LIDAR sensor may detect characteristics of the throw attempt of the basketball 106, which may include the angle and height of the basketball 106 during the throw attempt. The LIDAR sensor may indicate the angle of the basketball 106 relative to the basket 112 during the time increment of the trajectory of the throw attempt of the basketball 106. In addition, the LIDAR sensor may indicate the height of the basketball 106 relative to the basketball court 108 during the time increment of the trajectory of the throw attempt of the basketball 106. For example, the LIDAR sensor may indicate that the angle of basketball 106 is 45 degrees at time t0, 30 degrees at time t1, 22.5 degrees at t2, until the basketball reaches the basket 112 at -35 degrees at time tN. The LIDAR sensor may provide this data to the control unit as the user 102 plays the basketball. In some examples, the LIDAR sensor may measure and provide the shooting angle by measuring the angle between the basketball court at t0 and the tangent of the initial shooting arc. Typically, the throwing angle with a chance of making a shot is between 35 and 60 degrees, which the LIDAR sensor may measure.
[0058] In some embodiments, the motion sensor may be configured to detect one or more users 102 on the basketball court 108 within a close range of the backboard 110. For example, the motion sensor may detect movement on the basketball court 108 and may detect movement of the basketball 106. Based on the detection of the movement, in some examples, the motion sensor may indicate whether the movement corresponds to movement of the user 102 or movement of the basketball 106. In other examples, the motion sensor may provide the detection of the motion data to the control unit. As will be further described below, the control unit may collate the motion data, other sensor data, and image data to determine whether the movement corresponds to the user 102 or the basketball 106.
[0059] In some embodiments, the pressure sensor may be configured to detect and calculate the location where the basketball 106 impacts the backboard 110. Based on the location where the basketball 106 impacts the backboard 110, the control unit may calculate the ball trajectory of the throw attempt from the user 102. The location where the basketball 106 impacts the backboard 110 may be provided in position coordinates (e.g., Cartesian coordinates or polar coordinates) relative to the front side of the backboard 110. In other examples, the control unit may determine the location where the basketball 106 impacts the backboard 110 based on image data and sensor data from other sensors without using a pressure sensor. For example, the control unit may predict the location where the basketball 106 will impact on the backboard 110 based on the initial conditions of the basketball 106 being thrown by the user 102, the known gravity constant and air resistance, and the parabola mathematical equation. Data provided from the LIDAR sensor may help make this position determination.
[0060] As mentioned above, the multiple cameras may include a depth sensing camera and an RGB camera. The multiple cameras or imaging devices may each be configured to perform different functions. For example, backboard 110 may include one to three depth sensing cameras. Other examples are also possible. For example, backboard 110 may include more than three depth sensing cameras, such as eight depth sensing cameras or more. The depth sensing cameras may be configured to (i) detect user 102 on basketball court 108, (ii) track the movement of user 102, (iii) detect basketball 106 used by user 102 for a throwing attempt, (iv) track the movement of basketball 106, and (v) detect the posture of user 102. For example, the three depth sensing cameras may have overlapping fields of view (FOV) to include the widest field of view of basketball court 108.
[0061] For example, a depth sensing camera located in backboard 110 may be configured to detect user 102 on basketball court 108 based on software that detects movement of user 102 and identification of user 102. The depth sensing camera may detect and identify user 102 without using a machine learning model such as a convolutional neural network (CNN), which provides advantages over previous systems. In addition, the depth sensing camera may track the movement of user 102 as the user moves along basketball court 108. The user may, for example, move to a position behind the free throw line to attempt a shot, move to a position behind the three-point line to attempt a shot, or move to any other position on basketball court 108.
[0062] The depth sensing camera may track user 102 by generating position coordinates of user 102 along basketball court 108 and providing the position coordinates of user 102 to the control unit within the image data. Similarly, the depth sensing camera may track one or more basketballs 106 by generating position coordinates of basketball 106 along basketball court 108 and providing the position coordinates of basketball 106 to the control unit within the image data. For example, within each frame of image data recorded by the depth sensing camera, the depth sensing camera may draw boxes around the identified user 102 and basketball 106 and attach position coordinates to each box, respectively. In this way, the control unit may determine the position of user 102 and basketball 106 as they move from each frame of the image data. In addition, the depth sensing camera may track multiple basketballs and multiple users on basketball court 108. The depth sensing camera may also track and associate throw attempts and hits / misses for each of the multiple users.
[0063] The depth sensing camera may also track the posture of user 102. The posture of user 102 may correspond to the user's posture during dribbling, during a throw attempt, after a throw attempt (e.g., known as a follow-through, and moving to bounce basketball 106 if user 102's throw attempt is unsuccessful), to name a few. The posture detected by the depth sensing camera may indicate the location of the body in position coordinates. The depth sensing camera may provide the posture detection to a control unit, where further analysis of the posture detection data may be performed.
[0064] In some embodiments, backboard 110 may utilize a LIDAR sensor in place of a depth sensing camera. In addition to the functionality described above with respect to the LIDAR sensor, the LIDAR sensor may also be configured to perform the functionality described by the depth sensing camera. This functionality may include detecting the identification of one or more users and the movement on the basketball court. In addition, the LIDAR sensor may be configured to track the movement of the identified users over time. The LIDAR sensor may be configured to identify and monitor the movement of the users without the use of a machine learning model. Similarly, the LIDAR sensor may identify and track the movement of one or more basketballs on basketball court 108, similar to the tracking of the movement of one or more basketballs by a depth sensing camera.
[0065] In some embodiments, the backboard may include one or more RGB cameras configured to perform a specific function. Specifically, the specific function may include (i) recording image data of the field of view of the basketball court 108 and (ii) recording image data of the area below the backboard to detect when the throw attempt of the user 102 corresponds to a specific type of throw. For example, the backboard 110 may include one or two RGB cameras. The RGB camera can record the area of the basketball court 108 within the short range of the backboard 110 in real time. The area may include, for example, an area up to the half-court line on the basketball court 108, the length of the entire court when the court corresponds to a regular basketball court, a lane area, or some other area. In addition, the RGB camera can record image data of the area below the basket frame 112, which can be used when the user 102 is under the basket frame 112 or near the basket frame 112 to make a layup or perform another type of throw. The RGB camera can provide the recorded image data to the control unit for further analysis and use, which will be further described below.
[0066] See also Figure 1A , during stage (A), a user 102 may request to interact with the basketball system 107 by, for example, requesting to play a basketball game. In some embodiments, the user 102 may access a basketball application on their client device 104. The user 102 may log into the basketball application using authentication credentials such as a username and password, and may obtain applications such as games, training sessions, etc. provided by the basketball system 107. The user 102 may choose to interact with the basketball system 107 using a basketball 106. For example, as shown in the system 100, the user 102 may select a game to count the number of shots made within a predetermined amount of time.
[0067] In other embodiments, user 102 may communicate with basketball system 107 to request to play a basketball game using basketball system 107. User 102 may provide a verbal command, such as "Hi Huupe," to basketball system 107 or perform a waving motion to wake up basketball system 107. Components within backboard 110, such as a microphone and / or a depth sensing camera, may detect that the user is communicating with basketball system 107 and perform a function of detecting the user. For example, the microphone and the depth sensing camera may provide a verbal command, such as "Hi Huupe," and image data detecting user 102, respectively, to the control unit. The control unit may determine from the audio data and the image data (i) identification of user 102 and (ii) an indication that the control unit will provide user 102 with access to a list of games available for play.
[0068] The control unit may provide a list of games available for play to a display screen of backboard 110 or to a user's device 104. Additionally or alternatively, the control unit may provide the list of games available for play audibly to a speaker of backboard 110. User 102 may select which game to play by speaking to basketball system 107 or making a selection via the user's client device 104. For example, user 102 may indicate "Hey Huupe, play Shot Timer," and the control unit may recognize the voice command and determine that user 102 requested to play the Shot Timer game. Alternatively, user 102 may select which game to play by waving in front of basketball system 107. The display of backboard 110 may list the games available for play, and user 102 may raise their arm to act as a mouse on the display. The depth sensing camera may recognize via recorded image data that user 102 has raised their arm and provide the recorded image data to the control unit. The control unit may associate the arm raising during this request that user 102 wishes to select a game available for play from the list of games. The user 102 may move his arm in a continuous manner, and the depth sensing camera, control unit, and display may work together to present a mouse that moves in a manner similar to the user's 102 arm to enable selection of games available for play. The display may visually show the movement of the mouse over the game list to manipulate the movement as the user's 102 arm moves. In another example, each hand position of the user 102 may represent an area of the display screen. In this case, if the user 102 raises his hand vertically upward, the depth camera, control unit, and display will associate this movement by highlighting the button on the center top of the display screen. If the user 102 raises his right hand at a 45-degree angle (e.g., the upper right corner of the display screen), the depth camera, control unit, and display screen will associate this movement by highlighting the button on the upper right corner of the display screen. This process will be similar if the user places his right hand at a -45-degree angle (which will correspond to highlighting the button on the lower right corner of the display screen). If the user raises his left hand at a 45-degree angle, this will correspond to highlighting the button on the upper left corner of the display screen. If the user places his left hand at a -45-degree angle, this will correspond to highlighting the button on the lower left corner of the display screen. This process is similar for other hand positions around the display. In addition, when the user changes from an open hand to a clenched hand (e.g., an open palm to a clenched fist), the control unit can recognize that the user wishes to select or enter the selection. The selection can also be performed by highlighting for a predetermined amount of time (e.g., 3 seconds).
[0069] The user 102 can select a game to be played based on the arm movement by executing a selection. The selection can be, for example, a finger pointing, a verbal command, or some other gesture that indicates that the user has selected a particular game. In an example of the system 100, the user 102 can select a throw timer game, and in response, the control unit can initiate execution of the throw timer game. For example, the control unit can provide the throw timer 116 and the throw counter 114 to a display of the backboard in digital format. The control unit can indicate to the user 102 to start throwing the basketball 106 through the basket 112 once the throw timer 116 begins to count down.
[0070] In some embodiments, the control unit may adjust the shot counter 114 when the user 102 attempts a shot. For example, the user 102 may perform a shot attempt corresponding to throwing the basketball 106 through the basket 112. The depth sensing camera and sensor may generate image data and sensor data, respectively, and provide the image data and sensor data to the control unit, where the control unit may determine (i) whether the user attempted a shot and (ii) whether the shot attempt was successful. The control unit may then update the shot counter 114 in a heads-up display (HUD) on the display based on whether the user attempted a shot and whether the shot attempt was successful. For example, if the user 102 missed the first five shots, the control unit may display "0 / 5" for the shot counter on the HUD of the backboard.
[0071] In addition, the control unit can display the recorded screen from the RGB camera included in the backboard. For example, when the user 102 throws the basketball 106, the RGB camera can record a certain area of the basketball court 108 and provide the recorded image data to the control unit. The control unit can receive the recorded image data and display the recorded image data in real time on the display of the backboard 110 when the user 102 plays the game. In this way, the user 102 can visually see themselves on the backboard 110 when the user 102 plays the throw timer game. In addition, the control unit can digitally overlay the throw timer 116 and the throw counter 114 on the display of the backboard above the recorded image data. The recorded image data from the RGB camera can fill the entire display of the backboard 110. For example, the throw timer 116 can be positioned at the upper left corner of the display, and the throw counter can be positioned at the lower right corner of the display. Other positions are also possible.
[0072] During phase (B), the one or more sensors included in the backboard 110 acquire sensor data 118 corresponding to the user 102 on the basketball court 108. As previously mentioned, the sensor data 118 may include data from travel sensors, accelerometers, LIDAR sensors, and motion sensors. For the particular throw attempt shown in the system 100, the throw attempt by the user 102 was successful. The travel sensor may indicate that the throw attempt by the user 102 was successful because the basketball 106 passed through the basket 112. For example, the backboard 110 may include an array of 2 or more laser travel sensors aligned vertically in coincidence with the hoop system. If a series of laser travel sensors are passed in sequence from high to low, the control unit may determine that the shot attempt was successful. Alternatively, if the laser travel sensors are not passed in sequence from high to low, the control unit may determine that the shot attempt was unsuccessful.
[0073] The accelerometer may indicate small vibration patterns as the path of the basketball 106 through the rim 112 results in a hollow ball, or a successful throw that does not strike the rim 112 or backboard 110. Alternatively, if the basketball 106 does strike the backboard 110 or rim 112 through a successful throw attempt, the accelerometer may record specific vibration patterns to indicate where the basketball 106 landed. The LIDAR sensor may detect the throwing attempt of the user 102 and the characteristics of the throwing attempt of the user 102, such as the angle and height of the basketball 106 during the throwing attempt. The motion sensor may detect the user 102 on the basketball court 108. In addition, the motion sensor may detect the movement of the user 102 and the movement of the basketball 106.
[0074] In some embodiments, the backboard 110 may periodically acquire sensor data 118 from multiple sensors. For example, the sensor may be configured to acquire sensor data 118 every 2 seconds, 5 seconds, or 10 seconds. In other examples, the sensor may be configured to acquire sensor data 118 based on the type of game that the user 102 is playing. For example, if the user 102 is playing a throw timer game, the sensor may be configured to acquire sensor data 118 at a more frequent period to ensure that all throw attempts are taken into account, whether successful or not. In addition, if the user 102 notifies the basketball system 107 that the count of throw attempts is incorrect and the count of hits and misses (and vice versa) is incorrect, the user 102 may adjust the sensitivity or frequency of sensor acquisition via the client device 104 until the count is accurate. In another example, if the user 102 is playing a game in which the user needs to throw from a set position on the basketball court 108, the sensor may be configured to acquire sensor data 118 less frequently because between each throw, the user 102 must bounce and move the ball to a different set position. To reduce the processing that the sensor and control unit must perform during games where the user is aiming for fewer throws, but more precisely this may reduce the overall complexity and processing performed by the sensor and control unit.
[0075] During stage (C), the one or more cameras included in backboard 110 acquire image data 122 corresponding to user 102 on basketball court 108. As previously mentioned, image data 122 may include image data from a depth sensing camera and an RGB camera. For example, image data 122 may include images or videos from each of the cameras. For example, as shown in system 100, user 102's throwing attempt was successful because it passed through basket 112. The depth sensing camera may generate (i) detection data of user 102 on basketball court 108, (ii) movement data of user 102, (iii) detection data of basketball 106 used by user 102, (iv) tracking movement data of basketball 106, and (v) detection data of the user's posture.
[0076] As previously described, the depth sensing camera may generate and track each of the different detection and movement data of the user 102 and the basketball 106. The depth sensing camera may continuously record image data 122 monitoring the user 102 on the basketball court 108, and generate this detection / movement data for each frame of the recorded image data 122. In some examples, the depth sensing camera may provide this image data 122 and detection / movement to the control unit in real time. In addition, the RGB camera may record the user 102 on the basketball court 108, and may provide the recorded image data 122 to the control unit. The control unit may receive the sensor data 118 and the image data 122 to generate characteristics of the user and the throwing attempt.
[0077] During stage (D), the control unit may receive image data 122 from the depth sensing and RGB cameras and provide the image data 122 to a trained machine learning model. The control unit may train the machine learning model to perform a variety of functions. The functions may include (i) classifying or identifying each user on the court 108 as a unique and persistent user, (ii) identifying a user's throw, (iii) generating characteristics of a user on the basketball court 108, and (iv) generating characteristics of a user's throw attempt.
[0078] For example, the trained machine learning model may correspond to a convolutional neural network (CNN). The control unit may train the machine learning model using different image data of successful throwing attempts and unsuccessful throwing attempts from various positions on different courts. The control unit may also train the machine learning model using different image data of throwing attempts of professional basketball players to understand the ideal throwing attempt to assist other users interacting with the basketball system 107. The control unit may provide image data about these professional athletes dribbling, shooting from various positions on the court, and moving while dribbling. In another example, RGB and depth sensing cameras may be used to determine the posture of the user's throwing attempt, and the CNN may be trained based on the identified posture according to the ideal posture during the throwing attempt (e.g., the ideal posture of a professional athlete or other individual) or which posture recognition provides the best results.
[0079] In some embodiments, the control unit may also train the machine learning model to identify users interacting with the basketball system 107. For example, when a user, such as user 102, attempts to initially use the basketball system 107, the application on the client device 104 may request that the user enter their credentials and take a picture or selfie of themselves. This will provide the control unit with an initial image to train the machine learning model to detect the user 102. The control unit may then instruct the user 102 to play a game with the basketball system 107 by making a throw attempt. The control unit may acquire image data 122 of the user 102 and train the machine learning model using the newly acquired image data 122. The control unit may provide an indication on the display that the basketball system 107 is in a learning mode to learn the user 102. Then, at a later point in time, and once the machine learning model is sufficiently trained, the control unit may apply the trained machine learning model to (i) identify the user 102 as a unique user, (ii) identify the user's throws, (iii) generate characteristics of the user on the basketball court 108, and (iv) generate characteristics of the user's throw attempts.
[0080] In some embodiments, the trained machine learning model may output labeled data 126. Labeled data 126 may indicate an identification of user 102 and a location of user 102 relative to backboard 110 on a frame of image data 122. For example, labeled data 126 may indicate that user 102 is "Bob," and (X, Y, Z) coordinates indicating where user 102 is located on basketball court 108 relative to basketball system 107. Labeled data may also indicate an angle of the trajectory of basketball 106 for a particular frame of image data, and an indication of whether basketball 106 passed through basket rim 112. For example, the angle of the trajectory of basketball 106 may indicate that the basketball was at 101 degrees relative to the basketball system at a particular point in time. In addition, labeled data 126 may indicate that basketball 106 did pass through basket rim 112, e.g., successfully. In some examples, labeled data 126 may include statistics or percentages indicating the probability that user 102 is "Bob," the probability of where user 102 appears to be located, the probability of the angle of the basketball's trajectory, and the probability of whether the basketball passed through hoop 112. These probabilities may, for example, be in the range of 0-100% or 0-1.
[0081] If multiple users 102 are interacting with the basketball system 107, the control unit can simultaneously identify and track each of the users 102. For example, the control unit can receive sensor data from each of the sensors in the backboard 110 that monitor the characteristics of each of the different users. The control unit can also receive image data from each of the cameras in the backboard 110 and provide the image data to the trained machine learning model. In response, the machine learning model can identify each of the users on the basketball court 108, and can track each of the users on the basketball court 108 and their corresponding movements with and / or without a basketball. The labeled data 126 output by the trained machine learning model may include a frame of image data with a label for each of the users in the frame and the characteristics corresponding to each of the users, as described above. For example, the trained machine learning model can track each of the users interacting with the basketball system 107 based on their jerseys, jersey numbers, specific clothing types, and physical features. In this case, the trained machine learning model can also associate the hits and misses of each throw attempt from each of the users on the basketball court.
[0082] In some embodiments, the trained machine learning model can also be used to generate real-time predictions of the user's throws. For example, the trained machine learning model can generate real-time predictions for each throw as the basketball moves toward the basket 112. For example, based on previous sensor data, image data, and current input data from sensors and cameras, the basketball system 107 can initially predict the probability of a throw being successful when the ball is ten feet from the basket 112, such as a 70.0% probability of a successful throw. The trained machine learning model can adjust the probability after the basketball 106 hits the basket 112, such as a 55.0% probability of a throw being successful. The trained machine learning model can be continuously updated / retrained using the image data and sensor data captured from the basketball system 107 using the predictions and results of the throw attempts.
[0083] During stage (E), the control unit may analyze the received sensor data 118 to determine whether the throw attempt was successful. For example, the control unit may first determine from the LIDAR sensor whether a throw attempt was detected. If the control unit determines that a throw attempt was detected, the control unit may analyze data from a travel sensor located on the backboard 110 behind the basket to determine whether the basketball 106 passed through the basket 112. If the throw did not pass through the basket 112, the control unit may analyze data provided by the accelerometer to determine the location of the basketball 106 relative to the backboard based on the vibration pattern. If the vibration pattern is not detected and an unsuccessful throw attempt is detected, the control unit may determine that the user did not hit the basket 112 and the backboard 110 during the unsuccessful throw attempt, such as referred to as an "air ball". In another example, if the control unit determines that the throw did not pass through the basket, and a vibration pattern is detected, the control unit may determine whether the basketball 106 bounced off the front side of the backboard 110, the bottom side of the backboard 110, the top portion of the backboard, the side portion of the backboard, or a location on the basket 112 (or a combination of each). The control unit can analyze the sensor data provided by the motion sensor and the LIDAR sensor to determine the location of the throw attempt and the trajectory or arc of the throw attempt. If the control unit determines that the throw did pass through the basket, the control unit can determine a vibration pattern to indicate how the basketball passed through the basket 112, such as leaving the backboard 110, leaving the basket, or a hollow ball.
[0084] During stage (F), the control unit may perform analysis on the labeled data 126 provided by the trained machine learning model 124 and the data generated by the analysis of the sensor data 118 during stage (E). Based on the labeled data 126 and the analysis performed on the sensor data 118, the control unit may generate output data 130. The output data 130 may include the user's location 132, the user's identification 134, the ball trajectory angle 136, the throw attempt hit / miss 138, and the improvement suggestion 140. For example, the control unit may generate the output data and store the data in a digital data type such as a structure, class, or other. For example, as shown in the system 100, the user's location 132 may indicate the X, Y, Z coordinates relative to the basketball system 107 and the basketball court 108 - "12.00, 1.01, 0.00", measured in feet. The user's identification 134 may correspond to a name or another identifier that identifies the user 102, such as "Bob". Ball trajectory angle 136 may correspond to an angle on the flight time trajectory, such as 5 degrees at t9, where time may be measured, for example, in seconds or milliseconds. For example, throw attempt hit / miss 138 may correspond to a "hit." Additionally, the control unit may generate a recommendation 140 based on the analysis performed.
[0085] For example, the control unit may generate suggestions to be provided to the user 102 for improving the user's subsequent throwing attempts based on the user's 102 current throwing attempt and previous throwing attempts. The suggestions may correspond to improvements for: (i) the user's posture, (ii) the user's arm angle, (iii) the release point of the basketball during the throwing attempt, and (iv) the trajectory of the basketball during the throwing attempt. For example, the control unit may compare the posture of the user 102 during the throwing attempt with the stored image data of the posture of the professional athlete during the throwing attempt. The user 102 may indicate through the application of the client device 104 that he or she wishes to throw like a particular professional athlete. The control unit may display the image data of the user 102 shooting and the image data of the professional athlete shooting side by side to the display of the client device 104 and / or the backboard 110. In this way, the user may try to practice his or her shooting posture to match the shooting posture of the professional athlete shown on the display. The control unit may determine the proximity of the shooting posture by comparison, and, for example, provide a percentage to the user to indicate how close their posture is to the posture of the professional athlete during the throwing attempt. In this way, user 102 may be able to improve their posture during throwing attempts over time.
[0086] The control unit may also analyze the arm angle of the user 102 or the limbs of the user 102 during the throwing attempt. For example, the arm angle may be angled outward, inward, or some angle in between during the throwing attempt. The control unit may indicate to the user 102 to adjust its arm angle for subsequent throwing to better align with the arm angle of the professional. Alternatively, the control unit may indicate to the user 102 to adjust its arm angle to improve the probability of success of subsequent throwing attempts. The instructions may be displayed on the display of the backboard 110 or on the client device 104 as image data (e.g., video) or in another form. Similarly, the control unit may analyze the throwing point of the basketball 106 during the user's throwing attempt and the trajectory of the basketball 106 during the user's throwing attempt, and provide suggestions to improve these based on the throwing point and trajectory of another player, and improve the probability of a successful throw.
[0087] In some embodiments, the control unit may generate a profile of the user 102. The profile may include an identification of the user 102 (e.g., the user name is Bob), credentials of the user 102, one or more client devices associated with the user 102 (e.g., client device 104), and characteristics of the user 102 during a throwing attempt. For example, the characteristics of the user 102 during a throwing attempt may include a classification of the user and the position of the user on the basketball court 108 relative to the backboard 110 during the throwing attempt. In addition, the characteristics of the user 102 may include the user's wingspan, height, hand size, and speed. The characteristics may also include previous throwing attempts of the user 102, such as characteristics of the travel of the basketball 106 during the throwing attempt, such as height, angle, trajectory point, vertical jump of the user, and release point at different times. The control unit may also store in the profile whether the result of a previous shot attempt was a hit or a miss, and if a hit, what the type of shot was, such as a shot that bounced off the backboard 110, a shot that bounced off the rim 112, or a shot that whizzed through the net of the basketball system 107. The control unit may also store the sensor data 118, the image data 122, the labeled data 126, and the output data 130 with the profile.
[0088] In some embodiments, the control unit may also store the game data associated with the game performed by the user 102 or the game data associated with the game performed by the user 102 in the profile corresponding to the user 102. The type of the game performed may include, for example, a training course mode, a local head-to-head confrontation, a live broadcast mode, and a global competition mode. For example, the game performance may include what the game performed is, the date and time of the game, the number of players in the game, the identification of each player participating in the game, the final score of the game, each shot and miss of each player during the game, the position of each shot and miss of each player on the basketball court, and the time when each shot and miss of each player occurs in both absolute time and relative time (relative to the start of the game). The control unit may store the game data as a tuple, a structure, a class, or some other computer format. If multiple users are playing a single game, the control unit may store the game data of each of the users in its corresponding profile of the single game.
[0089] The control unit may store a profile for user 102 on an external server. The control unit may access and retrieve the profile of user 102 when the profile is updated based on newly received sensor and image data. For example, the control unit may identify the user profile based on facial recognition from received image data, user 102 entering a username and password, iris recognition from received image data, matching fingerprints, or some other suitable authentication or identification method. Once the control unit identifies a user profile corresponding to user 102, the control unit may access the corresponding profile from the external server and update its content. Once the update of the profile for user 102 is completed, the control unit may update the external server with the revised profile.
[0090] During stage (G), the control unit may provide output data 130 for review by user 102. The control unit may provide output data 130 for review by user 102 in a variety of ways. In one way, the control unit may provide output data 130 to client device 104 for review by user 102. In this way, user 102 may review output data 130 to view the most recent throw attempt for analysis, such as ball trajectory angle 136, suggestion 140, and user's position 132. In some examples, the control unit may provide output data 130 to a display of backboard 110, and the user may interact with backboard 110 via verbal or hand-wave commands to review output data 130. In this way, user 102 may attempt to improve their basketball skills for subsequent throw attempts based on output data 130 characterized by basketball system 107 for the most recent throw attempt.
[0091] In some embodiments, the control unit may also provide the generated user profile to the client device 104 for review by the user 102. The user 102 may access an application on their client device 104 to view the profile generated for them by the control unit. In particular, the user 102 may view data from previous throw attempts to analyze how the user's throw attempts progressed in the profile. For example, the user 102 may view the sensor data 118, image data 122, tagged data 126, and output data 130 for each previous throw attempt on their client device 104. In some examples, the control unit may display the profile on a display of the backboard 110, and the user may interact with the profile via verbal or hand-wave commands.
[0092] In some embodiments, the control unit may provide encouraging suggestions to the user 102 while the user 102 is playing the game. For example, if the control unit notifies the user 102 that the shot-to-shot attempt ratio is low, the control unit may provide an audible message 146 via the speaker of the backboard 110. The audible message 146 may be heard by the user 102, such as, "6 minutes left! Don't stop shooting." In another example, if the control unit notifies the user 102 that the shot attempt was successful, the control unit may provide another audible message 144 via the speaker of the backboard 110, which may say, "Great shot!" The control unit may also provide other audible messages to the speaker of the backboard 110 or to the client device 104. Each of these encouraging messages may also be stored with the profile of the user 102.
[0093] The user 102 may continue to play the shot timer game until the time displayed on the shot timer 116 has elapsed. The user 102 may then select another game available for play using the basketball system 107, review the display of the backboard 110 or the analysis generated by the basketball system 107 on the client device 104, or turn off the basketball system 107. In some examples, the user may turn the basketball system 107 on and off via an application on the client device 104.
[0094] In some embodiments, a micro backboard may be provided in the system 100. The micro backboard may include components and functionality similar to those described with respect to the backboard 110 of the system 100. The micro backboard may not include the rod 113, but may instead be placed in various locations in a residence, a company building, or other area. For example, a user may be able to mount the micro backboard on a wall of a residence or hang the micro backboard above a specific side of a door or cabinet. The micro backboard may include multiple anchor points on the back side of the backboard for attachment to various attachment points. These anchor points may be attached by, for example, ropes, hooks, screws, fasteners, and other attachments. The micro backboard may include a basket frame with a net connected to its front side. In other examples, a user may be able to mount the micro backboard on a surface in a garage, a basement, or other areas within a residence or company property. In other examples, the micro backboard may include a small rod with a base for support that can be placed in any position, and a user may throw a smaller size basketball at its basket frame. These locations may include, for example, bedrooms, basements, kitchens, office spaces, living rooms, and other places.
[0095] In some implementations, the micro backboard may include fewer components than backboard 110 while maintaining similar functionality. The number of sensors within the micro backboard may be reduced because the size of the micro backboard may be significantly smaller than backboard 110. For example, the micro backboard may include two depth sensing cameras, rather than three to eight, with overlapping fields of view for viewing the area where the user is shooting. The basket of the micro backboard is also significantly smaller than the basket of backboard 110. Due to the size of the basket of the micro backboard, the basketball used with the micro backboard is also significantly smaller.
[0096] The micro-backboard may include a similar number of games available for play as backboard 110, and include the ability to connect to other micro-backboards located in other geographic regions via a network. In some cases, the micro-backboard may include the ability to play basketball games with users at other regulation-sized backboards (e.g., such as backboard 110) in different geographic regions via a network. Users may interact with the micro-backboard by communicating with the micro-backboard, either verbally, with hand gestures, or by interacting with a smart application of their client device that communicates with the micro-backboard, similar to how users would interact with backboard 110.
[0097] Figure 1B 1 is a block diagram illustrating an example of a system 101 for backboard. System 101 illustrates in detail the structure of backboard 110. This backboard 110 illustrated in system 101 is similar to backboard 110 illustrated in system 100.
[0098] System 101 includes backboard 110, network 150, client device 104, and external server 170. Backboard 110 includes front layer 164, second layer 166, and display screen 168. Front layer 164, second layer 166, and display screen 168 are positioned in a longitudinal, layered, or stacked manner, but may be arranged in any suitable configuration. System 101 includes fasteners 158A and 158B, which may hold the various layers of backboard 110 together in place. In some cases, the number of layers of backboard 110 may be less than three layers, as will be further illustrated and described below. Backboard 110 is IP67 waterproof regulated and may, for example, weigh approximately 300 pounds.
[0099] Fasteners 158A and 158B may be shock absorbing fasteners that couple the different layers of backboard 110. Fasteners 158A and 158B may be any suitable device that absorbs and / or minimizes the transfer of impact forces and / or vibrations from basket frame 112 and / or front layer 164 to display screen 168. In one example, fasteners 158A and 158B may include one or more brackets and springs, screws, rivets, bolts, or other suitable mechanisms. In another example, fasteners 158A and 158B may include one or more adhesives, sealants, or other suitable mechanisms.
[0100] In some embodiments, front layer 164 of backboard 110 can protect components within backboard 110. For example, front layer 164 can be tempered glass covered with a protective coating having a translucent or transparent material (e.g., an anti-reflective coating), or both. Front layer 164 can also be configured to enable a user (e.g., user 102) interacting with backboard 110 to view display screen 168 in a third layer of backboard 110. Front layer 164 can also be connected to basket frame 112 at a bottom portion of front layer 164. Second layer 166 of backboard 110 can include a small area between front layer 164 and display screen 168 that includes the one or more components of backboard 110.
[0101] For example, second layer 166 may include camera sensing system 156, stream camera 157, sensor 154, and speakers 160A and 160B. Camera sensing system 156 may include one or more depth sensing cameras that view basketball court 108 and are positioned at the top portion and center of second layer 166. Stream camera 157 may include one or more RGB cameras that view basketball court 108 and may be positioned below camera sensing system 156. In some examples, second layer 166 may be a compressible material such as rubber or foam padding.
[0102] Sensors 154 may be positioned in a variety of locations within the second layer. For example, a LIDAR may be positioned within camera sensing system 156. A motion sensor may be positioned within camera sensing system 156. Travel sensor 162 may be a laser travel sensor positioned vertically at the bottom of backboard 110 on front layer 164 behind basket rim 112. Speakers 160A and 160B may be positioned within second layer 166 of backboard 110. Speaker 160A may be capable of playing music and / or providing audible feedback to user 102. Accelerometers may be positioned within sensors 154 of second layer 166. Microphones may be positioned within sensors 154 of second layer 166.
[0103] As previously mentioned, sensors 154, including travel sensors 162, can sense players, balls, and / or forces applied to basketball system 107. Sensors 154 generate data that is processed and analyzed by control unit 148. Each of sensors 154 can be configured in a particular manner to appropriately detect, acquire, and generate sensor data for a user playing basketball. Sensors 154 can be coupled to front layer 164, display screen 168, poles 113, or second layer 166. In some examples, sensors 154 can also be remote from basketball system 107 or disconnected from basketball system 107. For example, sensors 154 can be coupled to existing court lighting systems and / or auxiliary support structures along the sides or ends of basketball court 108.
[0104] The control unit 148 may be positioned behind the display screen 168 of the backboard 110. Each of the sensors, cameras, speakers, and microphones may be connected to the control unit 148 in a bidirectional manner. The control unit 148 may communicate with an external server 170 and the client device 104 via a network 150. The network 150 may be, for example, the Internet, Wi-Fi, Bluetooth, Ethernet, or some other form of wireless or wired connection.
[0105] Display screen 168 may display shot timer 116 and shot counter 114. Shot timer 116 and shot counter 114 may be visible to user 102 through front layer 164 and second layer 166. User 102 may also view graphical data displayed by the control unit on display screen 168, such as real-time video footage of user 102 playing basketball, or video footage of another user playing basketball from another connected control unit. The control unit may digitally overlay shot timer 116 and shot counter 114 over the video footage provided to display screen 168. In addition, a square of backboard 110 may be generated and displayed by display screen 168.
[0106] Display screen 168 can visually display information, signs, videos and / or pictures. Display screen 168 can have a brightness of 1000 nits or more. Display screen 168 can be any suitable display panel such as an LED or LCD display screen. For example, display screen 168 can be a smart TV. Display screen 168 can also be a screen for projecting information, signs, videos and / or pictures. For example, a projector can project information onto a display screen. In another example, display screen 168 can include a short throw projector for providing information, signs, videos and / or pictures as a display. The client device can stream any image data to display screen 168 for user viewing.
[0107] System 101 also includes power source 152. As previously mentioned, power source 152 can provide power to one or more components within second layer 166 of backboard 110, a display within backboard 110, and a control unit. Power source 152 can include, for example, a power plug that plugs into an outlet, a solar panel coupled to basketball system 107, or a rechargeable battery pack attached or coupled to basketball system 107. Power source 152 can be located on the rear side of display screen 168.
[0108] Figure 1C 1 is another block diagram illustrating an example of a backboard system 103. System 103 includes similar components to backboard 110 shown in system 101. However, backboard 110 in system 103 includes two layers, such as front layer 164 and display screen 167. The various components of backboard 110 may be positioned and configured to operate in three layers and two layers, respectively, as shown and described in systems 101 and 103.
[0109] In system 103, front layer 164 and display screen 168 may be spaced apart by a predetermined amount to reduce impact during a user's throwing attempt. For example, front layer 164 and display screen 168 may be spaced apart by 0.5-1.0 inches so that when a basketball strikes front layer 164, front layer 164 will not contact display screen 168 as front layer 164 vibrates or flexes.
[0110] In another example, the display screen 168 is tightly abutted against the front layer 164. In this example, the front layer 164 is formed of substantially rigid plexiglass or other suitable material, and there is minimal or no space between the front layer 164 and the display screen 168. As such, when a basketball contacts the front layer 164, the front layer 164 does not move relative to the display screen 168. Therefore, the display screen 168 is not damaged when the basketball contacts the front layer 164. In this example, the rod 113 or support member is embedded in the ground, and the connection between the rod 113 and the display screen 168 and / or the front layer 164 is rigid so that the force applied to the front layer 164 by the basketball is transmitted to the ground, thereby minimizing vibration and tremor of the display screen 168.
[0111] Figure 2A 2 is a block diagram illustrating an example of video analysis 200 for detecting a basketball passing through a basket. In particular, video analysis 200 illustrates the process performed by a control unit when determining that a throw attempt was successful (e.g., the basketball passed through the basket of a backboard). Video analysis 200 illustrates similar components to systems 100, 101, and 103, and similar components need not be described again here.
[0112] Video analysis 200 includes a basketball 202 thrown by a user, a backboard 204, and a virtual zone. The virtual zone includes a virtual inner cone 208 and a virtual outer cone 210. The control unit may apply the virtual inner cone 208 and the virtual outer cone 210 to the received image data from the depth sensing camera.
[0113] The virtual inner cone 208 and the virtual outer cone 210 may include one or more characteristics placed above the basket. For example, the virtual inner cone 208 may include an inner radius 206 that matches the radius of the basket. The virtual inner cone 208 includes a base that matches the basket. The virtual inner cone 208 also has a specific height extending from the basket. For example, the height of the virtual inner cone 208 may be 1 foot. Similarly, the virtual outer cone 210 may include an outer radius 212 that includes the radius of the basket and a distance greater than the radius of the basket. For example, the outer radius 212 may correspond to 2.5 feet or another distance. The virtual outer cone 210 may include an extruded profile of the size of the virtual inner cone 208 so that the two virtual cones do not overlap each other. The virtual outer cone 210 also has a specific height extending from the basket. For example, the height of the virtual outer cone 210 may be 4 feet or another height. The control unit may use the characteristics of the virtual cone displaced above the basket to determine whether the throwing attempt resulted in a successful throw or an unsuccessful throw.
[0114] For example, as shown in video analysis 200, a user performs a throwing attempt by throwing basketball 202 toward backboard 204. The control unit may deem the throwing attempt successful when: (i) basketball 202 enters virtual outer cone 210, then (ii) basketball 202 exits virtual outer cone 210, then (iii) basketball 202 enters virtual inner cone 208, and finally (iv) basketball 202 exits virtual inner cone 208 (e.g.) into a basket of the basketball system only via the base of virtual inner cone 208. At this point, the control unit may deem the throwing attempt successful because basketball 202 passed through the basket of the basketball system. The control unit may perform these video analyses during analysis 128 of system 100 to determine whether the throwing attempt was successful.
[0115] Figure 2B 2 is a block diagram illustrating an example of video analysis 201 for detecting that a basketball does not pass through a basket. Video analysis 201 illustrates similar components to video analysis 200, which will not be described again herein. Video analysis 201 may illustrate a process performed by a control unit when it is determined that a throw attempt was unsuccessful (e.g., the basketball did not pass through the basket of the backboard). In addition, video analysis 201 illustrates similar components to systems 100, 101, and 102, and similar components need not be described again.
[0116] As shown in video analysis 201, the user performs a throw attempt by throwing basketball 202 toward backboard 204. The control unit may consider the throw attempt unsuccessful in a variety of ways. In one example, if basketball 202 does not enter virtual outer cone 210 or virtual inner cone 208, the control unit may consider the throw attempt unsuccessful. In another example, basketball 202 may enter virtual outer cone 210 and then leave virtual outer cone 210 without entering virtual inner cone 208, then the control unit may consider the throw attempt unsuccessful. In another example, basketball 202 may enter virtual outer cone 210, then basketball 202 leaves virtual outer cone 210, and finally basketball 202 does not leave virtual inner cone 208 via substrate, then the control unit considers the throw attempt unsuccessful. The control unit may perform these video analyses during analysis 128 of system 100 to determine whether the throw attempt is unsuccessful.
[0117] Figure 3 is a block diagram showing an example of a system 300 in which two users play basketball using a connected basketball system. As previously mentioned, for example Figure 1A A basketball system such as the basketball system 107 of FIG. 300 enables a user to play various basketball games. The various basketball games may include, for example, a training course mode, a head-to-head matchup mode, a global competition mode, and a live broadcast mode, to name a few. System 300 shows an example of a head-to-head matchup mode between two different basketball systems.
[0118] System 300 shows basketball systems 302-1 and 302-N participating in a head-to-head competition. Basketball system 302-1 and basketball system 302-N communicate with each other via their respective control units via network 301. Network 301 may be, for example, the Internet, Wi-Fi, or another form of connection.
[0119] At basketball system 302-1, user 304 may register with basketball system 302-1 for a head-to-head game using his client device 314. User 304 may play on court 310 using basketball 312. Similarly, at basketball system 302-N, another user 306 may register with basketball system 302-N for a head-to-head game using his client device 316. User 306 may play on court 308 using basketball 315.
[0120] In some embodiments, user 304 may transmit a request to participate in a head-to-head competition to basketball system 302-1. User 304 may provide an indication or request to participate in a head-to-head competition game to basketball system 302-1 via an application on client device 314. The indication or request may also indicate whether user 304 wishes to play against a friend or a random match. If user 304 selects a friend, basketball system 302-1 may attempt to send the request to a basketball system associated with the friend, such as basketball system 302-N. Alternatively, if user 304 selects a random match, basketball system 302-1 may retrieve each of the other basketball systems currently online and currently listed in the queue that are also requesting to participate in a head-to-head competition, and may randomly select one of those basketball systems in the queue to play against user 304 from basketball system 302-1. Alternatively, basketball system 302-1 may select the top basketball system in the queue to play against user 304.
[0121] Similarly, user 306 may provide an indication or request to participate in a head-to-head competition game to basketball system 302-N via an application on client device 316. Basketball system 302-N may determine whether user 306 requests a friend match or a random match. Based on the request, basketball system 302-N may communicate to another basketball system to set up a head-to-head competition. For example, the control unit of basketball system 302-1 may transmit request 318 to the control unit of basketball system 302-N. The control unit of basketball system 302-N may respond to request 318 with an acceptance in response to receiving request 318 to set up the head-to-head competition.
[0122] Basketball system 302-1 and basketball system 302-N may be located in different geographic areas. For example, basketball system 302-1 may be located in a gymnasium in New York City, New York, and basketball system 302-N may be located in a gymnasium in Geneva, Switzerland. In other examples, basketball system 302-1 may be located at a house, and basketball system 302-N may be located at a neighbor's house. In other examples, basketball system 302-1 and basketball system 302-N may be located at opposite ends of the same basketball court. As long as the basketball systems can connect to a network connection (such as the Internet), the basketball systems can be connected to each other. In other embodiments, the basketball systems do not require an Internet connection to communicate with each other. In fact, the basketball systems can communicate via a cellular connection with a specific throughput and constant network connectivity.
[0123] In response to basketball systems 302-1 and 302-N accepting a head-to-head competition, the head-to-head module is executed at the two basketball systems. For example, the backboard of basketball system 302-1 displays the recorded real-time video stream of user 306 playing basketball. Similarly, the backboard of basketball system 302-N displays the recorded real-time video stream of user 304 playing basketball. One or more RGB cameras at basketball system 302-1 record image data of court 310 where user 304 plays, and provide the recorded image data to the control unit of basketball system 302-1. The control unit of basketball system 302-1 transmits the recorded image data to the control unit of basketball system 302-N via network 301 to be displayed on the display screen of basketball system 302-N. Similarly, one or more RGB cameras at basketball system 302-N record image data of court 308 where user 306 plays, and provide the recorded image data to the control unit of basketball system 302-N. The control unit of basketball system 302-N transmits the recorded image data to the control unit of basketball system 302-1 via network 301 for display on the display screen of basketball system 302-1. This process occurs simultaneously, so user 304 can see user 306 playing on the backboard of basketball system 302-1, and so user 306 can see user 304 playing on the backboard of basketball system 302-N.
[0124] Additionally, the two users may talk to each other during the head-to-head competition. For example, user 306 may say "I'm about to beat you, John!" The microphone of basketball system 302-N may pick up the audible message, provide the audible message to the control unit, and the control unit may transmit the audible message to the control unit of basketball system 302-1. Here, the control unit of basketball system 302-1 may play the audible message via the speaker of basketball system 302-1, where user 304 may hear the message "I'm about to beat you, John!"
[0125] User 304 may respond by speaking a verbal message “Soon!” The microphone of basketball system 302-1 may pick up the verbal message and transmit the message to the control unit of basketball system 302-1. The control unit of basketball system 302-1 may then transmit the verbal message to the control unit of basketball system 302-N, where the control unit of basketball system 302-N receives the verbal message and provides it for display by a speaker of basketball system 302-N, such as the speaker playing “Soon!”
[0126] When a head-to-head competitive game begins with a "3-point contest," "free throw contest," or "most shots within a predetermined time," basketball system 302-1 and basketball system 302-N will begin a visually and / or audibly played countdown, such as "3, 2, 1, go!" Both users 304 and 306 begin throwing their respective basketballs 312 and 315 toward their basketball systems 302-1 and 302-N, respectively. Both basketball systems 302-1 and 302-N may use their respective cameras and sensors to monitor each user's throws, throw attempts, using the process described above. The display screen of basketball system 302-1 may display both the ratio of hits to throw attempts for user 304 (e.g., 6 / 20), and the ratio of hits to throw attempts for user 306 (e.g., 5 / 10). Similarly, the display screen of basketball system 302-N may display similar ratios. The control units of both can track hits and throw attempts and provide this information to the other control unit along with the image data recorded by the RGB camera. In this way, the control unit of basketball system 302-N can display the ratio of hits to throw attempts for user 304 on the display screen of basketball system 302-N, and the control unit of basketball system 302-1 can display the ratio of hits to throw attempts for user 306 on the display screen of basketball system 302-1. Any updates on the throw attempts and hits for both users are provided to both control units via network 301 so their displays can be updated separately.
[0127] If this is a timed contest, the display screens of both basketball systems 302-1 and 302-N may display a timer. The timers between the two display screens will be synchronized to ensure that both users have the same amount of time to play the head-to-head contest. Once the time on the timer has elapsed, the winner with the highest ratio of field goal attempts to field goal attempts is deemed the winner of the contest.
[0128] Similar to stage (G) of system 100, the control unit of basketball system 302-1 may provide output data for each attempted throw to client device 314 for review by user 304. The control unit of basketball system 302-N may provide output data for each attempted throw to client device 316 for review by user 306. The control unit of basketball system 302-N may also provide output data to the control unit of basketball system 302-1 and / or client device 314 for user 304 to review the results of its opponent. The control unit of basketball system 302-1 may also provide output data to the control unit of basketball system 302-N and / or client device 316 for user 306 to review the results of its opponent. Similarly, the two control units may associate the image data, sensor data, and output data of each throw attempt with the profile of the corresponding user. For example, the control unit of basketball system 302-1 may store the image data, sensor data, and output data of each throw attempt of user 304 in its profile. The control unit of basketball system 302-N may execute the same stored program for user 306 and associated profile.
[0129] If user 306 and user 304 play basketball at the same basketball system (e.g., basketball system 302-1), a process similar to that described above with respect to system 300 may be performed. In this case, components of basketball system 302-1 may monitor both user 304 and corresponding basketball 312 as well as user 306 and corresponding basketball 315. The control unit of basketball system 302-1 may display the hit and throw attempt ratios of the two users when performing a throw attempt on the display screen of basketball system 302-1. In this example, the display screen of basketball system 302-1 may display the video recorded from the RGB camera of the basketball system 302-1 of the two users when the two users playing on basketball court 310 play. The hit and throw attempt ratios of the two users may be overlaid on the display screen above the video recorded from the RGB camera. In this case, the two users can see their respective scores (e.g., respective ratios) and the recorded video of the two users playing the game.
[0130] In some embodiments, these games can rank users who interact with the basketball system worldwide. For example, the result of a head-to-head competition can be that user 304 wins and user 306 loses. Thus, the ranking of user 304 will move up and the ranking of user 306 will move down in the list of other users. Users can attempt to play against other users using this player rating via their respective client devices or basketball systems.
[0131] In another game mode, live video highlights or real-time basketball can be streamed to the basketball system. For example, the basketball system 302-1 can display basketball highlights from a college or professional basketball game as the user 304 performs various shooting attempts with the basketball 312. These games can include NBA games, college games, and high school games.
[0132] In another game mode, the basketball system can enable a global competition mode. In the global competition mode, two basketball systems are connected and compete similar to the head-to-head confrontation mode. In addition, the global competition mode enables two users to invest in gaming costs for the head-to-head confrontation. The invested gaming costs can be escrowed to the winner's account, and the costs can be used to purchase additional paid features of the system. The paid features of the system can include paying for advanced real-time-remote one-on-one training, paying for special guest group coaching activities, and other exclusive activities.
[0133] In another game mode, the basketball system can enable the user to play in a training mode. When the user tries to improve his or her basketball skills, the user (e.g., user 304) can select the training mode. For example, when the training mode starts, the user can select a series of inputs from the user's client device or the backboard of the basketball system. The input of this series may include, for example, "commit to shooting", "ball control", "foot speed", "catch and shoot", and other practice modes. The user can select which input or multiple inputs to be practiced that he or she wants to practice. Then, the user is instructed to start practicing based on the instructions provided by the training module. The control unit of the basketball system 302-1 can sense throwing parameters such as throwing attempts, shots, misses, dribbling, posture, ball movement, and body movement based on the data provided by the sensors and cameras during the training module, just to name a few. In response, the control unit can generate output data (e.g., output data 130) for each attempted action such as throwing, dribbling, ball movement, and user movement, and the output data of each attempted action can be associated with the user's profile and stored.
[0134] When a specific event occurs for a training module (e.g., the user takes 50 free throws, the user takes 50 three-point shots, or the user learns how to dribble with his legs), the training module may end. In response to the end of the training module, the control unit may provide output data to a display screen of the basketball system and / or to a client device of the user. The output data may include training plans and other suggestions to help the user improve his basketball skills. If the user later comes back to perform a similar training module, the control unit may determine whether the user's ability has improved or declined compared to one or more previous training modules. The control unit may then indicate this information to the user via a client device or display screen, such as displaying "Your shooting percentage has improved by 20% in the 5 minutes since the last training session."
[0135] Figure 4 4 is a block diagram of an example computing system 400 illustrating a backboard. Computing system 400 may illustrate backboard 110, backboard 204, basketball system 302-1, and basketball system 302-N. As previously mentioned, the basketball system includes a control unit 402 that receives data from sensors 420 and cameras 418, and control unit 402 may process and / or analyze data from both sensors 420 and cameras 418. Control unit 402 includes memory 404, processor 406, and network interface card (NIC) 408, and is connected to other components of computing system 400 via wired or wireless communication links.
[0136] The control unit 402 may receive data from other inputs such as a user input device 422. In one example, the user input device 422 is a user's personal smartphone or personal computer. The user input device 422 may be connected to the control unit 402 via a wired connection via the NIC 408, such as the user input device 422 being connected to the basketball system using a USB connector, an HDMI connector, or a wireless connection. In one example, the wireless connection may be a Bluetooth connection between the user input device 422 and a transceiver and a control unit connected to the basketball system. In another example, the wireless connection may be a Wi-Fi network or a cellular data network connected to the Internet or a cloud 424. In this example, the Internet 424 may provide wireless access / connection between the control unit 402 and the user input device 422 (e.g., a personal smartphone). In some examples, software stored on the memory analyzes the input data and generates output data, which is transmitted to the player via the user input device 422 and / or the display panel 412. The control unit 402 may also transmit data to and / or receive data from a mobile application of a software module or user input device 422. For example, the mobile application of the user input device 422 may display data from the control unit 402 and / or provide an input field for the user to input data sent to the control unit 402 via a wired or wireless connection.
[0137] Power supply 410 can power components within the basketball system, as previously described. Control unit 402 can provide audible messages and music to speaker 414. Control unit 402 can receive audible messages and sounds from microphone 416.
[0138] As described above, the control unit 402 analyzes the input data to generate output data. The processor 406 processes the input data and uses a software program or module stored on the memory 404. Different modules can use the output data so that the user can use the basketball system in different ways. For example, the user can stream personalized training courses, and compete head-to-head with other players in remote locations (such as other basketball courts or local basketball courts). The output data can also be stored in the memory 404, the user input device 422 and / or the cloud storage system 424, so that the performance and other metrics can be tracked over time as the user interacts with the basketball system. Accordingly, the user can access the output data set to understand how its basketball skill development and performance trends (such as the number of missed shots), analyze or describe the throwing arc relative to the basket 112, and other characteristics. The control unit 402 can also display the user's performance data to the display panel 412.
[0139] Figure 5 is a flow chart illustrating an example of a process 500 for generating characteristics of a user playing basketball. Process 500 may be performed by backboard 110 of basketball system 107 and backboard 204 of systems 200 and 201.
[0140] The backboard may receive sensor data from a plurality of sensors and image data from one or more imaging devices connected to the backboard (502) regarding a user's throwing attempt. For example, a user may request to play a basketball game using the backboard. The user may authenticate to his client device and / or the backboard using an authentication component and may select a basketball game to play. For example, the basketball game may include a training course mode, a local head-to-head matchup, a live broadcast mode, and a global competition mode. Once the user has selected which basketball game to play via his client device or the backboard, the control unit of the backboard may instruct the user to start shooting towards the basket of the backboard. This may occur when a throw timer begins to count down or some other indication to start the selected basketball game.
[0141] As a user plays a game (e.g., moves with a basketball or makes a throw attempt), the camera and sensor may generate image data and sensor data, respectively, and may provide the image data and sensor data to a control unit, where the control unit may determine (i) whether the user has attempted a throw, and (ii) whether the throw attempt was successful. Each of the sensors and cameras may be configured to perform different and / or similar functions. For example, a LIDAR sensor may be configured to detect a user's throw attempt, and one or both of an angle and height of a basketball from the throw attempt. A motion sensor may be configured to detect one or more users on the court within close range of a backboard. A travel sensor may be configured to determine that a throw attempt was successful. An accelerometer may be configured to determine an indication of the position of the basketball relative to the backboard based on accelerometer data and vibration patterns during a throw attempt.
[0142] In addition, the backboard may include one or more cameras. The cameras may include one or more depth sensing cameras and / or one or more RGB cameras. Each of the cameras may be configured to perform different and / or similar functions. For example, the depth sensing camera may be configured to perform one or more of the following: (i) detecting a user on the court, (ii) tracking the movement of the user, (iii) detecting a basketball used by a user for a throwing attempt, (iv) tracking the movement of the basketball, (v) detecting the posture of the user. The one or more RGB cameras may be configured to perform one or more of the following: (i) recording image data of the field of view of the court, and (ii) recording image data of the area under the backboard to detect when the user's throwing attempt corresponds to a layup.
[0143] The backboard may include multiple layers, each layer housing different components. The front layer of the backboard may include a transparent coating for protecting the components within the backboard. For example, the front layer may include tempered glass that (i) protects the components within the backboard 110 and (ii) allows the user to see a display screen behind the front layer. The backboard may also include a second layer located behind the front layer. The second layer may include one or more components, such as sensors and cameras, for monitoring and generating data associated with users on the court (e.g., sensor and image data). The backboard may also include a third layer that houses a display screen. In some embodiments, the second layer of the backboard may include a display screen for providing feedback to the one or more users playing basketball. The basket may be connected to the protective layer of the backboard or the front layer of the backboard.
[0144] The backboard may also include a control unit that houses a CPU and a GPU for processing sensor and image data and providing output data to a display screen, one or more speakers, and / or a client device of the one or more users. The one or more speakers may provide audible output corresponding to the output data to the user.
[0145] The backboard may determine whether the throwing attempt was successful based on the received sensor data (504). In some embodiments, a throwing attempt by the user with the basketball may result in an unsuccessful attempt. In an unsuccessful attempt, the basketball may bounce off the basket, the backboard, or miss the backboard and components entirely. The backboard may further include one or more travel sensors configured to determine whether the basketball passed through the basket via the throwing attempt.
[0146] In other embodiments, a throwing attempt by a user with a basketball may result in a successful attempt. In a successful attempt, the basketball may pass through the basket hoop via the throwing attempt. The basketball may pass through the basket hoop by bouncing off the backboard, may pass through the basket hoop without bouncing off the backboard or the basket hoop, or may pass through the basket hoop by first bouncing off the basket hoop and then passing through the basket hoop.
[0147] The control unit of the backboard may instruct the sensor to acquire sensor data periodically. In other examples, the control unit of the backboard may instruct the sensor to acquire sensor data based on the type of basketball game being played. The control unit may also adjust the sensitivity of the sensor to improve the detection of throw attempts, successful throw attempts, and unsuccessful throw attempts.
[0148] The backboard may generate analysis indicating (i) characteristics of the user, (ii) characteristics of the throw attempt, (iii) suggestions for improving the throw attempt for subsequent throw attempts based on the received image data and whether the throw attempt was successful, and (iv) game performance (506). The camera within the backboard may acquire image data from the depth sensing camera and the RGB camera. For example, the image data may include an image or video of a user playing basketball on a court. For example, the depth sensing camera may generate (i) detection data of the user on the basketball court, (ii) movement data of the user, (iii) detection data of the basketball used by the user, (iv) tracking movement data of the basketball, and (v) detection data of the user's posture. The depth sensing camera may continuously and over a period of time generate and track each of the different detection and movement data of the user and the basketball.
[0149] The control unit may receive image data from the depth sensing and RGB cameras and provide the image data to a trained machine learning model. The machine learning model may generate data that (i) classifies or identifies each user on the court as a unique and persistent user, (ii) identifies a user's throw, (iii) generates characteristics of the user on the basketball court, and (iv) generates characteristics of a user's throw attempt. The trained machine learning model may simultaneously identify and track each of the users on the court. The characteristics of the user may include an identification of the user and a position of the user on the court relative to the backboard. The characteristics of the throw attempt may include an angle of the trajectory of the basketball during the throw attempt and an indication of whether the basketball passed through the basket. In addition, the control unit may associate the users identified by the trained machine learning model with stored user profiles.
[0150] Once identified, the control unit may update the stored user profile of the identified user with the newly generated characteristics of the user, including characteristics describing the user's throwing attempt. More specifically, the control unit may determine the resulting data based on the received sensor data, the resulting data including one or more of the following: (i) whether the throwing attempt resulted in the basketball passing through the basket, (ii) whether the throwing attempt resulted in the basketball bouncing off the front side of the backboard and not passing through the basket, (iii) whether the throwing attempt resulted in the basketball bouncing off the basket and not passing through the basket, (iv) the location of the user's throwing attempt, (v) the arc of the basketball during the user's throwing attempt. The control unit may store the resulting data in a corresponding profile together with the characteristics of the user, the characteristics of the throwing attempt output from the trained machine learning model, the received sensor data, and the received image data. The control unit may then provide the updated profile to an external server storing a plurality of profiles, each corresponding to a different user.
[0151] In some embodiments, the control unit may use the image data to determine whether the throw attempt was successful. For example, the control unit may apply a virtual inner cone and a virtual outer cone to the received image data from the depth sensing camera. The virtual inner cone may include an inner radius that matches the radius of the basket, may include a base that matches the basket, and may include a height extending from the basket. The virtual outer cone may include an outer radius that extends a certain distance from the basket, may include an extruded profile of the size of the virtual inner cone so that the two cones do not overlap, and may include a height extending from the basket, and is higher than the height of the virtual inner cone. Based on these cones, the control unit may determine whether the result of the throw attempt is a successful throw or an unsuccessful throw. For example, if the basketball passes through each cone and leaves the base of the virtual inner cone, the control unit may determine that the throw attempt was successful. Alternatively, if the basketball does not pass through these two cones or does not leave the base of the virtual inner cone, the control unit may determine that the throw attempt was unsuccessful.
[0152] In some embodiments, the control unit can use the data generated from the trained machine learning model and the received data to generate suggestions for the user. For example, the suggestions can include improvements for subsequent throwing attempts by the user. The suggestions can focus on one or more of the following during the user's subsequent throwing attempts: (i) posture, (ii) arm angle, (iii) release point of the basketball, and (iv) trajectory of the basketball. The suggestions can also be stored with the profile of a specific identified user.
[0153] In some embodiments, the control unit can store the game data or game performance associated with the game performed by the user in the user's profile.The type of the game performed can include, for example, training course mode, local head-to-head confrontation, live broadcast mode, and global competition mode.For example, the game performance can include the date and time of the game performed, the number of players in the game, the identification of each player participating in the game, the final score of the game, each player's each shot and miss during the game, each player's each shot and miss position on the basketball court, and the time when each player's each shot and miss occurs with both absolute time and relative time (relative to the start of the game).The control unit can store the game data as a tuple, a structure, a class, or a certain other computer format.If multiple users are playing a single game, the control unit can store the game data of each of the users in its corresponding profile of the single game.
[0154] The backboard may provide output data representing the analysis to one or more of: (i) a speaker, (ii) a display screen, and (iii) a user's client device (508). For example, the control unit of the backboard may provide output data including the generated suggestions to the display screen of the backboard. The output data may also correspond to an audible voice output, which may be provided to the speaker of the backboard to convey the generated suggestions to the user. In another example, the control unit may provide the generated suggestions to the user's client device via a network. The control unit may also provide media content from an RGB camera to the display screen of the backboard. In other examples, the control unit may receive media content from another control unit associated with another backboard, such as during a particular game type, and display the received media content from the other control unit on the backboard.
[0155] In some instances, the user can review its profile including generated suggestions and determined analysis. The user can review its corresponding profile on its client device and / or on the display screen of the backboard. The control unit can receive an instruction to access the corresponding profile from the user, and the control unit can identify the corresponding profile of the user about the user. The control unit can determine which user profile to access based on the authentication and identification of the user. Then, the control unit can provide the corresponding profile and its content to the client device of the user and / or to the display screen of the backboard. The user can review the analysis of previous throw attempts, such as recorded pictures, suggestions, comparisons between its throw attempts and professional athletes' throw attempts, data associated with the basketball game performed by the user (e.g., the type of the game performed), throw attempts, hits / misses, and opponent throw attempts, hits / misses, timestamp information, and other basketball information associated with the user interacting with the basketball system.
[0156] In general, the term device, system, computing entity, entity and / or similar words used interchangeably herein may refer to, for example, one or more computers, computing entities, desktops, mobile phones, tablet computers, tablet phones, notebook computers, laptop computers, distributed systems, game consoles (e.g., Xbox, Play Station, Wii), watches, glasses, keychains, radio frequency identification (RFID) tags, headphones, scanners, cameras, wristbands, kiosk, input terminals, servers or server networks, small USB flash drives, gateways, switches, processing devices, processing entities, set-top boxes, repeaters, routers, network access points, base stations, etc., and / or any combination of devices or entities suitable for performing the functions, operations and / or processes described herein. Such functions, operations and / or processes may include, for example, transmission, reception, retrieval, operation, processing, display, storage, determination, creation, generation, monitoring, evaluation, comparison and / or similar terms used interchangeably herein. In various embodiments, these functions, operations and / or processes may be performed on data, content, information and / or similar terms used interchangeably herein. Furthermore, in an embodiment of the present invention, the client device 104 may be a mobile device and may be operated by a user participating in an interactive physical confrontation game.
[0157] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to a storage system, at least one input device, and at least one output device.
[0158] These computer programs (also referred to as programs, software, software applications or code) contain machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages, and / or in assembly / machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to any computer program product, apparatus and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term machine-readable signal refers to any signal for providing machine instructions and / or data to a programmable processor.
[0159] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, as well as a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input from the user can be received in any form, including sound, voice, or tactile input.
[0160] The systems and techniques described herein can be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0161] A computing system may include clients and servers. Clients and servers are generally remote from each other and generally interact via a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
[0162] Although several embodiments have been described in detail above, other modifications are possible. For example, although the client application is described as accessing the delegate, in other embodiments, the delegate may be adopted by other applications implemented by one or more processors (e.g., applications executed on one or more servers). In addition, the logic flow depicted in the figure does not require the specific order or sequential order shown to achieve the desired result. In addition, other actions can be provided, or actions can be removed from the described process, and other components can be added to the described system or removed from the system. Accordingly, other embodiments are within the scope of the appended claims.
[0163] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or thing that may be claimed, but should be interpreted as descriptions of features specific to specific embodiments of specific inventions. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations or even initially claimed, in some cases, one or more features from the claimed combination may be deleted from the combination, and the claimed combination may be directed to sub-combinations or variations of sub-combinations.
[0164] Similarly, although operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed, to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products.
[0165] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing may be advantageous.
Claims
1. A system comprising: A housing comprising: a mounting attachment configured to couple the housing to a basketball backboard; a plurality of sensors configured to generate sensor data regarding a user's basketball throwing attempts; one or more imaging devices configured to generate image data of the basketball throwing attempt; A control unit, wherein the control unit is configured to: receiving the sensor data from one or more sensors of the plurality of sensors and the image data from the one or more imaging devices; determining, based on the received sensor data, whether the basketball throwing attempt was successful; generating an analysis based on the generated image data and whether the basketball throwing attempt was successful; and generating output data representative of the analysis; a display screen, wherein the control unit is configured to provide real-time or recorded video data to the display screen; and A network interface is coupled to the control unit, the network interface being configured to connect the system to one or more additional systems via a network.
2. The system according to claim 1, wherein: The display screen is configured to display user image data of one or more users.
3. The system according to claim 2, wherein: The network interface is configured to provide user image data received from the one or more additional systems to the control unit for display on the display screen.
4. The system according to claim 1, wherein: The control unit is configured to provide output data representative of the analysis to one or more of the display screen and a client device of the user.
5. The system according to claim 4, wherein: The analysis indicates (i) characteristics of the user, (ii) characteristics of the basketball throwing attempt, (iii) suggestions for improving the basketball throwing attempt for subsequent basketball throwing attempts, and (iv) game performance.
6. The system according to claim 1, wherein: The plurality of sensors include one or more of a LIDAR sensor, a motion sensor, a travel sensor, and an accelerometer.
7. The system according to claim 6, wherein: The LIDAR sensor is configured to generate sensor data indicative of the basketball throwing attempt by the user and one or both of an angle and a height of a basketball from the basketball throwing attempt; The motion sensor is configured to generate sensor data indicative of one or more users or one or more basketballs on a basketball court; the travel sensor being configured to generate sensor data indicating whether the basketball throwing attempt was successful; the accelerometer being configured to generate sensor data indicating a position of the basketball relative to a basketball backboard based on the accelerometer data and the vibration pattern; as well as Wherein, the control unit is configured to: (i) use sensor data from the LIDAR sensor to detect the user's basketball throwing attempt and one or both of the angle and height of the basketball from the basketball throwing attempt; (ii) use sensor data from the motion sensor to detect one or more users on the basketball court; (iii) use sensor data from the travel sensor to determine whether the basketball throwing attempt is successful; and (iv) use sensor data from the accelerometer to determine the position of the basketball relative to the basketball backboard based on the accelerometer data and the vibration pattern.
8. The system according to claim 7, wherein: The one or more imaging devices include one or more depth sensing cameras or one or more RGB cameras, wherein the one or more depth sensing cameras are configured to perform one or more of the following: (i) detect the user on the basketball court, (ii) track the movement of the user, (iii) detect the basketball used by the user for the basketball throwing attempt, (iv) track the movement of the basketball, (v) detect the user's posture, and wherein the one or more RGB cameras are configured to record image data of the field of view of the basketball court.
9. The system of claim 1, comprising a speaker, wherein the speaker is configured to provide an audible output representative of the analysis in response to receiving output data representative of the analysis from the control unit.
10. The system according to claim 2, wherein: The display screen is configured to display one or more of the following: (i) the image data from the one or more imaging devices, (ii) a heads-up display (HUD) displaying basketball throwing attempts and hits made by the user, and (iii) image data received from a second control unit connected via a network.
11. The system according to claim 5, wherein: The control unit is configured to provide image data received from the one or more imaging devices to a trained machine learning model to generate (i) characteristics of the user, (ii) characteristics of the basketball throwing attempt, and (iii) game performance, The characteristics of the user include the identification of the user and the location of the user, wherein the characteristics of the basketball throw attempt include the angle of the trajectory of the basketball, wherein the game performance includes data associated with the game played by the user; and The control unit is configured to store the characteristics of the user, the characteristics of the basketball throwing attempt, and the game performance in a profile of the user in an external server.
12. The system according to claim 11, wherein: The trained machine learning model is configured to simultaneously identify and track multiple users on the court, and the control unit is configured to: associating each of the plurality of users identified by the trained machine learning model with a stored user profile; as well as Each of the stored user profiles is updated with characteristics of the respective user and characteristics of the basketball throwing attempts of the respective user.
13. The system according to claim 11, wherein: The external server stores a plurality of profiles corresponding to different users.
14. The system according to claim 11, wherein: The control unit is configured to: generating a suggestion for improving the basketball throwing attempt for a subsequent throwing attempt, wherein the suggestion includes one or more of: (i) body posture, (ii) arm angle, (iii) release point of the basketball, and (iv) trajectory of the basketball; providing the generated suggestions for display on the display screen; as well as An output signal is provided for communicating the generated suggestion to the user.
15. The system of claim 14, wherein: The control unit is configured to provide the generated suggestion and the output signal to a client device of the user.
16. The system of claim 1, wherein: The control unit is configured to: receiving an instruction from the user to participate in a competition against a second user; connecting via the network to a second control unit associated with a second system used by the second user, wherein the second control unit is located at a geographically different location than the control unit; providing the received image data to the second control unit via the network; receiving second image data from the second control unit via the network; providing second image data received from the second control unit for display on the display screen; counting a number of throwing attempts made by the user based on the received sensor data and the received image data; receiving from the second control unit a second number of throw attempts made by the second user; providing (i) a number of throwing attempts made by the user and (ii) a second number of throwing attempts made by the second user for display on the display screen; as well as The number of throwing attempts made by the user is provided to the second control unit.
17. The system of claim 1, wherein: The control unit is configured to: receiving an instruction from the user to participate in a competition against a second user in a local competition; providing the received image data to the display screen; counting a number of basketball throwing attempts made by the user based on the received sensor data and the received image data; counting a number of basketball throwing attempts made by the second user based on the received sensor data and the received image data; as well as (i) the number of basketball throwing attempts made by the user and (ii) the number of basketball throwing attempts made by the second user are provided to the display screen, wherein the display screen overlays the number of basketball throwing attempts made by the user and the number of basketball throwing attempts made by the second user on top of the received image data.
18. The system of claim 17, wherein: The instructions from the user include audible voice commands, input from a client device of the user, or visual commands.
19. The system of claim 1, wherein: The control unit is configured to: In the generated image data, a virtual geometric region of a basket near a basketball backboard is generated; Determining whether the basketball enters the virtual geometric area; as well as A determination is made as to whether the basketball throw attempt is successful or unsuccessful in response to a determination that the basketball enters the virtual geometric zone.
20. The system of claim 1, wherein: The control unit is configured to: Receive commands from users; as well as In response to receiving a command from a user, one or more of the generated image data, the sensor data, and the output data representing the analysis is provided via a network.
21. The system of claim 1, further comprising: A cloud storage system configured to store user metrics that can be tracked over time and provide the user metrics to one or more of a user device, the control unit, and the network.
22. The system of claim 1, wherein: The control unit is configured to provide the recorded images of one or more users to at least one of the display screen, the client device, the cloud storage system, or via a network.
23. The system of claim 1, wherein: The control unit is configured to measure the speed of the user and to store the speed of the user in a corresponding user profile over time.
24. The system of claim 1, wherein: The control unit is configured to provide image data received from the one or more imaging devices to a trained machine learning model capable of using facial recognition to determine a user attempting a basketball throw.
25. The system of claim 1, wherein: The control unit is configured to receive image data from one or more of a client device of the user and a cloud storage system for display on the display screen.
26. The system of claim 1, wherein: The control unit is configured to receive gaming costs from one or more users and provide the gaming costs via the network.
27. The system of claim 1, wherein: The system is configured to rate users of the system based on user metrics and to share the rated user metrics with other systems via a network.
28. The system of claim 9, wherein: The control unit is configured to connect to one or more user devices via a Bluetooth connection and to receive audio signals from the one or more user devices and provide the audio signals to the speaker.
Citation Information
Patent Citations
Basketball sensing device
CN104190060A
Monitoring of physical training events
CN104815428A
Performance monitoring in a shooting sport using sensor synchronization
US20080015061A1
Basketball performance monitoring system
US20150265897A1
Basketball training system and method
US20160193518A1
