Trajectory extrapolation and origin determination for objects tracked in flight

Abstract

To provide tracking of an object such as a golf ball in flight, the tracking using data obtained from a sensor.SOLUTION: Methods, systems, and apparatus, including medium-encoded computer program products, for 3D flight tracking of objects include a method including: determining a golf ball trajectory based on observations by sensor(s), extrapolating the trajectory backward in time, calculating distance measure(s) between the extrapolated trajectory and physical locations, estimating a systemic error for observation(s), wherein the systemic error affects observed ball positions, estimating a stochastic error associated with the observation(s), wherein the stochastic error affects an angle of a trajectory determined from observed ball positions, combining the estimated systemic and stochastic errors to form error measure(s) for the distance measure(s), identifying one of the physical locations as an origin for the golf ball when the error measure(s) satisfy a criterion, and waiting for additional observations of the golf ball by the sensor(s) when the error measure(s) do not satisfy the criterion.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This specification relates to tracking the flight of an object such as a golf ball using data obtained from a camera, radar, and / or other sensor devices.

Background Art

[0002] U.S. Patent No. 5,413,345 describes a golf shot tracking and analysis system in which a range camera and a locator camera are arranged to view a golf ball when the golf ball is struck or after it has flown. As described in U.S. Patent No. 5,413,345, the locator camera displays the golf shot as it leaves the tee area, and the range camera displays the shot from a nearly vertical position to downrange. Further, even if the camera cannot "image" the ball on the tee, the starting point of a particular tee box of the ball in flight is determined. In addition, U.S. Patent Publication No. 20180011183 describes a system for tracking a plurality of projectiles using radar, in which one or more radar devices are arranged such that the field of view (beam coverage) of the radar device is maximized, each radar device has its own associated computer for defining its own three-dimensional radar coordinate system, and a central computer can trace the trajectory of each object backward to identify the strike bay from which each object was launched.

Summary of the Invention

Problems to be Solved by the Invention

[0003] This specification describes techniques related to flight tracking of an object such as a golf ball using data obtained from a camera, radar, and / or other sensor devices, particularly trajectory extrapolation and starting point determination during full-flight three-dimensional (3D) tracking.

[0004] In general, one or more aspects of the subject matter described in this specification include two or more defined physical locations from which a golf ball is launched into three-dimensional physical space, one or more golf ball sensors disposed in the three-dimensional physical space to detect a golf ball in flight after it has been launched from the two or more defined physical locations, and one or more computers communicatively coupled to the one or more golf ball sensors. The one or more computers include at least one hardware processor and at least one memory device coupled to the at least one hardware processor. The at least one memory device encodes instructions configured to cause the at least one hardware processor to perform operations, and can be embodied in one or more systems.At least one hardware processor is configured to determine a three-dimensional trajectory of a golf ball in a three-dimensional physical space based on an initial observation of the golf ball by one or more golf ball sensors, the at least one hardware processor is configured to extrapolate the three-dimensional trajectory of the golf ball backward in time to generate an extrapolated trajectory, the at least one hardware processor is configured to calculate one or more distance measures between the extrapolated trajectory and two or more defined physical positions, the at least one hardware processor is configured to estimate a systematic error that affects an observed ball position of at least one of the initial observations of the golf ball by one or more golf ball sensors, the at least one hardware processor is configured to estimate a probabilistic error that affects an angle of a trajectory determined from an observed ball position of at least one of the initial observations of the golf ball by one or more golf ball sensors, the at least one hardware processor is configured to combine the estimated systematic error and the estimated probabilistic error to generate one or more error measures of one or more of the distance measures, the at least one hardware processor is configured to identify one of the two or more defined physical positions as the starting point of the golf ball when one or more of the error measures meet a predefined criterion, and the at least one hardware processor is configured to wait for further observations of the golf ball by one or more golf ball sensors when one or more of the error measures do not meet a predefined criterion. Other embodiments of this aspect include corresponding methods, apparatuses, and computer program products.

Means for Solving the Problem

[0005] These and other embodiments may optionally include one or more of the following features. Two or more defined physical locations include two or more tee locations within two or more tee areas, and at least one hardware processor is configured to calculate a first distance measure of a first tee location of two or more tee locations within a first tee area of two or more tee areas, and a second distance measure of a second tee location of two or more tee locations within a second tee area of two or more tee areas, and the at least one hardware processor is configured to identify a first tee area including the first tee location as the starting point of a golf ball when the first distance measure meets a predefined threshold and the second distance measure does not meet the predefined threshold, and to identify a second tee area including the second tee location as the starting point of the golf ball when the first distance measure meets the predefined threshold, the second distance measure meets the predefined threshold, and the second tee location follows the first tee location along an extrapolated trajectory.

[0006] One or more golf ball sensors may include at least two separate golf ball sensor systems that independently track a golf ball within a three-dimensional physical space. A first observation is from a first golf ball sensor system of the at least two separate golf ball sensor systems, and at least one hardware processor can be configured to obtain additional observations of the golf ball from the first golf ball sensor system before a second golf ball sensor system of the at least two separate golf ball sensor systems detects the golf ball. At least one hardware processor can be configured to update a three-dimensional trajectory of the golf ball based on the additional observations to determine an updated three-dimensional trajectory. At least one hardware processor can be configured to extrapolate the updated three-dimensional trajectory of the golf ball backward in time to generate an updated extrapolated trajectory. At least one hardware processor can be configured to update one or more error measures according to the updated three-dimensional trajectory. At least one hardware processor can be configured to identify one of two or more defined physical positions as a starting point of the golf ball when one or more updated error measures meet a predefined criterion.

[0007] The initial observations are from a first golf ball sensor system of at least two individual golf ball sensor systems, and at least one hardware processor can be configured to obtain further observations of the golf ball from a second golf ball sensor system of the at least two individual golf ball sensor systems, and at least one hardware processor can be configured to determine an individual three-dimensional trajectory of the golf ball in a three-dimensional physical space based on the further observations of the golf ball by the second golf ball sensor system, and at least one hardware processor can be configured to extrapolate the three-dimensional trajectory of the individual golf ball backward in time to generate an individual extrapolated trajectory, and at least one hardware processor can be configured to calculate one or more individual distance measures between the individual extrapolated trajectory and two or more defined physical positions, and at least one hardware processor can be configured to estimate at least one individual systematic error of the further observations of the golf ball by the second golf ball sensor system, and at least one hardware processor can be configured to estimate an individual probabilistic error associated with at least one of the further observations of the golf ball by the second golf ball sensor system, and at least one hardware processor can be configured to combine the individual estimated systematic error and the individual estimated probabilistic error to generate one or more individual error measures of one or more individual distance measures, and at least one hardware processor can be configured to identify one of the two or more defined physical positions as the starting point of the golf ball when the one or more individual error measures meet a predefined criterion.

[0008] One or more golf ball sensors include a camera, and at least one hardware processor can be configured to estimate an inherent calibration error based on the focal length of the camera. The camera can be a stereo camera, and at least one hardware processor can be configured to calculate the parallax of the stereo camera based on the distance between the stereo camera and the first observation point. At least one hardware processor can be configured to estimate the stereo calibration error of the stereo camera as the estimation error of the calibrated rotation of the stereo camera. At least one hardware processor can be configured to estimate the total random parallax error of the extrapolated trajectory, and at least one hardware processor can be configured to adjust the error measure from the total random parallax error based on the distance from the first observation point to the baseline of the stereo camera.

[0009] The extrapolated trajectory can be within the field of view of one or more golf ball sensors, but it is not necessary to identify observations of the golf ball for this portion of the golf ball's flight. At least one hardware processor can be configured to check for intersections of the extrapolated trajectory with geometric shapes representing two or more defined physical positions. At least one hardware processor can be configured to determine the distances between the extrapolated trajectory and the impact positions within each of the geometric shapes representing two or more defined physical positions. At least one hardware processor can be configured to estimate systematic and probabilistic errors and select one of the two or more defined physical positions for identification as a starting point based on the determined distances when only one of the determined distances is below a threshold distance. At least one hardware processor can be configured to estimate systematic and probabilistic errors and select one of the two or more defined physical positions for identification as a starting point based on the last intersection along the extrapolated trajectory toward at least one of the first observation points when both or neither of the determined distances are below the threshold distance.

[0010] Geometric shapes representing two or more defined physical positions can include three-dimensional geometric shapes. At least one hardware processor can be configured to determine a strike position based on an input to the system. The input can include an input from at least one electronic location system. At least one hardware processor can be configured to set a golfer's strike position based on sensor data obtained from one or more test golf shots struck by the golfer and position data from a mobile communication device associated with the golfer, where the mobile communication device is communicatively coupled to at least one electronic location system. The at least one electronic location system can include a global navigation satellite system. The two or more defined physical positions can be a golf bay or tee area of a golf practice facility and various targets for a golf ball that include three-dimensional physical space.

[0011] In addition, one or more aspects of the subject matter described herein include determining a three-dimensional trajectory of a golf ball in a three-dimensional physical space based on an initial observation of the golf ball by one or more golf ball sensors, extrapolating the three-dimensional trajectory of the golf ball backwards in time to generate an extrapolated trajectory, calculating one or more distance measures between the extrapolated trajectory and two or more defined physical positions, estimating a systematic error that affects the observed ball position for at least one of the initial observations of the golf ball by one or more golf ball sensors, estimating a probabilistic error that affects the angle of the trajectory determined from the observed ball position for at least one of the initial observations of the golf ball by one or more golf ball sensors, combining the estimated systematic error and the estimated probabilistic error to generate one or more error measures for one or more of the distance measures, identifying one of the two or more defined physical positions as the starting point of the golf ball when one or more of the error measures meet a pre-defined criterion, and waiting for further observations of the golf ball by one or more golf ball sensors when one or more of the error measures do not meet a pre-defined criterion, encoding instructions configured to cause at least one hardware processor to perform operations including the foregoing, and embodying the instructions in one or more methods and / or one or more tangible computer-readable media (e.g., at least one memory device).

[0012] Calculating includes calculating a first distance measure of a first tee position among two or more tee positions within a first tee area of two or more tee areas, and calculating a second distance measure of a second tee position among two or more tee positions within a second tee area of two or more tee areas. Identifying may include identifying the first tee area including the first tee position as the starting point of the golf ball when the first distance measure meets a pre-defined threshold and the second distance measure does not meet the pre-defined threshold, and identifying the second tee area including the second tee position as the starting point of the golf ball when the first distance measure meets the pre-defined threshold, the second distance measure meets the pre-defined threshold, and the second tee position comes after the first tee position along the extrapolated trajectory.

[0013] One or more golf ball sensors include at least two individual golf ball sensor systems that independently track a golf ball in a three-dimensional physical space. The first observation is from a first golf ball sensor system of at least two individual golf ball sensor systems. The operations include obtaining further observations of the golf ball from the first golf ball sensor system before a second golf ball sensor system of at least two individual golf ball sensor systems detects the golf ball, updating a three-dimensional trajectory of the golf ball based on the further observations to determine an updated three-dimensional trajectory, extrapolating the updated three-dimensional trajectory of the golf ball backwards in time to generate an updated extrapolated trajectory, updating one or more error measures according to the updated three-dimensional trajectory, and identifying one of two or more defined physical positions as the starting point of the golf ball when one or more updated error measures meet a pre-defined criterion.

[0014] The initial observations are from a first golf ball sensor system of at least two individual golf ball sensor systems, and the operations include obtaining further observations of the golf ball from a second golf ball sensor system of at least two individual golf ball sensor systems, determining an individual three-dimensional trajectory of the golf ball in a three-dimensional physical space based on the further observations of the golf ball by the second golf ball sensor system, extrapolating the individual three-dimensional trajectory of the golf ball backward in time to generate an individual extrapolated trajectory, calculating one or more individual distance measures between the individual extrapolated trajectory and two or more defined physical positions, estimating at least one individual systematic error of the further observations of the golf ball by the second golf ball sensor system, estimating an individual probabilistic error associated with at least one of the further observations of the golf ball by the second golf ball sensor system, combining the individual estimated systematic error and the individual estimated probabilistic error to generate one or more individual error measures of one or more individual distance measures, and identifying one of the two or more defined physical positions as the starting point of the golf ball when one or more individual error measures meet a predefined criterion.

[0015] One or more golf ball sensors include a camera, and estimating the systematic error may include estimating an inherent calibration error based on the focal length of the camera. The camera can be a stereo camera, and estimating the inherent calibration error may include calculating the parallax of the stereo camera based on the distance between the stereo camera and the first observation point, and estimating the systematic error may include estimating the stereo calibration error of the stereo camera as the estimated error of the calibrated rotation of the stereo camera. Estimating the probabilistic error may include estimating the total random parallax error of the extrapolated trajectory and adjusting the error measure from the total random parallax error based on the distance from the first observation point to the baseline of the stereo camera.

[0016] The extrapolated trajectory may be within the field of view of one or more golf ball sensors, but it is not necessary to identify an observation of the golf ball for this portion of the golf ball's flight. Calculating one or more distance measures includes checking for an intersection of the extrapolated trajectory with a geometric shape representing two or more defined physical positions, determining the distance between the extrapolated trajectory and the impact position within each geometric shape representing two or more defined physical positions, estimating systematic and probabilistic errors and identifying a starting point based on the determined distance by selecting one of the two or more defined physical positions when only one of the determined distances is below a threshold distance, and estimating systematic and probabilistic errors and identifying a starting point based on the last intersection along the extrapolated trajectory toward at least one of the first observation points when both of the determined distances are below or neither is below the threshold distance, which may include selecting one of the two or more defined physical positions.

[0017] The geometric shapes representing two or more defined physical positions include three-dimensional geometric shapes, and the operation includes determining an impact position based on an input to the system, where the input may include an input from at least one electronic location system. Determining the impact position includes setting the impact position of the golfer based on sensor data obtained from one or more test golf shots struck by the golfer and position data from a mobile communication device associated with the golfer, where the mobile communication device is communicatively coupled to at least one electronic location system. The at least one electronic location system may include a global navigation satellite system. The two or more defined physical positions may be a golf bay or tee area of a golf practice facility and various targets for the golf ball that include three-dimensional physical space.

[0018] The various embodiments of the subject matter described herein can be implemented to achieve one or more of the following advantages. Even when a golf ball tracking system is used to simultaneously track golf balls coming from multiple different golf bays (or other defined physical locations), the starting point of the golf ball being tracked can be quickly and accurately identified, thus reducing the number of golf shots that are assigned to the wrong starting point golf bay and / or cannot be assigned to the starting point golf bay until after the golf shot has been hit. This can occur when the golf ball tracking system starts tracking the golf ball later than normal and the golf ball makes an angle that results in a significant error (e.g., parallax error in a stereo camera system) in selecting the correct starting point golf bay relative to the golf bay.

[0019] To address this problem, the error associated with the first point of each detected golf shot trajectory can be estimated, and an assessment can be made as to how that error affects the selection of the golf bay. This can include estimating the error in two parts: (1) a systematic error that affects the position error of the first point in the same way as for a point extrapolated behind the golf bay, and (2) a probabilistic error that affects the angle of the extrapolated trajectory generated from the points within each trajectory having a random position error. The systematic error can be calculated by estimating the vector value error of the first observed position of the trajectory, projecting this value onto the selected golf bay, and determining the extent to which this error affects the selection of the golf bay. The probabilistic error can be calculated by estimating the angular error of the first observation of the trajectory, determining the extent to which this error affects the backward extrapolation algorithm, and multiplying this error by the distance to the selected golf bay to determine the extent to which this error affects the selection of the golf bay. Considering these two types of errors, the number of golf shots incorrectly assigned to the starting point can be significantly reduced without increasing the delay (e.g., waiting for additional data and / or a new version of the trajectory) until the golf shot is shown to the golfer.

[0020] Furthermore, a separate dedicated golf ball tracking system for each golf bay is not required, which reduces the cost of a system where multiple golfers hit golf balls simultaneously. Having fewer golf ball tracking systems in a place such as a driving range reduces the work required for system management, and for correcting hardware errors or fixing malfunctions. Furthermore, a wider field of view can be achieved, and fewer components that need to be placed near the golfer, such as in a golf bay, can be used. For example, a golf ball tracking system using the systems and techniques detailed in this disclosure does not need to include tracking units installed in each golf bay of a golf entertainment facility. Furthermore, having fewer tracking systems reduces the overall complexity of the system from a software perspective, especially when a golf facility is completely covered by only a single system.

[0021] Details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the invention will become apparent from the description, the drawings, and the claims.

[0022] Like reference numerals and designations in the various drawings indicate like elements.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 3E

Figure 4A

Figure 4B

Figure 5A

Figure 5B

[0024] FIG. 1 shows an example of a system 100 that performs 3D flight tracking of a golf ball through three-dimensional space. In this example, the system 100 is part of a golf facility that includes a target 120 at a golf driving range 110 and a building 115 that includes a golf bay 130. In FIG. 1, the building 115 is shown as a rectangle, but it should be noted that a typical implementation has a curved portion of the building facing the golf driving range 110 such that the golf bay 130 has a semi-circular shape. The target 120 may have a radio frequency identification (RFID) tag interrogator that reads an RFID-tagged golf ball hit from a golf bay 130 that is one or more levels higher than the building 115. Additionally, one or more of the targets 120 may have individual areas of netting that place golf balls into respective RFID reader boxes associated with different areas. However, in some implementations, golf ball sensor systems 140, 150 can be used to track a golf ball in flight and identify where the golf ball lands without using such an RFID system, and thus such RFID tags and RFID tag interrogators are not required.

[0025] In the example of FIG. 1, the three-dimensional space in which the golf ball is tracked is a golf driving range 110, which can be of various shapes and sizes, but typically has a width of 300 to 500 feet and a length of 600 to 900 feet. The golf driving range 110 can be flat, or have small hills or one or more slopes, and can also have hazards such as water and sand traps. It should be noted that such hazards do not necessarily include actual water and sand, and may simply be colored like water and sand. The golf driving range 110 can be composed of real grass or artificial turf. Further, the targets can generally be grouped into categories representing their distances from the building 115, and the targets can have various shapes such as the circular shape of the main target and the rectangular shape of the trench target at the end of the driving range 110, and can have different colors for each target 120 or group of targets 120. Other shapes and sizes of the target 120, as well as a number of targets 120 different from those shown, are also possible. However, in some implementations, no specific targets and / or buildings are required in the three-dimensional space. For example, the golf ball sensor systems 140, 150 can be installed in an open field or a sports stadium or arena.

[0026] Generally, golf ball sensor systems 140, 150 are used to identify from which of a plurality of defined physical locations 130 a golf ball is being struck. Golf ball sensor systems 140, 150 include at least one golf ball sensor 140 and at least one computer 150 communicatively coupled to the golf ball sensor 140. The golf ball sensor 140 can be one or more sensors of one or more different types. For example, the golf ball sensor 140 can be an optical sensor (e.g., a stereo camera or two cameras operating together to provide stereo vision of a golf ball in flight), a radar sensor, or a combination thereof. In some implementations, two or more stereo cameras 140 are used to track a golf ball in flight within a three-dimensional space. In some implementations, at least one golf ball sensor 140 is a sensor unit that integrates a radar device and a camera to track a golf ball in three dimensions, the camera being used to provide angle information of the golf ball in a two-dimensional plane, and the radar device being used (in combination with the camera) to provide depth information of the golf ball in a dimension perpendicular to the plane, for example, in each in-flight camera observation of a golf shot, and the radial distance to the golf ball is used (using a pinhole camera model, triangulation, and a known separation distance between the camera and the radar device, which can be zero) to calculate the depth distance to the ball based on the camera angle of the in-flight camera observation. Combinations of other sensor types and sensor data are also possible, such as one or more phased array radar devices for determining the angle and distance to the ball and converting this information to a 3D position, or two or more radar devices for combining measurement data to construct a 3D flight trajectory.

[0027] The golf ball sensor 140 is arranged with respect to a three-dimensional physical space to detect a golf ball in flight after it has been launched from a defined physical location 130 into the three-dimensional physical space. In some implementations, the golf ball sensor 140 is arranged such that the sensor cannot observe the golf ball at the instant it is launched from the defined physical location 130. For example, the golf ball sensor 140 is mounted on a sunshade above and in front of the golf bay 130, thereby providing a wide field of view by the golf ball sensor 140 and obviating the need to include a tracking unit within the golf bay for the systems and techniques detailed in this disclosure. In some implementations, the golf ball sensor 140 is arranged such that the sensor can observe the golf ball at the instant it is launched from the defined physical location 130. However, as will be appreciated, even in such an arrangement, each individual golf ball may not be detected until it is launched, and thus sensor observations of the golf ball near the launch point may often not be available. Accordingly, regardless of whether the golf ball sensor 140 can observe the tee position, the systems and techniques detailed in this disclosure can be used to determine the ball trajectory and identify the location 130 from which each ball was launched.

[0028] The golf ball sensor 140 is communicatively coupled to one or more computers 150. This can be a wired connection that enables the golf ball sensor 140 to provide data to the computer 150, a wireless connection that enables the golf ball sensor 140 to provide data to the computer 150, or a combination thereof, and these connections can be simplex connections, duplex connections, or half-duplex connections. In some implementations, at least one computer 150 is connected to or integrated with each of two or more sensors 140 to create individual sensor systems, which independently / individually detect and track golf balls in three-dimensional space and thus provide separate trajectory predictions based on separate observations of the same golf ball moving through three-dimensional space. As used herein, an "observation" is the identification of sensor data indicative of a golf ball based on a predefined criterion, regardless of the type of sensor used.

[0029] Such individual golf ball sensor systems can also be communicatively coupled to a central computer system 150, such as one or more server computer systems, which integrates the trajectory predictions received from the individual golf ball sensor systems and makes a final determination as to which of the defined physical locations 130 should be identified and reported as the starting point of the particular golf ball being tracked. Note that the central computer 150 can be part of a computer system (e.g., of a golf facility) that manages a golf game and transmits information regarding golf shots (e.g., simulated golf shot animations in a virtual golf game and / or ball trace overlays in an augmented reality golf shot viewer) to a display device associated with the physical location 130. In any case, the computer 150 includes at least one hardware processor configured and / or programmed to perform the operations detailed in this disclosure, and at least one memory device coupled to the at least one hardware processor.

[0030] Furthermore, the defined physical location 130 can be a golf bay, a tee position within a golf bay, or generally a tee position. In some implementations, the three-dimensional space is not a golf driving range as shown. Thus, the defined physical location 130 where the golf ball is struck can be a designated strike position indicated, for example, by chalk, tape, or rope on the ground, and the three-dimensional space can be any location safe for hitting a golf ball within, for example, a sports stadium or arena or open field reserved for a golf event. For example, in some implementations, the three-dimensional space is an open grass field and the defined physical location 130 is a spot along a tee line selected by an individual golfer. As used herein, a reference to a "golf bay" should generally be understood to include a tee area or tee position unless explicitly described as an implementation limited to a golf bay having more than one tee area within the golf bay.

[0031] FIG. 2A is a schematic view of two golf bays 220A, 220B, one of which is identified as the starting point of a golf shot by a golf ball sensor system 200. The golf ball sensor system 200 is an example of the golf ball sensor systems 140, 150 from FIG. 1. The golf ball sensor system 200 detects a golf ball 210 in flight after being struck from one of two or more golf bays. From this initial observation of the golf ball and one or more subsequent observations of the golf ball, the system 200 determines a three-dimensional trajectory 212 (note that the figure represents only two dimensions for clarity of explanation). The three-dimensional trajectory 212 is then extrapolated backwards in time to generate an extrapolated trajectory 214, which intersects both the golf bay 220A and the golf bay 220B. Thus, it is not possible to readily recognize from the initial observation which of the two golf bays 220A, 220B should be identified as the starting point position of the golf shot.

[0032] Measuring and estimating the trajectory of a golf ball is useful for many applications to improve the golf experience. One example is a driving golf range where such processing of sensor-based golf ball observations is used to provide feedback and metrics to golfers, but such processing is also useful for entertainment purposes such as playing virtual golf courses and other games. In any case, for example, in order to save costs associated with having a dedicated sensor system for each golf bay, when one golf ball sensor system 200 is used to track golf balls hit from two or more golf bays 220A, 220B, the system for estimating the trajectory should be able to handle multiple golfers simultaneously and thus be able to distinguish which shot was hit by which golfer. In the example shown in FIG. 2A, the system 200 can wait for further observations of the golf ball 210 to improve the accuracy of the estimated trajectory, but this will result in a delay in identifying the starting golf bay. In contrast, the earlier the system 200 can identify the starting golf bay of a golf shot, the higher the probability of misidentifying which of the golf bays 220A, 220B is the starting golf bay, which is unacceptable from the user's perspective. This results in an undesirable trade-off between (1) unnecessarily delaying the identification of the starting golf bay for a golf shot having a trajectory that can be easily traced back to a single bay even when only a few observations are made, and (2) misidentifying the starting golf bay for a golf shot having a trajectory that is more difficult to distinguish which of two adjacent golf bays is the starting point. By using the systems and techniques described in this application, this undesirable trade-off can be eliminated, and thus both 1 and 2 can be avoided.

[0033] FIG. 2B is a schematic diagram of a data processing system including a data processing device 250 that identifies one of two or more golf bays as the starting point of a golf shot. The data processing device 250 can be connected through a network 280 to one or more computers 290, display devices 290, or both. Although only one computer is shown as the data processing device 250 in FIG. 2A, multiple computers can be used. Thus, the data processing device 250 can be used to implement one or more of the golf ball sensor systems 140, 150, 200, 410, 420, 490, 500 from FIGS. 1, 2A, 4A, 4B, and 5.

[0034] The data processing device 250 can include various software modules that can be distributed between an application layer and an operating system. These can include executable and / or interpretable software programs or libraries that can include a program 270 that operates as a 3D object flight tracking system. The number of software modules used can vary from implementation to implementation, and the software modules can be distributed over one or more data processing devices connected by one or more computer networks or other suitable communication networks. Further, in some cases, the described functionality is (partially or fully) implemented in the firmware and / or hardware of the data processing device 250 to increase the operating speed. Thus, the program and / or circuit 270 can be used to implement a ball detector, a tracker, and an orbit determination & ball starting point identifier, as detailed in the present disclosure.

[0035] The programs / circuits 270 of the ball detector, tracker, and trajectory determination & ball start point identifier use a physical model of the flight of the golf ball to extrapolate portions of the trajectory that are outside the field of view of the sensor or otherwise missed by the sensor for other reasons. The data processing device 250 can include a hardware device or a firmware device that includes one or more hardware processors 252, one or more additional devices 254, a computer-readable medium 256, a communication interface 258, and one or more user interface devices 260. Each processor 252 can process instructions for execution within the data processing device 250. In some implementations, the processor 252 is a single-threaded or multi-threaded processor. Each processor 252 can process instructions stored in a storage device such as the computer-readable medium 256 or one of the additional devices 254. The data processing device 250 uses its communication interface 258 to communicate with one or more computers / display devices 290, for example, via a network 280. Thus, in various implementations, the described processes can be executed in parallel or serially, on a single-core or multi-core computing machine, and / or on a computer cluster / cloud, etc.

[0036] Examples of the user interface device 260 include a display device, a touch screen display device, a camera, a speaker, a microphone, a tactile feedback device, a keyboard, and a mouse. The data processing device 250 can store instructions for performing the operations detailed in this disclosure in, for example, a computer-readable medium 256 or one or more additional devices 254 such as one or more of a floppy disk device, a hard disk device, an optical disk device, a tape device, and a solid state memory device. In general, the computer-readable medium 256 storing the instructions and the one or more additional devices 254 are examples of at least one memory device configured to encode instructions that cause at least one hardware processor to perform the operations detailed in this disclosure.

[0037] Additional device 254 may also include one or more sensors 140, 410 when the sensor and computer are integrated into a self - contained golf ball sensor system, such as systems 200, 490, 500. The one or more sensors 140, 410 can also be located remotely from data processing device 250, and data from such sensors 140, 410 can be obtained using one or more communication interfaces 258, such as an interface for wired or wireless technology. Such communication interfaces 258 can also be used to communicate extrapolated trajectories, proposed starting golf bays, the confidence level (one or more error measures) of the proposed starting golf bay, and / or other data to another computer system. For example, two or more data processing devices 250 can be individual golf ball sensor systems that independently track a golf ball in three - dimensional physical space and report their results to another data processing device 250, and the data processing device 250 determines which results to use and which golf bays 220A, 220B to identify as the starting golf bay, and this information can be communicated to a computer / display device 290, which can be a display device located at the identified golf bay, or to a data processing device 250 (e.g., a smartphone or tablet computer held by a person at the identified golf bay).

[0038] Figure 3A is a flowchart showing an example of a process for identifying the starting physical position of a golf ball detected and tracked during flight. Note that this is merely an example. The described operations can be performed in a different order and still achieve the desired result. At 300, an observation of a golf ball is identified in the sensor data (e.g., by computers 150, 200, 250, 420, 490, 500). This involves processing data received from one or more sensors (e.g., sensors 140, 200, 410, 490, 500) to find data indicating a golf ball based on pre-defined criteria for each type of sensor. For example, in the case of radar data, a ball speed criterion can be used that follows a known speed range of a golf ball immediately after being struck, or corresponds to the expected measure of a golf shot previously detected and currently being tracked. As another example, in the case of camera data, streaming image data can be processed in real time (e.g., using an object classifier) to identify various objects within a video stream that are candidate golf balls.

[0039] At 302, observations of a golf ball are associated with golf shots previously detected, and new golf shots are detected (e.g., by computers 150, 200, 250, 420, 490, 500). Note that detection & association 302 and identification 300 can be performed together when sensor data of a golf ball in flight is received. This can include simultaneous and / or parallel processing using parallel processing or a multi-tasking processor architecture. In the case of camera data, a golf shot can be detected at 302 when a series of candidate balls across a video frame set meet or exceed one or more established criteria for a golf shot. In some implementations, the analysis of image data involves the automatic adjustment of one or more thresholds (e.g., thresholds optimized per pixel) to maximize the sensitivity of objects of interest (e.g., objects that look like golf balls), as well as real-time filtering to enable the detection of a golf shot before all of the image data of the golf shot is received.

[0040] In the case of radar data, a ball speed criterion can be used so that the radar time series can only start in a specific speed range corresponding to a likely speed range of a golf ball (e.g., 10 to 250 miles per hour). Similarly, if range data is directly available from the radar sensor, objects detected outside of a predefined range can be ignored. When these measurements are received in real time, additional criteria can be used for a series of radar measurements. For example, a series of radar measurements of a golf shot should show a decrease in speed over time, which can be used to identify a golf shot in the radar data. Thus, golf balls detected at unexpected distances or speeds, as well as other objects such as birds and airplanes, can be easily ignored.

[0041] Furthermore, when more than one sensor type is used, data from different sensor types can be used to enhance the detection and tracking of golf shots. For example, when a golf shot is identified from radar data, a signal can be sent to trigger an adjustment of one or more criteria used when analyzing image data from a camera. This prioritizes the selection of an object identification set corresponding to the golf shot in the analysis and thus increases the likelihood of identifying the golf shot. It should be noted that the interaction between the processing of data from different sensor types (e.g., data from a radar device and a camera device) can proceed in both directions to improve the robustness of shot detection. By implementing more than one such cross-check of data from different sensor types, the system can be made more robust in a driving range (or similar golf-themed entertainment venue) where multiple golfers are present.

[0042] When using the radar in combination with optical tracking, for example, when using the depth distance calculation described above, it should be noted that to associate the ball velocity of the radar with the correct optical tracking, it may involve designing or programming the radar device to report the velocities of multiple objects each time a measurement is made (e.g., by mode setting). Thus, rather than picking up the fastest velocity (or the strongest reflection) and transmitting only that velocity, the radar device can be set to report the velocity of the fastest object or the velocity of the object with the strongest radar reflection. In such an operating mode, a certain degree of robustness against multiple balls in the air can be achieved in the following ways: (1) identifying the correct radar time series based on temporal correlation, (2) for each new set of radar measurements, trying all received velocities against the velocity predicted by the model, and (3) if any of the reported velocities is within a threshold distance of the ball velocity predicted by the existing radar time series model, that value is added to the time series, the model is updated, and then the system waits for the next set of measurements. Still, in some implementations, only one sensor type is used, for example, two or more stereo camera systems.

[0043] If there are no observations remaining in the current sensor data at 304 that are not associated, the process continues to receive and analyze incoming sensor data and identify ball observations at 300. If ball observations are identified at 300 and cannot be associated with previously detected golf shots at 302, these ball observations continue to be considered in an attempt to detect a new golf shot at 302. The process identifies ball observations at 300 and, if there are observations remaining in the current sensor data at 304 that are not associated and no new golf shot has been detected at 306, continues to receive and analyze incoming sensor data. Additionally, when a new golf shot is detected at 306, a separate process is generated to determine the starting point of the new golf shot. This separate process operates during the reception and analysis of new sensor data, identifies additional golf ball observations at 300, associates new ball observations with the newly detected golf shot at 302, and the starting point of that golf shot may or may not have been determined. In other words, the detection of golf shots, the determination of the starting points of golf shots, and the tracking of the flight of golf balls can all be performed simultaneously and in real time for multiple golf balls while the golf ball is still in flight and additional golf balls are being hit.

[0044] When a new golf shot is detected, based on the first observation of the golf ball identified at 300, a three-dimensional trajectory of the golf ball in three-dimensional physical space is determined at 310 (e.g., by computers 150, 200, 250, 420, 490, 500). This may involve using a physical model of the flight of the golf ball applied to three-dimensional coordinates in three-dimensional space determined from the first observation of the golf ball. Thus, at least the effects of gravity (e.g., drag, lift, and gravity) can be taken into account, and other physical parameters such as wind speed and direction, estimated ball spin, etc. can also be taken into account.

[0045] In some implementations, physical modeling and extrapolation of the trajectory are performed prior to associating different shots. The physical model can be determined from all observations of the golf ball, not just the first observation. The physical model can include modeling forces affecting a golf ball in flight, such as gravity, drag, and lift that are environment-dependent, physical characteristics of the golf ball, wind speed and direction, ball velocity, and spin of the golf ball.

[0046] At 312, the three-dimensional trajectory of the golf ball is extrapolated backward in time (and optionally forward) to generate an extrapolated trajectory (e.g., by computers 150, 200, 250, 420, 490, 500). However, rather than simply finding the intersection of the extrapolated trajectory with the geometric shape representing the golf bay, which can result in two intersections as shown in FIG. 2A, at 314, one or more distance measures between the extrapolated trajectory and two or more defined physical positions are calculated (e.g., by computers 150, 200, 250, 420, 490, 500). For example, at 314, each intersection of the extrapolated trajectory with one or more geometric shapes (e.g., square, rectangle, annular sector, cube, box, cuboid, 3D annular sector, etc.) representing one or more golf bays can be calculated, and the distance between the center point of each golf bay (or a pre-defined tee area within the golf bay, or a main strike position within the golf bay or tee area) can be calculated as the distance measure.

[0047] As another example, at 314, the minimum distance can be calculated between the extrapolated trajectory and the outside of a geometric shape representing a golf bay (or a predefined tee area within the golf bay, or a primary strike location within the golf bay or tee area), or between the extrapolated trajectory and the center point of these geometric shapes. Other distance measures are possible, including combinations of two or more measurements such as the average of the shortest distances (1) between the trajectory and the outside of the geometric shape, and (2) between the trajectory and the center point of the geometric shape. The distance measure can also take into account, for example, the geometric relationship of the golf bay (or tee area or strike location) to the current golf shot when a geometric shape that was last intersected is considered to be preferred over a geometric shape that was first intersected (starting from the modeled starting point of the golf shot).

[0048] Then, a measure of the certainty of the calculated distance measure can be determined in order to address errors in the observation of the trajectory and errors in extrapolating the trajectory behind the position of the golf bay when determining whether and when to identify the golf bay as the starting point of the golf shot. This can be particularly important when, as shown in Figure 2A, the extrapolated trajectory of the golf shot intersects two bays at similar distances from the center points of both bays, especially when there are two or more tracking systems present on-site and another tracking system may soon provide better results, such that it may be better not to display a shot to the user at all rather than display an incorrect shot to the user. Thus, for example, an estimate of the error of the extrapolated trajectory at the point of intersection with the golf bay can be calculated to estimate the level of certainty of the starting point golf bay, and the system can choose not to display the golf shot to the user if there is too much uncertainty regarding that starting point.

[0049] It should be noted that the error measure generated for the distance measure calculated at 314 need not use that distance measure as an input, although it can be used in some implementations. For example, a first distance measure can be calculated to determine which golf bays to consider as possible starting bays, and a second distance measure can be calculated for use in generating the error measure for the selected golf bay. Further, one or more error measures can be calculated based on the geometric relationship between (1) at least one of the first observations and the extrapolated trajectory, and (2) one or more of two or more defined physical locations.

[0050] The error of the extrapolated trajectory depends on various characteristics of the tracking sensor, but in order to facilitate the construction of an executable system that can quickly generate useful estimates, the error of the extrapolated trajectory can be reduced to the following two types of errors: (1) systematic errors that affect the observed ball position, and (2) probabilistic errors that affect the angle of the trajectory determined from the observed ball position. Further, the system can estimate these two types of errors separately. At 316, at least one systematic error of at least one of the first observations of the golf ball by one or more golf ball sensors can be estimated (e.g., by computers 150, 200, 250, 420, 490, 500), and at 318, at least one probabilistic error associated with at least one of the first observations of the golf ball by one or more golf ball sensors can be estimated (e.g., by computers 150, 200, 250, 420, 490, 500). For example, one or more golf ball sensors include a stereo camera (one or more cameras), and estimating the systematic error can include estimating the inherent calibration error based on the focal length of the camera and the parallax of the stereo camera, and estimating the stereo calibration error of the stereo camera as the estimated error of the calibrated rotation of the stereo camera.

[0051] FIG. 3B shows an example of an error of the tracking device and / or a systematic error caused by calibration of the tracking system. The systematic error may be caused by an error when calibrating the sensor itself or when the system is set up, for example, when the systems of the sensors are calibrated together. For example, in the case of a stereo camera sensor system, the systematic error may be caused by the individual calibration of each camera and also by the stereo calibration of each stereo system.

[0052] The systematic error generally results in some offset and affects two consecutive sensor readings in a similar manner. This means that the error of one observation is approximately equal to the error of the previous observation. Thus, as shown in FIG. 3B, the error between the actual position 316BO of the ball observed by the sensor and the observed position 316SO remains constant, including the extrapolated part of the trajectory. Therefore, this position error remains the same all the way to the golf bay 316GB between the actual unobserved position 316BE of the ball and the extrapolated ball position 316SE. Further, the systematic error can be estimated for any observation 316SO of the golf ball, but all that is required is the estimation of the error vector e1 of the first observation of the golf ball, and this error vector e1 is essentially equal to the error vector e bay and since the systematic error of the first (or subsequent) golf ball observation results in an error of similar size and direction in the golf bay, this is the systematic position error at the golf bay 316GB, i.e., this error is independent of the extrapolation behind the golf bay 316GB.

[0053] Therefore, the estimation of systematic error can be calculated using the first, second, third, fourth, or subsequent observations of the golf ball, and an error vector can be determined, which can be considered to be the same as the error vector of the position of the ball in the bay. This error vector is then projected onto a direction vector indicating along the rows of adjacent golf bays to determine how the systematic error affects the selection of the golf bay as the starting golf bay for the golf shot. Therefore, the geometric relationship between the golf bay and the current golf shot is taken into account.

[0054] This detailed example is provided here in relation to a stereo camera tracking system. Note that in a stereo camera system, the position error can be affected by the following source errors: (1) errors in intrinsic calibration, and (2) errors in stereo calibration. The errors in intrinsic calibration can be understood to have the following effects on the stereo camera system: (1) errors in focal length, which linearly increase from zero at the principal point of the image to larger errors towards the edge (errors proportional to the distance r from the principal point of the image), (2) errors in distortion coefficients, which exhibit a polynomial increase from zero at the principal point of the image to larger errors (errors proportional to r 2 +r 4 ), and (3) errors in distortion models, which increase and / or decrease non-linearly from zero at the principal point of the image to other magnitudes of error (errors proportional to f(x,y)). In light of these factors affecting the error, in order to simplify the error model, the second and third of these two factors (polynomial increase and non-linear increase / decrease) can be ignored, and the error in intrinsic calibration can be considered to be zero at the center of the image and increase linearly towards the edge of the image. Note that the system can adopt a distortion model that attempts to eliminate all these errors. However, since the distortion model and the attempts to remove distortion are not perfect, some residual error may remain in the system, and some parts of this residual error are more important than other parts of this residual error.

[0055] Furthermore, it can be understood that the stereo calibration error has the following effects on the stereo camera system: (1) the error in the calibrated rotation of the camera, which is approximately equal to the distance from the camera to the point multiplied by the angular error of rotation, and (2) the error in the calibrated position of the camera, which is directly converted into the position error of the point. It should be noted that this (2) is very likely to be small, and the impact on the position error in a given stereo camera implementation is negligible. Therefore, the impact of this (2) can be ignored. What needs to be addressed regarding the impact when selecting a golf bay as the source of the golf shot is the more important part of the residual error.

[0056] In the following detailed example, bold variables are vectors, the "hat" (^) symbol indicates a unit vector in the direction of length 1, |x| indicates the absolute value of x, ||a|| indicates the vector norm of a, × indicates the cross product between two vectors, and · indicates the scalar (dot) product between two vectors. The systematic error vector e1 for one or more observations of a golf ball can be calculated according to the following formula:

Equation

Equation

Equation

Equation

[0057] How much the error vector e1 affects the estimated error of the golf bay is determined by the direction of the error vector e1 compared to the (striking) direction as shown in Figure 3C. In Figure 3C, e is a part of the error vector e that affects the selection of the golf bay. This can be calculated according to the following equation: bay which is a part of. This can be calculated according to the following equation:

Equation

Equation

[0058] Figure 3D shows an example of the probabilistic error caused by noise in sensor readings. The probabilistic error may be caused by the error in the tracking of the golf ball, for example, by the noise present in various tracking operations. For example, a stereo camera tracking system has small random errors in two-dimensional (2D) tracking operations, which can cause pixel errors in the image coordinates of the observation of the golf ball, parallax errors that affect the estimated depth (distance) e_disp to the golf ball, and / or errors in the estimated direction of the golf ball e_dir. This results in an angular error between the first observation point and the last observation point, and the physical model will perform a backward extrapolation in a slightly wrong direction.

[0059] Therefore, the probabilistic error affects different parts of the observed trajectory differently, and the error in each observation of the golf ball is independent of the error in the previous observation. As shown in the example of Figure 3D, the error between the actual position 316BO of the ball observed by the sensor and the observed position 318SO is not constant. The extrapolation algorithm used to determine the trajectory of the golf ball is based on the physical forces acting on the golf ball, and therefore the algorithm attempts to estimate the change in the state of the golf ball between time steps. This means that it is more sensitive to the relative error between data points. Therefore, the position error between the unobserved actual position 316BE of the ball at this golf bay 316GB and the extrapolated ball position 318SE can be significantly different from the position error between any given actual ball position 316BO and its observation 318SO.

[0060] Similar to systematic errors, the estimation of probabilistic errors can start at the first (or subsequent) observation point. In general, probabilistic errors can be estimated at any point on the trajectory, but in many implementations, the accuracy and usefulness of the data may decrease as the distance from the first observation of the golf ball increases. Therefore, the estimation of probabilistic errors starts at the first point on the trajectory. In any case, to understand how this error affects the extrapolation position error at the golf bay 316GB, since the extrapolation algorithm is more affected by the angular error at the first point (or subsequent points) than the offset (position error), this error is converted into an angular error. Furthermore, the probabilistic error is "angular" in the sense that this error results in an error in the position of the golf bay 316GB that increases with the extrapolation distance.

[0061] In some implementations, this angular error is estimated as follows: Calculate the magnitude M of the error vector e1 of the first observation of the golf ball, determine the length L (either time or space) over which the golf ball has been observed, construct a function f(L) of L that can be non-linear to account for the fact that observations beyond a certain point are not useful for extrapolation, and then calculate the angle a of a triangle with sides M, f(L), and f(L). For example, f(L) = x * L / (y * (time_of_last_observation - time_of_first_observation)), where x and y are variables determined experimentally.

[0062] This angle a is the angular error at the first (or subsequent) observation point. Then, this angle is multiplied by the distance between the first (or subsequent) observation point and the golf bay 316GB to obtain the effect this error has on the extrapolation position at the golf bay 316GB. The direction of this error can be considered orthogonal to the direction of movement of the golf ball at the first (or subsequent) observation position. Therefore, this error vector e bayAlso, to determine how much this error affects the selection of the golf bay, it is projected onto a vector orthogonal to the general direction of strikes from that golf bay 316GB. Thus, the geometric relationship between the golf bay and the current golf shot is taken into account.

[0063] Referring to FIG. 3D, the probability error vector e1 of the first observation of the golf ball can be calculated according to the following formula:

Number

Number

Number

Number

Number

[0064] Regarding the systematic error, this error is converted into a position error in the formula of parallax. Since this error is also caused by the parallax error, the direction of the error is the same (see Equation (7)). Furthermore, the random pixel error may also affect the position error in the direction orthogonal to the parallax. In this case, it is a direction error, and the magnitude of the error is proportional to the distance from the baseline and can be calculated by the formula of the pinhole camera (see Equation (8)).

[0065] As already stated, the part of interest of this error is the part orthogonal to the direction of impact. This can be calculated according to the following formula:

Number

Number

Number

Number

Number

[0066] It should be noted that the estimation of the probabilistic error does not have to start exactly at the first observed position of the golf ball and can start at the second, third, fourth, or subsequent observed positions. In general, the estimation of the probabilistic error can be calculated using two or more of the first, second, third, fourth, and subsequent observations of the golf ball (or using all observations currently associated with the identified golf shot). In other words, it is also possible to take into account the number of observation points. In that case, the angle β is not calculated with p0 and p1, but with p0 and p nIt is calculated between here, where n depends on the number of available observations. Furthermore, the error can also be added to p0 instead of p1, and the angle β at p1 can be calculated, which provides very similar results. Therefore, if the error in 2D tracking is considered to be the only source of this error, the total random parallax error can be estimated.

[0067] Another way to estimate how much the angle β affects the probabilistic error in the bay is to construct a function f(β,r) that represents how much the angular error β and the extrapolation distance r affect the error in the bay. For example, f(a)=sin(β) * (r + z * r * r), where z is a secondary factor of the angular error. The component sin(β) * z * r * r takes into account part of the non-linear effect that extrapolation can have on the error, and the value of z is determined experimentally. Other methods are also possible to estimate how much the extrapolation is affected by the angular error. However, note that the relationship is not linear, i.e., it is a more generalized function of (angular) error and extrapolation distance rather than just multiplying the angle by the distance.

[0068] Other changes are also possible. The calculated error measure can be adjusted based on the effective area of the golf bay, which can vary depending on the direction from the golf bay to the observation point and the geometric shape of the golf bay. For example, if the geometric shape representing the golf bay is rectangular, when viewed in perspective, the effective width of that rectangle shrinks. This can be taken into account by comparing the direction of the golf shot with the direction of the rectangular bay and scaling the error accordingly:

Number

Number

[0069] However, in general, the individual handling of systematic errors and probabilistic errors (errors that increase with the same error at the bay position) leads to an improvement in the system performance regardless of the specific source of error identified by the geometric shape of a particular golf bay and a given implementation. When using different types of sensors, such as a radar sensor and a stereo camera sensor, the equations for calculating the actual errors are different, but generally the way different types of errors affect the selection of the bay is the same. For example, when using an FMCW (Frequency Modulated Continuous Wave) radar, the expected errors are similar to those of a stereo camera pair. Systematic errors in the angle and range with respect to the ball are expected. Additionally, probabilistic errors in the angle and range are also expected. This applies to all data points within the trajectory measured by the radar. Therefore, the same or very similar error propagation can be used to determine the bay error of a radar-based system.

[0070] Returning to FIG. 3A, at 320, one or more error measures of one or more distance measures can be generated by combining the estimated systematic error and the estimated probabilistic error (e.g., by computers 150, 200, 250, 420, 490, 500). For example, the estimated systematic error and the estimated probabilistic error can be added together. Other combinations are possible. Adding the two errors is one way to estimate the "worst-case" scenario, i.e., a scenario where both errors affect the observation / measurement in the same direction. If it is shown that this does not apply, the errors can be combined in another way. Further, combining at 320 can take into account (1) at least one of the first observations and the extrapolated trajectory, and (2) one or more of two or more defined physical positions, as well as the geometric relationship between the geometric shape and / or layout of the golf bay and / or the tee position therein.

[0071] At 322, a check is made (e.g., by computers 150, 200, 250, 420, 490, 500) as to whether one or more error measures meet a predefined criterion. At 322, when one or more error measures do not meet the predefined criterion, the process can wait for further observations of the golf ball by one or more golf ball sensors. Thus, the process can return to 310 to update the three-dimensional trajectory of the golf ball in the three-dimensional physical space based on new observations of the golf shot. For example, if the same (or another redundant) system provides a newer version of that golf shot with lower error within a short time frame, the first version can be safely discarded, so if the total error is higher than a specific threshold, the identified golf shot is not immediately displayed to the user. FIG. 3A shows that systematic error and probabilistic error are also recalculated 316, 318, but note that in some implementations, depending on the details of how these error measures are calculated in a given implementation, it may not be necessary to recalculate one or both of the systematic error and probabilistic error for the updated trajectory.

[0072] For example, as described above, since the systematic error may be the same as that previously calculated, when the updated trajectory does not change the systematic error, no updated calculation is required. In contrast, since the probabilistic error can substantially change when new ball observations are received from the sensors, this part of the error can be recalculated 318 in the second and subsequent error estimations of the updated trajectory. In some implementations, the total length of the observed trajectory can be used as an input to the error formula, and thus information from further observations is also used. Thus, each recalculation 318 can calculate the complete error combined at 320 of the start position of the ball in the golf bay using information about the entire trajectory.

[0073] Furthermore, the predetermined criteria checked at 322 can be a single criterion, such as a single error threshold, or two or more criteria. For example, two error measures for each of two golf bays are determined, and if both of these error measures are below the error threshold, at 322, the two error measures can be compared to each other to identify the golf bay corresponding to the lower error measure as the starting point of the golf shot. As another example, in a multi-detector system, after a certain amount of time has elapsed during which no other detection system picks up the golf shot, the first detector of the golf shot can test its tracked trajectory against a more lenient error threshold in order to give a better chance of identifying the starting-point golf bay of the golf shot.

[0074] For example, the first version of a golf shot detected by a stereo camera golf ball tracking system can be compared to a threshold of 0.15 using the combined error calculated in Equation (12), while the second and subsequent versions of the golf shot detected by the same stereo camera golf ball tracking system can be compared to a threshold of 0.25. As another example, a more stringent threshold (e.g., 0.15) can be applied only to the first version of a golf shot detected by any of two or more golf ball tracking systems, and a more lenient threshold (e.g., 0.25) can be applied to all subsequent versions of the golf shot (detected by any of two or more golf ball tracking systems). Using such two-level criteria at 322, the first version of the golf shot (e.g., from a non-primary system of the golf bay rather than from the primary system of the golf bay) can be accepted only if the error in the selection of the golf bay is very low, and thus, in some cases, the waiting time for the selection of the golf bay can be further reduced without the risk of misselecting the golf bay in a more general case.

[0075] FIG. 4A shows an example of a system 400 that performs 3D flight tracking of a golf ball through three-dimensional space. Two or more sensors 410 are communicatively coupled (wired connection, wireless connection, or both) to a computer system 420. The number of sensors 410 used varies depending on the size of the three-dimensional space to be covered, but generally, a set of sensors 410 is installed to cover the entire three-dimensional space (e.g., an entire golf driving range). Further, the number of sensors 410 can be increased, for example, so that at least two sensors 410 cover each golf bay, to provide redundancy in the spatial coverage. In the example of FIG. 4A, for clarity of this description, only two sensors 410A, 410B are shown, and each sensor 410 covers all the golf bays 430 arranged in three tiers.

[0076] As shown in the figure, the golf bay 430 is a 3D space within a building, e.g., the building 115 of FIG. 1. Since this is a 3D structure, the geometric shape representing the golf bay 430 in the golf ball tracking system can also be three-dimensional (having width and height) to identify starting golf bays on different floors of the building. Further, each sensor 410 (or combination of sensors 410A, 410B) tracks all golf balls within its field of view, and a physical model of the golf ball flight (operating in the computer system 420) is used to extrapolate portions of the trajectory that are outside the field of view or missed by the sensor for other reasons. Note that a single sensor 410A can have multiple sensor components, such as in the case of a stereo camera that has two optical sensors but can output one signal. Additionally, when multiple sensors 410A, 410B share the computer hardware 420 rather than having dedicated processing hardware (as shown in the figure), the sensor-computer combinations 410A, 420 and 410B, 420 can be individual sensor systems that attempt to independently identify golf shots, extrapolate each identified golf shot both forward and backward in time, and determine the starting golf bay based on the backward extrapolation of the golf shot. A collaborative process, which can also be executed on the computer 420 or another computer, can obtain data from these individual sensor systems and make a final decision as to which golf bay to identify as the starting point of a particular golf shot.

[0077] Using the extrapolated trajectory, the system calculates the physical location where each golf ball was struck so that the 3D tracking of the golf balls can be displayed to the correct person in the correct golf bay. For example, the first individual golf ball sensor systems 410A, 420 identify the first observation of the golf shot 412 and detect the golf shot 412, but there may not be a starting bay where the confidence level is initially identified as sufficient to display the golf shot 412 in any of the golf bays 430 (the first error threshold is not met). Then, before the second individual golf ball sensor systems 410B, 420 detect the golf shot 412, further observations of the golf shot 412 can be obtained by the first individual golf ball sensor systems 410A, 420. Thus, all of updating the 3D trajectory based on the further observations, extrapolating the updated trajectory backwards in time, calculating the updated distance measure, updating the estimated error (e.g., updating the probability error using the entire currently observed trajectory), and combining the estimated systematic error and the estimated probability error to generate an updated error measure of the updated distance measure can be done before the second individual golf ball sensor systems 410B, 420 detect the golf shot 412.

[0078] After this update, the first individual golf ball sensor systems 410A, 420 can identify the golf bay 432 as the starting point of the golf shot 412 when the updated error measure meets a pre-defined criterion, e.g., when an easier error threshold (than that used for the first check) is met. In this case, using such an easier error threshold is advantageous because the second individual golf ball sensor systems 410B, 420 may not actually detect the golf shot 412 at all. Further, even if subsequent versions of the golf shot 412 obtained by the first system 410A, 420 do not significantly improve the error measure, these subsequent versions are reconsidered with a more lenient threshold so that the starting point golf bay of all golf shots is reliably identified. In other words, in the absence of further observations available within a particular pre-defined time, the first observation (and further observations) can be processed and compared against another (less strict) pre-defined criterion. In some implementations, more than two thresholds are used over a given time.

[0079] In some implementations, multiple systems track the golf ball simultaneously and deliver new versions at specific intervals. A stricter threshold is used for the first version of the detected golf shot, which means there is time for a second tracking system to deliver its first version before the first tracking system delivers its second version of that golf shot. That version of the golf shot is used to determine the starting point golf bay only if the first version from the system passes the stricter threshold. This makes it easier to reduce latency and also ensures that the selection of the bay is not based on an incorrect shot version when a better version is available soon.

[0080] For example, the first systems 410A, 420 identify the first observation of the golf shot 414 and detect the golf shot 414, but there may not be a starting bay where the confidence level sufficient to display the golf shot 414 in any of the golf bays 430 is initially identified (the first error threshold is not met). While the first systems 410A, 420 continue to track the golf shot 414, further observations of the golf shot 414 are obtained by the second systems 410B, 420, and the same golf shot 414 can be detected by the second systems 410B, 420. The second systems 410B, 420 determine an individual three-dimensional trajectory of the golf ball in three-dimensional physical space based on the further observations, extrapolate the individual three-dimensional trajectory of the individual golf ball backward in time, calculate an individual distance measure between the individual extrapolated trajectory and the golf bays 434 and 436, estimate an individual systematic error and an individual probabilistic error, combine the individual estimated systematic error and the individual estimated probabilistic error to generate an individual error measure of the individual distance measure, and can identify one of the golf bays 434 and 436 as the starting point of the golf shot 414 when the first error threshold is met.

[0081] Accordingly, while the first systems 410A, 420 first detect the golf shot 414 and then use a less stringent error threshold to determine either the golf bay 434 or the golf bay 436 as the starting point of the golf shot 414, the second systems 410B, 420 detect the golf shot 414 and can accurately identify the golf bay 434 as the starting golf bay due to its position relative to the ball's trajectory. Also, while this is occurring, the second systems 410B, 420 first detect the golf shot 416 but do not have sufficient confidence to identify one of the golf bays 434 and 436 as the starting point. Subsequently, the first systems 410A, 420 also detect the golf shot 416, and there is little error affecting the selection of the golf bay for the golf shot 416 (due to the geometric relationship between the trajectory of the golf shot 416, the position of the sensor 410A, and the positions of the golf bays 434, 436). Thus, the first systems 410A, 420 can quickly identify the golf bay 436 as the starting point of the golf shot 416. It should be noted that this simultaneous process can be carried out.

[0082] Returning to FIG. 3A, at 322, when one or more error measures meet a predefined criterion, at 324, one of two or more defined physical positions is identified as the starting point of the golf ball (e.g., by computers 150, 200, 250, 420, 490, 500). Then, at 326, the identified starting point is used as an input for further processing (e.g., by computers 150, 200, 250, 420, 490, 500) such as using the identified starting point to facilitate further tracking of the golf ball in flight and / or presenting golf ball tracking data on a display device related to the identified starting point position. Various types of display devices can be used and can be located, for example, at different physical positions within various golf bays of a building.

[0083] Furthermore, each golf bay of a building, such as building 115 in FIG. 1, can be the same or have different levels of adaptation for different types of golf bays, and there can be different shapes, sizes, and layouts. The golf bays at the first level can have direct access to the driving range, while the golf bays at higher levels typically include safety nets that extend horizontally from the building to prevent injury in case someone accidentally falls from the front of the bay. Additionally, each golf bay can include one or more tee-off positions.

[0084] FIG. 4B shows an example of a system that performs 3D flight tracking of golf balls in relation to an example layout of two golf bays 440A, 440B that can be used in the system of FIG. 4A. The golf bays 440A, 440B can include furniture 445 such as sofas and tables to facilitate meals and conversations during the game. As will be understood, many layouts of the furniture 445 are possible, and the furniture 445 and layout of the golf bays 440A, 440B can be designed to provide flexibility when allocating the golf bays 440A, 440B to one or more groups of people to play together or separately.

[0085] Each golf bay 440A, 440B includes two tee-off positions, and each tee-off position includes a tee area 450 and a golf ball dispenser 455. Each golf ball dispenser 455 can be directly connected to a pneumatic tube system, and thus can automatically retrieve golf balls from the target and return them to the player without human intervention. Alternatively, the golf balls can be collected from a central location in a building, such as building 115 in FIG. 1, and manually dropped into the receptacle of the golf ball dispenser 455.

[0086] The two golf bays 440A, 440B can share an electronic hub, which can include a computer processor that controls what is displayed on each display device or can be a display device 470, such as a dam terminal that is communicatively coupled (wired, wirelessly, or both) to the computer processor, and can include various power lines and cables that support a separate display device for each golf bay. In some implementations, there is no shared electronic hub, and the display devices are individually associated with their respective golf bays 440A, 440B, each tee area 450 or dispenser 455 within golf bays 440A, 440B, and / or each personal portable electronic device 475 within golf bays 440A, 440B, such as a smartphone or tablet computer. Each display device can include a touch screen device that provides direct control of the game play to the player, including connecting to a central computer system of a building, such as building 115 of FIG. 1, and selecting the type of game they are playing and the current player.

[0087] In any case, one or more players can step into the tee area 450, obtain a golf ball from the dispenser 455, and then hit the ball. The golf ball sensor system 490 is an example of the golf ball sensor systems 140, 150 from FIG. 1 and includes both a computer (e.g., the data processing device 250) and a sensor (e.g., the stereo camera 254 integrated with the data processing device 250). The system 490 detects a golf ball 460 in flight after being hit from one of the four tee areas 450. From this first observation of the golf ball 460 and one or more subsequent observations of the golf ball 460, the system 490 determines a three-dimensional trajectory 464 (note that the figure represents only two dimensions for clarity of explanation). The three-dimensional trajectory 464 is then extrapolated backwards in time to generate an extrapolated trajectory 462, which intersects both the tee area 450A in the golf bay 440A and the tee area 450B in the golf bay 440B. Thus, from the first observation, it is not possible to easily recognize which of the golf bays 440A, 440B and which of the tee areas 450A, 450B should be identified as the starting physical position of the golf shot.

[0088] Therefore, system 490 needs to determine which of tee areas 450A, 450B to consider as the potential starting tee area. In some implementations, system 490 generates one or more error metrics for each of tee areas 450A, 450B and compares them. In some implementations, since the error metrics of adjacent tee areas 450A, 450B (or golf bays) result in very similar values, such a comparison may not be useful even if the error metric of one of these adjacent tee areas 450A, 450B (or golf bays) is very useful in determining when to confirm the starting point of a golf shot. Thus, in some implementations, system 490 selects only one of tee areas of 450A, 450B based on one or more calculated distance metrics and generates one or more error metrics only for the tee area selected in relation to the current extrapolated trajectory 462. For example, system 490 can determine which of tee areas 450A, 450B to consider as the potential starting point of a golf shot based on the distance between the intersection of extrapolated trajectory 462 and the geometric shapes representing tee areas 450A, 450B and a predefined point within tee areas 450A, 450B. Although detailed examples are provided below, as described above, various distance metrics can be used in various combinations.

[0089] In some implementations, system 490 compares distances DA, DB between (1) the intersections of extrapolated trajectory 462 and tee areas 450A, 450B and (2) the midpoint or center point of tee areas 450A, 450B. System 490 can also use as a distance metric the last golf bay and / or tee area that the golf ball intersects with extrapolated trajectory 462 as the golf ball moves forward, since it is considered that golfers do not hit the golf ball through each other's golf bays or tee areas. Thus, in the example of the intersection shown in Figure 4B, tee area 450B can be shown as the starting tee area.

[0090] Furthermore, system 490 can use other pre-defined (or defined on-the-fly) positions within the golf bay or tee area to measure distances. For example, system 490 can compare the distance between (1) the intersection of the extrapolated trajectory 462 and the tee areas 450A, 450B, and (2) the respective hitting positions HA, HB within the tee areas 450A, 450B. These hitting positions HA, HB can be pre-defined in the system based on information regarding the typical stance taken by the player when golfing, or by details of the tee area such as the tee position in a tee-up system. These hitting positions HA, HB can also be determined based on input to the system. For example, if it is known that the current golfer assigned to the tee area is left-handed, the hitting position can be adjusted accordingly, or if a camera image from the tee area shows the position where the ball is placed prior to the golf shot, the hitting position for that tee area can be updated on-the-fly based on the camera image.

[0091] In some implementations, system 490 checks whether the extrapolated trajectory 462 is within a pre-defined distance of the hitting positions HA, HB of the tee areas 450A, 450B. If it is within the pre-defined distance, the golf shot is considered to have hit that tee. If the extrapolated trajectory hits only one tee, this tee can be selected by system 490 for determination of the error measure and potential identification as the starting tee. If the extrapolated trajectory hits more than one tee based on the pre-defined distance, system 490 can select the last tee area intersected, e.g., tee 450B in the example of FIG. 4B. If the extrapolated trajectory does not hit a tee based on the pre-defined distance, system 490 can similarly select the last tee area intersected. Note that this process can be similarly applied to a golf bay when there is only one tee area per golf bay, etc.

[0092] Furthermore, the selection of the tee area and / or the golf bay can be used to identify a display device for rendering or animating a golf shot in a virtual golf game that may include information regarding the golf shot, such as golf shot statistics, and / or a representation of the golf course or other virtual game features. For example, if the tee area 450A is selected as the source of the golf shot and the error measure provides a sufficient degree of certainty, the golf shot information is presented on the display device 470 associated with the golf bay 440A or the tee area 450A. As another example, if the tee area 450B is selected as the source of the golf shot and the error measure provides a sufficient degree of certainty, the golf shot information can be presented on the display device 475 associated with the golf bay 440B or the tee area 450B or a person associated with the golf bay 440B or the tee area 450B.

[0093] Furthermore, as described above, multiple versions of each golf shot can be generated by the same golf ball sensor system 490 and / or by other golf ball sensor systems observing golf balls hit from the same golf bays 440A, 440B. FIG. 5A is a flowchart showing another example of a process for identifying the starting physical position of a golf ball detected and tracked during flight. At 560, a strike position can be determined (e.g., by computers 150, 200, 250, 420, 490, 500) within a geometric shape representing a defined physical position (e.g., a golf bay, a tee area, or other physical location). These strike positions can be predefined for the system or determined dynamically, and the geometric shape can be a three-dimensional shape.

[0094] As described above, the input to the system used to dynamically determine the impact location can be information regarding the current golfer or a camera image of the tee area. Further, in some implementations, the input to the system used to dynamically determine the impact location can be from an electronic location system that includes a mobile device associated with the golfer and a communication system such as a global navigation satellite system (GNSS), e.g., a global positioning system (GPS), a mobile phone network, or other wireless network, e.g., a WiFi network. FIG. 5B shows an example of a system that performs 3D flight tracking of a golf ball in relation to a golfer's personal mobile device.

[0095] The example of FIG. 5B is similar to the example of FIG. 4B in that it can be used with the system of FIG. 4A, and the golf ball sensor system 500 is similar to the aforementioned golf ball sensor system 490. Regions 510A, 510B can be a golf bay or tee area, or simply a considered area of the golfer, e.g., a designated area along the tee line. In any case, regions 510A, 510B can generally be referred to as golf bays 510A, 510B and can have a geometric shape that represents them such that the intersection of the extrapolated trajectory and these geometric shapes can be easily identified.

[0096] System 500 detects a flying golf ball 540 after it is hit from one of golf bays 510A, 510B. From this first observation of golf ball 540 and one or more subsequent observations of golf ball 540, system 500 determines a three-dimensional trajectory 546 (note that the figure represents only two dimensions for clarity of explanation). The three-dimensional trajectory 546 is then extrapolated backwards in time to generate an extrapolated trajectory 542 that intersects both golf bay 510A and golf bay 510B. Thus, system 500 needs to determine which of regions 510A, 510B to consider as the potential starting region.

[0097] To assist in this determination, signals can be obtained from mobile devices 520A, 520B associated with golfers in respective golf bays / areas 510A, 510B to determine the strike positions 530A, 530B associated with the golfers. For example, mobile devices 520A, 520B can be GPS devices that communicate over a wireless network enabling triangulation or other device location services, as shown in FIG. 5B, or a smartphone or tablet computer. In some implementations, the strike positions 530A, 530B are set for each golfer based on sensor data obtained by system 500 with one or more test shots by each golfer and position data from respective mobile devices 520A, 520B associated with the golfer. These strike positions 530A, 530B can then be used as described above or as will be described in more detail below in connection with FIG. 5A. Note that mobile devices 520A, 520B can also be display devices that transmit golf shot information when the start point of a golf shot is confirmed.

[0098] Referring back to FIG. 5A, at 562, one or more golf shot versions are generated or received (e.g., by computers 150, 200, 250, 420, 490, 500). For example, in some implementations, each sensor 410A, 410B of FIG. 4A uses a physical model of the flight of the golf ball to process sensor data, extrapolate portions of the trajectory that are outside its field of view (or otherwise missed by the sensor), and perform an assessment of an error measure, having dedicated computer hardware, thus forming individual sensor systems 410A, 410B, and their results can be reported to a central computer system 420, which can obtain results from these individual sensor systems 410A, 410B and make a final determination as to which golf bay 430 to identify as the starting point of a particular golf shot. Thus, the central computer 420 can receive different versions of a golf shot from each of the golf ball sensor systems 410A, 410B, as well as more than one version of a golf shot from the same golf ball sensor systems 410A, 410B.

[0099] In some implementations, as soon as a golf ball sensor system in a larger system starts tracking a golf ball, it generates several versions of that golf shot at regular intervals. The first version includes a first portion of the trajectory, and the second version includes all the observations from the first version and additional newer observations. In some implementations, subsequent versions inherit the golf bay assigned to the first version. In some implementations, the golf bay assigned is re-determined for each new version of the golf shot. In any case, the version generation process can reduce the waiting time until the system starts showing the trajectory to the golfer.

[0100] At 564, an intersection of the extrapolated trajectory with a geometric shape representing two or more defined physical positions (e.g., the intersection of the extrapolated trajectory 542 with regions 510A, 510B) is identified (e.g., by computers 150, 200, 250, 410, 490, 500), and at 564, the distance between the extrapolated trajectory and the impact positions (e.g., impact positions 530A, 530B) within each geometric shape is determined (e.g., by computers 150, 200, 250, 410, 490, 500). In some implementations as shown in FIG. 5B, generally intersections are found, except possibly for the extrapolated trajectories around the two ends of a full set of golf bays. Thus, the distance calculation can be between the intersection point and the defined impact position. In a situation where there is no intersection of the extrapolated trajectory with a given golf bay, the distance calculation can be the length of the line perpendicular to the extrapolated trajectory that intersects the impact position.

[0101] At 566, the calculated distances to the impact positions can be compared with a threshold value, which can be set experimentally for a given implementation, e.g., 40 centimeters. If only one of these calculated distances to the impact positions passes the threshold (i.e., is less than the threshold), at 568, the golf bay containing the impact position is selected to estimate systematic and probabilistic errors (e.g., by computers 150, 200, 250, 410, 490, 500). If both of these calculated distances to the impact positions pass the threshold, or if neither of these calculated distances to the impact positions passes the threshold, at 570, the golf bay that last intersected along the extrapolated trajectory (in the direction of the initial observation direction of the golf ball) is selected to estimate systematic and probabilistic errors (e.g., by computers 150, 200, 250, 410, 490, 500).

[0102] Next, at 572, one or more error measures are calculated / updated (e.g., by computers 150, 200, 250, 410, 490, 500). This may include operations 316, 318, 320 described above in connection with FIG. 3A. At 574, a check is made as to whether one or more error measures meet a pre-defined criterion (e.g., by computers 150, 200, 250, 420, 490, 500). This may include operations described above for the check at 322 in connection with FIG. 3A. Thus, at 574, when one or more error measures do not meet a pre-defined criterion, the process can wait for further observations of the golf ball by one or more golf ball sensors, and thus can wait for the next set of one or more versions of the golf shot generated at 562 (e.g., by computers 150, 200, 250, 410, 490, 500) and received at 562 (e.g., by computers 150, 250, 420).

[0103] For example, the central computer 420 can receive different versions of a golf shot from each golf ball sensor system 410A, 410B having a different perspective of the same golf shot. Each received version of the golf shot can include both an extrapolated trajectory and a confidence level (one or more error measures) of the starting golf bay of the golf shot. Thus, each golf ball sensor system 410A, 410B can perform an independent calculation of all the parameters of each golf ball shot trace it has found and send the results of its independent calculation to the central computer 420. The central computer 420 can compare the trajectory data to determine whether the two golf ball sensor systems 410A, 410B are observing the same golf ball in flight, and then the central computer 420 can use the best set of trajectory data from the two sensor systems 410A, 410B according to the received confidence levels provided by the two sensor systems 410A, 410B.

[0104] Thereafter, this process can be repeated and, as described above, the criteria can be changed each time a check is made at 574. Further, at 574, when one or more error measures meet a predefined criterion, at 576, the selected golf bay is identified as the starting point of the golf shot (e.g., by computers 150, 200, 250, 420, 490, 500). Then, as detailed above, the identified starting point is used as an input for further processing (e.g., by computers 150, 200, 250, 420, 490, 500) by using the identified starting point to facilitate further tracking of the golf ball in flight and / or presenting golf shot information to a display device associated with the identified starting point location.

[0105] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, or in combinations of one or more of them, including the structures disclosed in this specification and their structural equivalents. Embodiments of the subject matter described in this specification can be implemented using one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, a data processing apparatus. The computer-readable medium can be a hard drive of a computer system, or an optical disk sold through a retail distribution channel, or an industrial product such as an embedded system. The computer-readable medium can be obtained individually and later encoded with one or more modules of computer program instructions, such as by delivery of one or more modules of computer program instructions via a wired or wireless network. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.

[0106] The term "data processing apparatus" includes, by way of example, all apparatus, devices, and machines for processing data, including programmable processors, computers, or multiple processors or computers. The apparatus may include, in addition to hardware, code for creating an execution environment for the computer program, such as code constituting the firmware of the processor, protocol stack, database management system, operating system, runtime environment, or one or more combinations thereof. Further, the apparatus may employ various computing model infrastructures such as web services, distributed computing, and grid computing infrastructures.

[0107] A computer program (also known as a program, software, software application, script, or code) can be written in any suitable form of programming language, including compiled or interpreted languages, declarative languages, or procedural languages, and can be deployed in any suitable form, either as a stand-alone program or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. The program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to the program, or multiple related files (e.g., files that hold one or more modules, subprograms, or portions of code). A computer program can be executed on one computer or deployed so as to be executed on multiple computers located in one place or distributed across multiple places and interconnected by a communication network.

[0108] The processes and logical flows described in this specification can be implemented by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logical flows can also be implemented by, and as, special-purpose logic circuits, such as, for example, FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0109] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors. In general, a processor receives instructions and data from a read only memory or a random access memory or both. Essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. In general, a computer also includes one or more mass storage devices for storing data, for receiving data, for sending data, or for doing all three, such as, magnetic, magneto-optical disks, or optical disks, or is operatively coupled thereto. However, a computer need not have such devices. Further, a computer can be embedded in another device, such as, by way of example only, a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive). Devices suitable for storing computer program instructions and data include, by way of example, all forms of nonvolatile memory, media, and memory devices, including semiconductor memory devices, such as, EPROM (erasable programmable read only memory), EEPROM (electrically erasable programmable read only memory), and flash memory devices; magnetic disks, such as, internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0110] To provide interaction with a user, embodiments of the subject matter described herein can be implemented on a computer having a display device for displaying information to the user, such as an LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diode), or other monitor, and a keyboard and a pointing device, such as a mouse or trackball, by which the user can provide input to the computer. Other types of devices can be used to provide interaction with the user as well. For example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and the input from the user can be received in any form, including acoustic input, speech input, or tactile input.

[0111] A computing system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on respective computers that have a client-server relationship to each other. Embodiments of the subject matter described herein can be implemented on a computing system that includes back-end components, such as a data server, or includes middleware components, such as an application server, or includes front-end components, such as a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described herein, or includes any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., an ad hoc peer-to-peer network).

[0112] This specification includes many implementation details, but these should not be construed as limitations on the scope of the invention or what can be claimed, but rather as descriptions of features specific to particular embodiments of the invention. The specific features described herein in connection with separate embodiments can also be implemented in combination in a single embodiment. Conversely, the various features described in connection with a single embodiment can also be implemented separately or in any suitable sub-combination in multiple embodiments. Furthermore, features are described above as functioning in a particular combination and initially claimed as such, but one or more features from the claimed combination can, in some cases, be separated from the combination, and the claimed combination can be directed to a sub-combination or variation of a sub-combination. Accordingly, unless otherwise explicitly stated or the knowledge of those skilled in the art clearly indicates otherwise, any of the features of the foregoing embodiments can be combined with any of the other features of the foregoing embodiments.

[0113] Similarly, operations are shown in the drawings in a particular order, but this should not be understood as requiring that such operations be performed in the particular order or sequential order shown in order to achieve the desired result, or that all of the illustrated operations need to be performed. In certain situations, multitasking and / or parallel processing may be advantageous. Furthermore, the separation of various system components in the foregoing embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0114] Accordingly, specific embodiments of the invention have been described. Other embodiments are within the scope of the following claims and / or within the scope of the teachings of this application. For example, the above description focuses on the tracking of golf ball shots, but the described systems and techniques are also applicable to other types of object / projectile flight tracking, such as baseball or skeet shooting and non-sports applications. Further, the actions recited in the claims can be performed in a different order and still achieve the desired results.

Claims

1. Two or more defined physical positions from which a golf ball is launched into a three-dimensional physical space, One or more golf ball sensors disposed with respect to the three-dimensional physical space for detecting the golf ball in flight after the golf ball has been launched from the two or more defined physical positions into the three-dimensional physical space, One or more computers communicatively coupled to the one or more golf ball sensors, A system comprising: The one or more computers comprise at least one hardware processor and at least one memory device coupled to the at least one hardware processor, The at least one memory device is configured to: Determine a three-dimensional trajectory of the golf ball in the three-dimensional physical space based on a first observation of the golf ball by the one or more golf ball sensors; Extrapolate the three-dimensional trajectory of the golf ball backwards in time to generate an extrapolated trajectory; Calculate one or more distance measures between the extrapolated trajectory and the two or more defined physical positions; Estimate a systematic error that affects the observed ball position for at least one of the first observations of the golf ball by the one or more golf ball sensors; Estimate a probabilistic error that affects the angle of the trajectory determined from the observed ball position for at least one of the first observations of the golf ball by the one or more golf ball sensors; Combine the estimated systematic error and the estimated probabilistic error to generate one or more error measures for the one or more distance measures; Identify one of the two or more defined physical positions as the starting point of the golf ball when the one or more error measures meet a pre-defined criterion; Wait for further observations of the golf ball by the one or more golf ball sensors when the one or more error measures do not meet the pre-defined criterion; Encode instructions configured to cause the at least one hardware processor to perform operations including: System.

2. The two or more defined physical positions include two or more tee positions within two or more tee areas, and the calculating includes calculating a first distance measure of a first tee position of the two or more tee positions within a first tee area of the two or more tee areas, and calculating a second distance measure of a second tee position of the two or more tee positions within a second tee area of the two or more tee areas, and the identifying includes when the first distance measure meets a predefined threshold and the second distance measure does not meet the predefined threshold, identifying the first tee area including the first tee position as the starting point of the golf ball; when the first distance measure meets the predefined threshold, the second distance measure meets the predefined threshold, and the second tee position comes after the first tee position along the extrapolated trajectory, identifying the second tee area including the second tee position as the starting point of the golf ball; The system according to claim 1, comprising.

3. The one or more golf ball sensors include at least two individual golf ball sensor systems that independently track the golf ball within the three-dimensional physical space, the system according to claim 1.

4. The first observation is from a first golf ball sensor system of the at least two individual golf ball sensor systems, and the operations are acquiring further observations of the golf ball from the first golf ball sensor system before a second golf ball sensor system of the at least two individual golf ball sensor systems detects the golf ball; updating the three-dimensional trajectory of the golf ball based on the further observations to determine an updated three-dimensional trajectory; extrapolating the updated three-dimensional trajectory of the golf ball backward in time to generate an updated extrapolated trajectory; updating the one or more error measures according to the updated three-dimensional trajectory; when the one or more updated error measures meet the predefined criteria, identifying one of the two or more defined physical positions as the starting point of the golf ball; The system according to claim 3, comprising. Claim 5 The initial observation is from a first golf ball sensor system of the at least two individual golf ball sensor systems, and the operation is obtaining the further observation of the golf ball from a second golf ball sensor system of the at least two individual golf ball sensor systems, determining an individual three-dimensional trajectory of the golf ball in the three-dimensional physical space based on the further observation of the golf ball by the second golf ball sensor system, extrapolating the individual three-dimensional trajectory of the golf ball backward in time to generate an individual extrapolated trajectory, calculating one or more individual distance measures between the individual extrapolated trajectory and the two or more defined physical positions, estimating at least one individual systematic error of the further observation of the golf ball by the second golf ball sensor system, estimating an individual probabilistic error associated with the at least one of the further observation of the golf ball by the second golf ball sensor system, combining the individual estimated systematic error and the individual estimated probabilistic error to generate one or more individual error measures of the one or more individual distance measures, identifying one of the two or more defined physical positions as the starting point of the golf ball when the one or more individual error measures meet the predefined criteria, The system according to claim 3, comprising. Claim 6 The one or more golf ball sensors include a camera, and estimating the systematic error includes estimating an inherent calibration error based on the focal length of the camera. The system according to claim 1. Claim 7 The camera is a stereo camera, and estimating the inherent calibration error includes calculating the parallax of the stereo camera based on the distance between the stereo camera and the first observation point. Estimating the systematic error includes estimating the stereo calibration error of the stereo camera as an estimated error of the calibrated rotation of the stereo camera. The system according to claim 6. Claim 8 Estimating the probability error includes estimating the total random parallax error of the extrapolated trajectory and adjusting an error measure from the total random parallax error based on the distance from the first observation point to the baseline of the stereo camera, the system of claim 7.

9. The system of claim 1, wherein the extrapolated trajectory is within the field of view of the one or more golf ball sensors, but observations of the golf ball for this portion of the flight of the golf ball are not identified.

10. Calculating the one or more distance measures includes checking for an intersection of the extrapolated trajectory with a geometric shape representing the two or more defined physical positions; determining a distance between the extrapolated trajectory and a strike position within each of the geometric shapes representing the two or more defined physical positions; when only one of the determined distances is below a threshold distance, selecting one of the two or more defined physical positions to estimate a systematic error and a probability error and to identify as the starting point based on the determined distance; when both of the determined distances are below or neither is below the threshold distance, selecting one of the two or more defined physical positions to estimate a systematic error and a probability error and to identify as the starting point based on the last intersection along the extrapolated trajectory toward at least one of the first observation points; The system of claim 1, comprising:

11. The system of claim 10, wherein the geometric shape representing the two or more defined physical positions includes a three-dimensional geometric shape.

12. The system of claim 10, wherein the operation includes determining the strike position based on an input to the system.

13. The system of claim 12, wherein the input includes an input from at least one electronic location system.

14. Determining the strike position includes setting a strike position of the golfer based on sensor data obtained from one or more test golf shots struck by the golfer and position data from a mobile communication device associated with the golfer, the mobile communication device being communicatively coupled to the at least one electronic location system, the system of claim 13.

15. The at least one electronic location system is the system according to claim 14, including a global navigation satellite system.

16. The system according to claim 12, wherein the two or more defined physical locations are a golf bay or a tee area of a golf practice range including the three-dimensional physical space and various targets of the golf ball.

Images