Ball spin axis detection

Abstract

A method and apparatus, including a computer program product, implementing and using techniques for determining the spin axis of a ball in flight. At least a portion of an observed trajectory of the ball is obtained. The trajectory includes the three-dimensional position of the ball over time determined from individual observations of the ball by one or more sensors. A point in three-dimensional space is selected. An optimization problem is solved, using the three-dimensional acceleration at the selected point as the optimization objective, and the spin axis, spin magnitude, and wind vector as optimization parameters, with the solution to the optimization problem being a combination of optimization parameter values. The spin axis determined in the solution to the optimization problem is selected as the spin axis of the ball. The spin axis is used to generate ball trajectory information for presentation on an output device.

Description

[Technical Field]

[0001] The present invention relates to determining the spin axis of an object, and more particularly to determining the spin axis of a sports ball traveling through the air. [Background technology]

[0002] The spin axis of a sports ball (e.g., a golf ball, hereafter simply referred to as the "ball") refers to the axis about which the ball rotates during flight. Rotation about the spin axis generates lift forces acting on the ball. The orientation of the spin axis affects the direction of lift on the ball and, therefore, the curve of a golf shot. The spin axis remains constant throughout the ball's flight, even though the wind may "push" the ball in different directions, and is typically relative to the horizon. The spin axis can be thought of as analogous to an airplane wing: if the spin axis is parallel to the horizon, the ball will fly straight; if the spin axis is tilted to the left (i.e., the right wing is higher than the left wing), the ball will curve to the left; and if the spin axis is tilted to the right (i.e., the left wing is higher than the right wing), the ball will curve to the right.

[0003] For example, it is often desirable to know the spin axis for purposes of developing clubs or other equipment, or for informational purposes for players so that they can make reasonably accurate predictions about how a ball will move through the air when struck in a particular manner with a particular type of equipment.

[0004] It is well known that for a rotationally symmetric ball in flight, three main forces act on the ball: gravity, air resistance (also called "drag"), and the "lift" of the ball as explained above. These forces can be represented by vectors, which together form the total acceleration of the ball. The total acceleration can be determined from measurements of the ball's position by camera and / or radar sensors as it moves through the air along its trajectory. Typically, observations of the trajectory are made in three orthogonal planes: the horizontal plane (x-direction), the vertical plane (y-direction), and the depth plane (z-direction).

[0005] Using a set of simplifying assumptions and mathematical calculations, an estimate of "lift" can be derived that can be used to estimate the ball's spin vector. From the estimated spin vector, the ball's spin axis can be derived. However, measured trajectory values ​​are typically very noisy due to several factors, such as inaccurate wind speed observations, the mathematical model used to represent the camera when converting camera images to distance, the precision with which direction / vector can be determined, and the camera hardware (e.g., type of image sensor), to name just a few possible error sources. Summary of the Invention [Means for solving the problem]

[0006] In some aspects, the techniques described herein relate to a method for determining the spin axis of an in-flight ball, the method including the steps of: obtaining at least a portion of an observed trajectory of the ball, the observed trajectory including the three-dimensional position of the ball over time determined from individual observations of the ball by one or more sensors; selecting a point in three-dimensional space; and solving an optimization problem, wherein three-dimensional acceleration at the selected point in three-dimensional space is used as the optimization objective, and the spin axis, spin magnitude, and wind vector are used as optimization parameters, and the solution to the optimization problem is a combination of optimization parameter values; selecting the spin axis determined in the solution to the optimization problem as the spin axis of the ball; and using the selected spin axis of the ball to generate ball trajectory information to be presented on an output device.

[0007] In some embodiments, solving the optimization problem includes initializing values ​​of optimization parameters, calculating a regularized weighted loss function, optimizing m steps using descent-based optimization, updating point weights, and repeating the calculation, optimization, and weighting n times or until convergence.

[0008] In some embodiments, the sensor observation data is collected by one or more of a radar, a camera, or both.

[0009] In some embodiments, at least a portion of the observed trajectory of the ball is smoothed using a Savitzky-Golay filter.

[0010] In some embodiments, acceleration outliers are removed before solving the optimization problem.

[0011] In some embodiments, a wind speed representative of the prevailing wind conditions during the flight of the ball is obtained, and the obtained wind speed is used in initializing the wind vector.

[0012] In some embodiments, the wind speed is obtained by estimating the systematic error between the estimated trajectory of the ball and corresponding portions of the observed trajectory of the ball.

[0013] In some embodiments, the three-dimensional acceleration of the ball at each selected point in three-dimensional space includes an acceleration component due to gravity, an acceleration component due to drag, and an acceleration component due to lift, and the acceleration component due to lift is used to determine the initial value of the spin axis.

[0014] In some embodiments, the ball is a golf ball and the method is performed at a driving range or golf course.

[0015] In some embodiments, the normalized weighted loss function is a Huber loss function.

[0016] In some embodiments, it is determined whether the optimization has converged, and in response to determining that the optimization has converged, the iterative operation is performed less than n times.

[0017] In some embodiments, the step of calculating the normalized weighted loss function is performed by automatic differentiation.

[0018] In some embodiments, the descent-based optimization is the Broyden-Fletcher-Goldfarb-Shanno algorithm.

[0019] In some aspects, the techniques described herein relate to computer software products configured, when executed, to perform method steps.

[0020] In some aspects, the techniques described herein relate to a system for determining the spin axis of a ball in flight, the system including: means for obtaining at least a portion of an observed trajectory of the ball, the observed trajectory including three-dimensional positions of the ball over time determined from individual observations of the ball by one or more sensors; means for selecting a point in three-dimensional space; means for solving an optimization problem, wherein three-dimensional acceleration at the selected point in three-dimensional space is used as an optimization objective, and the spin axis, spin magnitude, and wind vector are used as optimization parameters, and the solution to the optimization problem is a combination of optimization parameter values; means for selecting the spin axis determined in the solution to the optimization problem as the spin axis of the ball; and means for using the selected spin axis of the ball to generate ball trajectory information to be presented on an output device.

[0021] In some aspects, the techniques described herein relate to a system for determining the spin axis of a ball in flight, the system including: means for obtaining at least a portion of an observed trajectory of the ball, the observed trajectory including the three-dimensional position of the ball over time determined from individual observations of the ball by one or more sensors; means for solving an optimization problem by using three-dimensional acceleration at a selected point in three-dimensional space as an optimization goal and a combination of optimization parameter values ​​of spin axis, spin magnitude, and wind vector; and means for presenting ball trajectory information to be presented on an output device, the ball trajectory information including the spin axis determined in the solution of the optimization problem.

[0022] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features and advantages of the invention will be apparent from the description and drawings, and from the claims. [Brief explanation of the drawings]

[0023] [Figure 1]FIG. 1 is a schematic diagram of a spin axis determination system according to some embodiments. [Figure 2] FIG. 1 is a schematic diagram of a data processing system of a spin axis determination system according to some embodiments. [Figure 3] 1 is a flowchart illustrating the operation of a spin axis determination system in determining the spin axis of a ball in flight, according to some embodiments. [Figure 4] 10A-10C are graphs of measured and smoothed acceleration values, respectively, of a ball in flight according to some embodiments. [Figure 5] 1 is a flowchart illustrating a method for wind speed estimation according to some embodiments. [Figure 6A] 1A-1C are schematic diagrams illustrating a user interface showing ball trajectories for a measured shot with a particular spin axis and an estimated shot under the same circumstances with a different spin axis, according to some embodiments. [Figure 6B] 1A-1C are schematic diagrams illustrating a user interface showing ball trajectories for a measured shot with a particular spin axis and an estimated shot under the same circumstances with a different spin axis, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0024] Like reference symbols in the various drawings indicate like elements.

[0025] Various embodiments of the present invention relate to techniques for determining the spin axis of a ball, such as a golf ball, in flight based on recorded data about the trajectory traveled by the ball. Typically, data captured by sensors, such as cameras and / or radar, as the ball travels through the air is processed by software to generate a trajectory for the ball. From the generated trajectory, the spin axis of the ball is determined by using a physical model that describes the relationship between the ball's acceleration and spin axis and solving an optimization problem with the ball's three-dimensional acceleration as the optimization goal.

[0026] In some embodiments, noise in the sensor observations is managed by smoothing the raw sensor data in a first stage (i.e., before optimization), using, for example, a Savitzky-Golay (SG) filter, and / or by selecting an appropriate loss function, such as a Huber loss function, in a second stage during optimization. As a result, spin axis determination using the systems and techniques described herein can reduce noise in the measured trajectory values, thereby providing one or more advantages, including more robust and accurate spin axis settings compared to those currently possible, and enabling the use of other types of sensors that would otherwise typically provide data that is too noisy to be usable in the context of spin axis calculation. Because small differences in spin axis calculation, such as one or two degrees, can make a large difference in determining the ball's trajectory, the optimization methods described herein can have a significant impact on the accuracy of the spin axis calculation.

[0027] As noted above, the determined spin axis can be used for a variety of purposes, such as developing clubs and other equipment, or for informational purposes for players so that they can make a reasonably accurate guess as to how a ball will move through the air when struck in a particular manner with a particular type of equipment, or to simulate portions of trajectory that cannot be measured in a gaming establishment, for example.

[0028] FIG. 1 illustrates a computer 102 configured to communicate with one or more sensors 106 via a network 104 to obtain data captured by the sensors 106 about a ball in flight through three-dimensional (3D) space. It should be noted that in some embodiments, the computer 102 does not communicate directly with the sensors 106, but instead communicates through an intermediate repository that stores data captured by the sensors 106. It should also be noted that the sensors 106 need not be permanently connected to the network 104, but can be removably connected via various types of interfaces, such as, for example, USB-3. The computer 102 obtains data about the ball's individual trajectory collected by the one or more sensors 106 and applies a physics model 108 to the data to determine a spin axis, which can then be used for various purposes. A more detailed description of the operation of the computer 102 according to some implementations is provided below with reference to the flowcharts of FIGS. 3 and 5.

[0029] The network 104 may be any combination of wired and / or wireless networks, including a local network, such as an intranet, or a global network, such as the Internet. The network 104 may also include any combination of public and / or private networks.

[0030] Computer 102 may optionally communicate with client computer 110, which may be used to display data on a user interface, for example, regarding single shots, wind speed, statistics, player data, etc. Various software applications may be executed on client computer 110 to display the data in a manner preferred by a user of the spin axis determination system. For example, client computer 110 may include software that presents the user with a spin axis calculation of a recent shot, along with a recommendation of one or more different types of clubs to try and how such equipment changes are predicted to affect the ball's trajectory. Computer 102 and / or client computer 110 may be created using dedicated computer hardware or using shared computer hardware that can be programmed to generate virtual processing environments corresponding to any suitable computer architecture.

[0031] 6A-6B show example graphs that can be displayed to a user on a user interface in response to a selection made by the user. In this case, the user has selected to view a comparison between two trajectories of the ball: one measured trajectory 602, from which the spin axis has been calculated, and one estimated trajectory 604, which shows what the shot would look like if the spin axis were slightly different, e.g., as a result of using a different club, while other factors remain unchanged. By visualizing such a comparison on a user interface, the user can obtain valuable information that can inform the user on, for example, how to adjust their technique or club selection to become a better player.

[0032] As mentioned above, computer 102 receives input data from a set of sensors 106 that capture data about a ball in flight through three-dimensional (3D) space. The ball may be, for example, a golf ball or another type of object that is hit, kicked, or thrown to move through the air (e.g., a baseball, soccer ball, football / rugby ball). On a more general scale, the principles of various embodiments of the invention described herein apply to any object in which the Magnus effect (i.e., the path of a rotating object is deflected in a way that is not present when the object is not rotating due to the pressure difference in fluids on opposite sides of the rotating object and the rotational speed) forms a significant part of the resultant force acting on the object.

[0033] In some embodiments, the 3D space is a golf practice area, such as a driving range, a grass field, or another open area from which objects can be launched. For example, the 3D space can be a play area of ​​a sport, such as a golf course, where balls are hit from a launch area, such as a golf tee on a particular hole on the golf course, or an intermediate landing point for the golf ball in play on the course, toward a target, such as a cup at the end of a particular hole on the golf course, or an intermediate landing point for the golf ball in play on the course. Other implementations are possible, for example, the launch area is one of multiple designated tee areas along a tee line from which golfers can launch golf balls into an open field, or the launch area is one of multiple designated tee areas in the stands of a sports stadium from which golfers can launch golf balls onto and over the playing field of the sports stadium.

[0034] Two or more sensors 106, such as cameras (e.g., a stereo camera pair), radar devices (e.g., a Doppler radar device), or a combination thereof (e.g., a camera used to sense the angle relative to the ball combined with a radar used to sense the distance and / or speed of the ball), are connected to the computer 102 either directly or through one or more computing devices, which may perform various levels of processing on the data collected by the sensors 106 before transmitting the processed data to the computer 102.

[0035] Typically, the sensors 106 are positioned near the ball launch area. However, in some implementations, one or more of the sensors 106 can be positioned along one or both sides of the 3D space, or on the other side of the 3D space opposite the launch area. For example, in a golf tournament, a camera can be positioned behind the green and looking back at the golfer, assuming the shot is being hit toward the green. Thus, in various implementations, the sensor can observe and track objects moving away from the sensor, toward the sensor, and / or passing through the sensor's field of view.

[0036] The sensors 106 may have different sensitivities (e.g., different image sensor resolutions) and may be of various types (e.g., radar, camera, or a combination thereof), which may affect the quality of the data sent to the computer 102. However, the sensitivity of the sensors 106 does not necessarily affect how the computer 102 operates, as will be explained in more detail below with respect to Figure 3. Thus, there are a wide variety of sensor setups and configurations.

[0037] Different types of computers can be used in the system. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. As used herein, a "computer" can include a server computer, a client computer, a personal computer, an embedded programmable circuit, or a special-purpose logic circuit. Figure 2 is a schematic diagram of a data processing system including a data processing device 200, which represents an implementation of computer 102. Data processing device 200 can be connected to one or more computers 290 via a network 280.

[0038] The data processing device 200 may include various software modules that may be distributed between the application layer and the operating system. These may include executable and / or interpretable software programs or libraries, including, for example, a program 230 that operates as an orbital spin axis determination program. The number of software modules used may vary depending on the implementation. Also, in some cases, the program 230 may be implemented in embedded firmware, while in other cases, the program 230 may be implemented as software modules distributed across one or more data processing devices connected by one or more computer networks or other suitable communication networks.

[0039] Data processing apparatus 200 may include hardware or firmware devices including one or more hardware processors 212, one or more additional devices 214, a non-transitory computer-readable medium 216, a communication interface 218, and one or more user interface devices 220. Processor 212 may process instructions for execution within data processing apparatus 200, such as instructions (e.g., of program 230) stored on non-transitory computer-readable medium 216, which may include a storage device such as one of the additional devices 214.

[0040] In some implementations, the processor 212 is a single-core or multi-core processor, or two or more central processing units (CPUs). The data processing device 200 uses its communication interface 218 to communicate with one or more computers 290, for example, via a network 280. Thus, in various implementations, the described processes may be executed in parallel, simultaneously, or serially on single-core or multi-core computing machines, and / or computer clusters / clouds, etc.

[0041] Examples of user interface devices 220 include displays, touchscreen displays, speakers, microphones, haptic feedback devices, keyboards, mice, and headsets or heads-up displays for virtual reality or augmented reality environment systems. Furthermore, the user interface devices need not be local devices 220 but can also be remote from the data processing apparatus 200, e.g., user interface devices 290 accessible via one or more communications networks 280. For example, the user interface devices 220 / 290 can be a user's smartphone or tablet computer, e.g., for augmented reality implementations. The data processing apparatus 200 can store instructions for implementing operations as described herein on a non-transitory computer-readable medium 216, which can include, for example, one or more additional devices 214, e.g., a floppy disk device, a hard disk device, an optical disk device, a tape device, and a solid-state memory device (e.g., a RAM drive).

[0042] Additionally, instructions implementing the operations described herein can be downloaded from one or more computers 290 (e.g., from the cloud) over a network 280 to the non-transitory computer-readable medium 216. In some implementations, the data processing device 200 is a smartphone or tablet computer. In some implementations, the RAM drive is a non-volatile memory device into which instructions are downloaded each time the computer is turned on.

[0043] FIG. 3 is a flowchart illustrating a method 300 performed by computer 102 in determining spin axis, according to some embodiments. As seen in FIG. 3 , method 300 begins by acquiring sensor data of the ball trajectory, collected by sensor 106 as described above, and determining acceleration at multiple points along the trajectory (302). Collecting (302) may include acquiring data from another computer / system or local memory to which the data is actively pushed, or acquiring (302) may include passively receiving data on an ongoing basis. It should be noted that in some embodiments, sensor 106 may capture ball velocity data instead of position data. Regardless of the type of data captured by sensor 106, the data can be converted to acceleration data because the ball's acceleration at any given point along its trajectory can be determined as either the first derivative of the ball's velocity or the second derivative of the ball's position.

[0044] The jagged curves in FIG. 4 show example acceleration data calculated for the ball at multiple positions along its trajectory based on position observations by the sensors 106. As can be seen in FIG. 4, in the illustrated embodiment, there are 30 recorded observation points per second, i.e., the ball's position is captured 30 times per second, or, in other words, at a sampling frequency of 30 Hz. Again, it should be noted that this is merely an example, and other sensors 106 may have other sampling frequencies. For clarity, it should be noted again that the concept of acceleration used in this description refers to a 3D vector quantity, i.e., direction and magnitude, and not simply a quantity, as is often the case in colloquial usage. Thus, graph 402 shown in FIG. 4 shows the measured magnitude of the ball's acceleration (vertical axis) versus time (horizontal axis). The next graph 404 shows the measured acceleration in the horizontal direction (i.e., along the x-axis) versus time. The next graph 406 shows the measured acceleration in the vertical direction (i.e., along the y-axis) versus time. Finally, graph 408 at the bottom of Figure 4 shows the measured acceleration along the depth direction (i.e., along the z-axis) versus time. That is, graphs 404-408 collectively show the acceleration components of the ball along three orthogonal axes. These three components, along with measured magnitude 402, form the total acceleration of the ball.

[0045] As can be seen by the jagged lines in Figure 4, this data can be very noisy, especially in the z-direction (i.e., graph 408), and is not very useful by itself for purposes of calculating the spin axis. Therefore, the sensor data is smoothed (304), for example, by applying a filter to the sensor data, resulting in the smooth curve shown in Figure 4. In one embodiment, an SG filter is applied. For the SG filter, the base polynomial can be of a low order, such as a quadratic or cubic polynomial, and the size of the window can depend on the sampling frequency and the amount of noise (i.e., a higher sampling frequency and / or a greater amount of noise requires a larger window).

[0046] However, there are many other alternatives to SG filters that are also available, such as smoothing splines, polynomial least-squares fits, etc. The basic criterion for selecting an appropriate filter is to achieve a good compromise between reducing noise in the signal and also ensuring that the signal itself is not lost, for example, when a high-order polynomial fit is used. Furthermore, because spin axis calculation results typically need to be displayed to the user on the client computer 110 with "low latency," it is desirable to use a method that is not too complex. In practice, "low latency" in this context means that all spin axis calculations should be completed within approximately 0.5 seconds, which may affect the choice of which model to use. Regardless of which model is used, the end goal is the same: to reduce noise in the observations and allow the spin axis to be determined so that an optimized model can be applied to the received data.

[0047] Even after smoothing, outliers may still affect the data. As a simplified illustration of this phenomenon, consider a time window in which observations [1, 10, 1, 1] occur. It is clear that the value 10 is an outlier in this situation, and that a more accurate representation of the observation during the time window is the value 1. Calculating the mean over the time window yields 3.25, which does not reflect the observation 1 very well. That is, outliers still have an undesirable effect on the smoothed data. Therefore, before proceeding, such outliers are removed (306). Various criteria can be used to determine which outliers to remove. For example, in some implementations, outliers with excessively high accelerations (e.g., several g's) are removed. In some implementations, outlier removal may be based on the standard deviation from the mean measurement (the so-called Z-score), to mention a few possible criteria by way of example.

[0048] Next, the wind speed is obtained (308). As explained in more detail below, knowing the wind speed is important so that it can be subtracted from the ball's velocity when determining the ball's relative speed with respect to the air. Being able to do so avoids systematic errors due to the wind, which can lead to overestimating or underestimating the length and trajectory of the trajectory. It should be noted that zero systematic error is not the same as zero error, but rather means that the estimate is not biased in any particular direction.

[0049] In some embodiments, the wind speed is obtained from external, dedicated wind measurement equipment. In some embodiments, the wind vector can be estimated by calculating the wind speed for each of several individual shots during a predefined time window and determining a weighted average of the overall wind speed. Using several shots eliminates the uncertainty associated with calculating wind speed based on individual shots and provides a more robust value of the wind vector.

[0050] If wind speed cannot be obtained using any of the above methods, wind speed can instead be estimated using other techniques. One such technique involves a two-stage wind estimation technique, where a first stage is used to determine whether "strong" or "weak" wind is present, followed by a second stage in which the actual wind is estimated and the "high" or "low" value from the first stage is used as a parameter during the optimization process. Method 500 shown in FIG. 5 illustrates an example implementation of the first stage.

[0051] First, a spin optimization problem is solved (502) using only a subset of the ball trajectory data recorded by the sensor 106, without a wind estimate. The optimization equations are described below with respect to items 310-320 in Figure 3, and the reader is referred to that section of this specification for details on spin determination. The reason for using only a subset of the sensor data is to minimize the impact that other errors may have on the wind speed estimate. In other words, a much simpler optimization problem is solved. Through experimentation, the inventors have found that, particularly for golf shots, approximately the first 1.5 seconds of observed data of the trajectory is an appropriate criterion for selecting the subset. For other sports objects, the time period for selecting the subset may be different. If the subset is too large, there will be no systematic error due to wind, but on the other hand, a sufficient amount of captured data is also required. However, it should be noted again that this is merely an example value, and there may be many factors that affect how the subset of measurement data is selected. The solution to this optimization problem includes initial conditions for the spin axis and spin magnitude, which represent conditions in the absence of wind.

[0052] The estimated spin magnitude is then integrated 504 from this initial state, i.e., initial position and initial spin parameters, to a reference state (the value of the spin axis remains at its initial state). In general, the reference state is preferably in the latter part of the orbit, preferably as late as possible, as this provides more data from which systematic errors can be determined.

[0053] The velocity difference between the integrated position / velocity of the ball and the observed position / velocity of the ball in the reference state is then determined (506). This velocity difference is considered an estimate of the systematic error in velocity due to wind, i.e., a rough estimate of wind speed. The difference can be determined with respect to either the position or the velocity of the ball. If determined with respect to position, the distance is divided by the time over which the integration was performed to express the difference with respect to velocity.

[0054] Finally, the estimated wind speed is converted 508 to a scalar in the range [0,1], where 0 represents "calm" and 1 represents "strong wind." This scalar is then used in the second stage of the wind estimation process, which occurs as part of the optimization process. In some embodiments, a minimum scalar value is set, e.g., "this value will never be less than x." This minimum value acts as a safety net for situations where, even in the presence of strong winds, the shot is so short that systematic error due to wind cannot be determined. Through experimentation, the inventors have found that a suitable value for x for golf shots is 0.3, and that different values ​​of x may be more appropriate for other sports objects.

[0055] Returning now to Figure 3, after the wind speed is received (308) by one of the described methods, an optimization objective and a set of optimization parameters are defined (310) so that the optimization problem can be solved. In the embodiment described with respect to Figure 3, the optimization problem is defined such that a physical model of the golf ball is used to estimate the 3D acceleration of the ball, given the unknown optimization parameters of spin magnitude, spin axis, and wind vector, and using the 3D acceleration calculated by low-pass filtering described in act 304 as the optimization objective.

[0056] Noise in the measurements and the accuracy of the underlying physical model of ball behavior (which makes certain simplifying assumptions) pose many challenges to real-world data, some of which are managed as described above with respect to the operation of 304 and 306, but not all. In particular, the shape and structure of the loss function is important, and the inventors have determined through experimentation that the L2-squared loss function commonly used in many optimization problems typically does not perform well on real-world data collected by sensors 106.

[0057] Instead, the loss function used in the embodiment of FIG. 3 is a Huber loss function, which is commonly used in regression analysis and is less sensitive to outliers than an L2-squared loss function. Additionally, a weight is assigned to each data point (i.e., a point in time at which trajectory data is captured for the ball). Outliers are assigned a lower weight and therefore have less impact on the estimated parameters. This is similar to the iteratively reweighted least squares method also used in regression analysis techniques. Specific details of how these weights are updated are described below, particularly with respect to operation 318. The optimization model may also use regularization terms that penalize spin damping and wind strength. Regularization can be understood as a process of altering the resulting answer to make it "simpler" in some sense. For example, the regularization term could use a wind estimate calculated as described above with respect to FIG. 5.

[0058] Furthermore, in the exemplary embodiments described herein, the optimization methods are descent-based, i.e., they use the derivative of the loss function to determine in which direction the optimization parameters should be changed to minimize the loss function. Given the external, player-driven need for spin axis determination to be perceived as essentially instantaneous, the calculation of the derivative needs to be fast and accurate. For that reason, the derivative can be calculated using automatic differentiation (also known as algorithmic differentiation, computational differentiation, or autodifferentiation). Essentially, automatic differentiation can be understood as a set of techniques for evaluating the derivative of a function specified by a computer program by taking advantage of the fact that even the most complex computer program performs a series of basic arithmetic operations (addition, subtraction, multiplication, division, etc.) and basic functions (exponential, logarithm, sine, cosine, etc.). Thus, by repeatedly applying the chain rule to these operations, derivatives of any order can be calculated automatically and accurately to working precision. This calculation involves at most a fixed scale of arithmetic operations compared to those used in the original program. This is useful when considering external constraints for optimization methods that are perceived as operating in real time. The use of derivatives in the loss function also means that the physical model used in the calculations needs to be differentiable. The parts of the physical model that are not "naturally" differentiable are the measurements of the drag and lift coefficients. Therefore, surface fitting and function approximation can be applied to make these functions differentiable.

[0059] There are many possible options for choosing which optimization method to use. A common method for solving unconstrained nonlinear optimization problems such as those described herein is the iterative Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. The BFGS algorithm determines the descent direction by preconditioning the gradient using curvature information. In the embodiment described herein, a specific variant of the BFGS algorithm is used, known as the L-BFGS algorithm, which uses a strong Wolfe line search as the optimizer. The L-BFGS algorithm approximates the BFGS algorithm using a limited amount of computer memory and is therefore a common algorithm for parameter estimation in machine learning.

[0060] Mathematically, in some embodiments, the optimization problem may be defined as follows:

[0061]

number

[0062] pa = constant where d is the Huber loss function. Unknown:a m (t), a d , v m (t,h) Definition: a m (t): The magnitude of the spin, a function of time. Any function is possible, a linear function is just an example. a d : spin vector (unit vector), constant across the orbit. ·v m(t,h): Wind vector. This can be described by many different functions, but is generally a function of time and altitude. The most basic form is that it is constant over time and altitude. More advanced forms have magnitude that increases with altitude (but magnitude is constant at the same altitude throughout the shot), but direction that is constant. · v(t): Ball velocity. ·v rel (t): The speed of the ball relative to the air. ·K: A constant related to the mass of the ball, area, air density, etc. · CD(v,a), CL(v,a): Drag and lift coefficients of the ball as a function of velocity and spin. ·

[0063]

number

[0064] :Modeled drag acceleration. ·

[0065]

number

[0066] :Modeled lift acceleration. ·

[0067]

number

[0068] : Modeled total acceleration. · a(t): observed total acceleration. · L: loss function. ·w t :The weight of the loss function at this point. L r : A regularization term in the loss function, which helps guide the optimizer to a desired solution. ·p w: Wind penalty term. ·w rough : Rough wind estimate. obs dist : The length of the observed portion of the orbit. The more measurements you have, the more wind you want to allow for. ·t r ef,h r ef: Where to normalize the parameters. Typically, t r ef=0 and h r ef=10

[0069] After the optimization goal and optimization parameters are determined, the parameters are initialized 312. Through experimentation, the inventors have found that for a particular example golf ball, appropriate initial parameter values ​​are zero values ​​for spin axis (i.e., pure backspin) and zero values ​​for wind (i.e., no wind), with the spin magnitude being less influential than the other parameters and can be chosen arbitrarily.

[0070] Next, a normalized weighted loss function, as described above, is calculated (314). The optimizer is then called m times using the L-BFGS optimization model described above (316). The weights assigned to the data points are then updated (318). In some embodiments, the weights are updated using the L1 norm of the optimization objective. The weight of a data point is defined as the inverse of the L1 norm between the target value and the estimated value of the data point. Operations 314-318 are then repeated a total of n times. It should be noted that it is not necessary, of course, to continue iterating simply to reach m or n numbers, respectively; instead, the method can terminate early if the optimizer is determined to have converged, e.g., if the norm of the gradient falls below a certain threshold or if the difference in the objective function between two iterations falls below a certain value. In the embodiments described herein, the values ​​of m and n are determined empirically based on the quality of the data from the sensors and the available computational power.

[0071] After completing n operations 314-318, estimates of the unknown optimization parameters, namely, spin axis, spin magnitude, and wind vector, exist, which can be used for various purposes. For example, the spin axis can optionally be stored, used, or presented in various types of applications (320), such as allowing a user to benefit from the spin axis information to improve their game by, for example, being provided with statistics and / or suggestions for different equipment to use, which concludes method 300.

[0072] Spin axis determination using the systems and techniques described herein can improve the robustness and accuracy of spin axis determination compared to what is currently possible, or enable the use of different types of sensors that would otherwise be too noisy to use in the context of spin axis calculations. Such sensors tend to be simpler, more affordable, and at the same time more robust, which allows for more widespread use than what is currently possible. In addition, the spin axis can be determined so quickly that a user perceives the determination as instantaneous.

[0073] Because small differences, such as one or two degrees, in spin axis calculations can make a big difference in determining ball trajectory, the optimization methods described herein can have a significant impact on the accuracy of spin axis calculations and, therefore, improve the accuracy of ball trajectory determination. The techniques described herein improve the capabilities of computers in efficiently providing players with more accurate information about ball trajectory than is currently possible, both in outdoor environments and indoor environments, such as gaming facilities, where space constraints may require trajectories to be simulated. Spin axis information can be presented to players in a variety of formats. For example, some players may want to know numerical information, while others may be satisfied with knowing whether a shot was a slice, fade, draw, hook, etc. However, other players may want to see graphical information on a display screen outlining the shot. Ultimately, the information provided can improve a player's skill, which not only increases the enjoyment of the game but also increases the safety of those sharing common spaces with the player. Spin axis information can also be used to evaluate how well different types of equipment (e.g., balls, clubs, etc. made from different materials and designs) perform under different conditions, thus ultimately contributing to the development of better equipment.

[0074] The present invention also relates to computer software functions for determining the spin axis in accordance with the above description. Such computer software functions are arranged, when executed, to perform the above-described measuring, selecting, determining, using, calculating, repeating, selecting, and using operations, as also set forth in the claims. The computer software functions are arranged to run on the physical or virtual hardware of the computer 102 of the system and / or data processing device 200, as described above.

[0075] Embodiments of the subject matter and functional operations described herein can be implemented in digital electronic circuitry, or computer software, firmware, or hardware, or a combination of one or more of them, including the structures disclosed herein and structural equivalents thereof. Embodiments of the subject matter described herein can be implemented using one or more modules of computer program instructions encoded on a non-transitory computer-readable medium for execution by or to control the operation of a data processing apparatus. The computer-readable medium can be a hard drive in a computer system or an optical disk sold through retail channels, or an article of manufacture such as an embedded system. The computer-readable medium can be obtained separately and later encoded with one or more modules of computer program instructions, such as by distribution of one or more modules of computer program instructions over a wired or wireless network. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.

[0076] The term "data processing apparatus" encompasses all apparatuses, devices, and machines for processing data, including, by way of example, a programmable processor, computer, or multiple processors or computers. In addition to hardware, an apparatus can include code that creates an execution environment for the computer program in question, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a runtime environment, or one or more combinations thereof. In addition, an apparatus can use various computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0077] A computer program (also known as a program, software, software application, script, or code) can be written in any suitable form of programming language, including compiled or interpreted, declarative or procedural, and can be deployed in any suitable form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored within part of a file that holds other program data or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple cooperating files (e.g., files storing one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0078] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0079] Processors suitable for executing a computer program include, by way of example, both general-purpose and special-purpose microprocessors, as well as one or more processors of any suitable digital computer. Typically, a processor receives instructions and data from a read-only memory (ROM) or a random-access memory (RAM), or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices, e.g., magnetic, magneto-optical, or optical disks, for storing data, or is operatively coupled to receive data from or transfer data to them, or both. However, a computer need not have such devices. Furthermore, a computer can be incorporated into another device, e.g., a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name a few. Devices suitable for storing computer program instructions and data include, by way of example, all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices such as EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, CD-ROM and DVD-ROM disks, network-attached storage, and various forms of cloud storage. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0080] To provide for user interaction, embodiments of the subject matter described herein can be implemented on a computer having a display device, e.g., an LCD (liquid crystal display), OLED (organic light emitting diode), or other monitor, for displaying information to the user, and a keyboard and pointing device, e.g., a mouse or trackball, by which the user can provide input to the computer. Other types of devices can also be used to provide for user interaction; for example, feedback provided to the user can be sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in various forms, including acoustic input, voice input, or tactile input.

[0081] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communications network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. Embodiments of the subject matter described herein may be implemented in a computing system that includes back-end components, e.g., data servers, or middleware components, e.g., application servers, or front-end components, e.g., client computers having a graphical user interface or web browser through which a user can interact with an implementation of the subject matter described herein, or any suitable combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any suitable form or medium of digital data communication, e.g., a communications network. Examples of communications networks include local area networks (“LANs”) and wide area networks (“WANs”), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0082] While this specification contains details of many implementations, these should not be construed as limitations on the scope of the invention or what may be claimed, but rather as descriptions of features specific to implementations of the invention. Certain features described herein in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as working in a particular combination, and may even initially be claimed as such, one or more features from a claimed combination may, in some cases, be excluded from that combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination. Thus, unless expressly stated otherwise or unless the knowledge of one of ordinary skill in the art clearly dictates otherwise, any feature of an embodiment described above can be combined with any other feature of an embodiment described above.

[0083] Similarly, while acts are shown in a particular order in the figures, this should not be understood as requiring such acts to be performed in the order shown or sequential order, or that all of the shown acts be performed, to achieve desirable results. In certain situations, multitasking and / or parallel processing may be advantageous. Furthermore, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments; the described program components and systems may be integrated into a single software product or packaged into multiple software products.

[0084] Thus, embodiments of the present invention have been described. Other embodiments are within the scope of the following claims. For example, while the above description focuses on determining spin axis in the context of golf ball movement, the described systems and techniques are applicable to other types of objects that move through the air and are affected by wind, such as baseball or skeet shooting, as well as non-sports applications. Spin axis determination can be further improved by improving wind speed estimation, for example, by considering that wind varies with ground height. Furthermore, in the above examples, the ball's acceleration was assumed to be affected only by gravity, lift, and drag. However, there may be other forces at work, such as some type of Coriolis effect, which can be used to further improve the physics model. The improvements that can be achieved by such changes to the physics model need to take into account the accuracy of data registered by sensors, wind measurements / estimates, etc., which may have a greater overall impact on the resulting spin axis determination.

[0085] It should also be noted that the techniques described herein are applicable to setups that use one or more sensors to observe the ball's trajectory. For example, a single sensor (such as a Doppler radar) can be used to observe both angle and distance, and therefore no additional sensors are required. Other setups may use various combinations of cameras and radar sensors.

[0086] Additionally, it should be noted that while the various embodiments described herein refer to a broad class of descent-based optimization methods, there may be other optimization methods that can be used that do not fall within this family of optimization methods. The various embodiments described above refer to "automatic differentiation." However, it should be understood that embodiments that use numerical values ​​may also exist. Thus, many variations on the above examples are well within the scope of the appended claims and within the ability of one of ordinary skill in the art. [Explanation of symbols]

[0087] 102 Computer 104 Network 106 Sensors 108 Physical Model 110 client computers 200 Data processing device 212 Hardware Processors 214 Additional Devices 216 Non-transitory computer-readable medium 218 Communication Interface 220 User Interface Device, Local Device 230 Programs 280 Networks, Communication Networks 290 Computers, User Interface Devices 402 Graph, size 404 graphs 406 graphs 408 graphs 602 Measurement trajectory 604 Estimated trajectory

Claims

1. 1. A method for determining the spin axis of a ball in flight, comprising: acquiring at least a portion of an observed trajectory of the ball, the observed trajectory including a three-dimensional position of the ball over time determined from individual observations of the ball by one or more sensors; selecting a point in three-dimensional space; solving an optimization problem, wherein three-dimensional acceleration at the selected point in three-dimensional space is used as an optimization objective, and spin axis, spin magnitude, and wind vector are used as optimization parameters, and a solution to the optimization problem is a combination of optimization parameter values; selecting the spin axis determined in the solution of the optimization problem as the spin axis of the ball; using the selected spin axis of the ball to generate ball trajectory information to be presented on an output device; A method comprising:

2. The steps to solve the optimization problem are initializing the values ​​of the optimization parameters; Calculating a normalized weighted loss function; optimizing m steps using descent-based optimization; updating the weights of the points; repeating said calculation, optimization, and weighting n times or until convergence; 2. The method of claim 1, comprising:

3. The method of claim 2 , wherein the one or more sensors include a radar, a camera, or both.

4. The method of claim 2 , further comprising smoothing at least a portion of the observed trajectory of the ball using a Savitzky-Golay filter.

5. The method of claim 2 , further comprising removing acceleration outliers before solving the optimization problem.

6. obtaining a wind speed representative of prevailing wind conditions during the flight of the ball; using the obtained wind speed when initializing the wind vector; 6. The method of claim 2, further comprising:

7. The method of claim 6 , wherein the wind speed is obtained by estimating a systematic error between corresponding portions of the estimated trajectory of the ball and the observed trajectory of the ball.

8. 6. The method of claim 2, wherein the three-dimensional acceleration of the ball at the selected point in three-dimensional space includes an acceleration component due to gravity, an acceleration component due to drag, and an acceleration component due to lift, and the acceleration component due to lift is used as the initial value of the spin axis.

9. The method of claim 8 , wherein the ball is a golf ball and the method is performed at a golf driving range or golf course.

10. 10. The method of claim 9, wherein the normalized weighted loss function is a Huber loss function.

11. determining whether the optimization has converged; responsive to determining that the optimization has converged, performing the iterative operation less than n times; 11. The method of claim 10, further comprising:

12. The method of claim 11 , wherein the step of calculating the normalized weighted loss function is performed by automatic differentiation.

13. The method of claim 12 , wherein the descent-based optimization is the Broyden-Fletcher-Goldfarb-Shanno algorithm.

14. A computer software product configured, when executed, to perform the method steps of any one of claims 1 to 13.

15. 1. A system for determining the spin axis of a ball in flight, comprising: means for obtaining at least a portion of an observed trajectory of the ball, the observed trajectory including a three-dimensional position of the ball over time determined from individual observations of the ball by one or more sensors; means for selecting a point in three-dimensional space; means for solving an optimization problem, wherein three-dimensional acceleration at the selected point in three-dimensional space is used as an optimization goal, and spin axis, spin magnitude, and wind vector are used as optimization parameters, and a solution to the optimization problem is a combination of optimization parameter values; means for selecting the spin axis determined in the solution of the optimization problem as the spin axis of the ball; means for using the selected spin axis of the ball to generate ball trajectory information to be presented on an output device; and Including, the system.

Images