Spin estimation of marked objects

Abstract

To provide a spin estimation system that may include an image capture sensor positioned and configured to capture an image of an object within a field of view of the image capture sensor.SOLUTION: A spin estimation system may be configured to perform one or more operations to analyze spin characteristics of an object. The operations may include setting an image capture frame rate corresponding to a minimum spin motion of the object, printing an orientation marker on an exterior surface of the object, and capturing an image of the object after the object begins to move with an image capture sensor at the set image capture frame rate. The operations may include isolating the object in each image to generate a separated object image. The operations may include generating, on the basis of the separated object image, an object marker segmentation map. A spin rate and a spin axis can be estimated on the basis of the object marker segmentation map, using deep learning technique.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] TECHNICAL FIELD This disclosure relates generally to spin estimation of marked objects. [Background technology]

[0002] A moving object can be described by its properties, such as its position, launch velocity, launch angle, spin rate, spin axis, and other environmental factors. The moving object may include objects used in sports, such as a ball. Evaluating the properties of the moving object can provide information about the drag and lift forces acting on the moving object, which can provide insight into the flight path of the moving object.

[0003] The subject matter claimed in this disclosure is not limited to embodiments that solve any shortcomings or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one example technology area where some embodiments described in this disclosure may be practiced. Summary of the Invention

[0004] According to one aspect of an embodiment, a spin estimation system may include an image capture sensor positioned and configured to capture images of an object within a field of view of the image capture sensor. The spin estimation system may be configured to perform one or more operations to analyze spin characteristics of the object. The operations may include setting an image capture frame rate corresponding to a minimum spin motion of the object, printing orientation markers on an outer surface of the object, and capturing images of the object after the object begins to move with the image capture sensor at the set image capture frame rate. The operations may include separating the object in each image to generate a separated object image. The operations may include generating an object marker segmentation map based on the separated object images. The spin rate and spin axis may be estimated based on the object marker segmentation map.

[0005] The object and advantages of the embodiments will be realized and attained at least by the elements, features, and combinations particularly pointed out in the claims. It is to be understood that both the foregoing general description and the following detailed description are explanatory only and are not restrictive of the invention, as claimed. [Brief explanation of the drawings]

[0006] Example embodiments will be described and explained with additional specificity and detail through the accompanying drawings. [Figure 1A] 1 illustrates an example embodiment of an environment including a spin estimation system, in accordance with at least one embodiment of the present disclosure. [Figure 1B] 1 illustrates a second diagram of an example embodiment of an environment including a spin estimation system, in accordance with at least one embodiment of the present disclosure. [Figure 2] 1A and 1B illustrate a back view and a front view of a spin estimation system in accordance with at least one embodiment of the present disclosure. [Figure 3A] 1 illustrates a three-dimensional model of an object observed by a spin estimation system and a three-dimensional cuboid model corresponding to the object, in accordance with at least one embodiment of the present disclosure. [Figure 3B] 1 illustrates a textured cuboid based on a 3D cuboid model, in accordance with at least one embodiment of the present disclosure. [Figure 3C] 10 illustrates an open marker texture based on a textured cuboid, in accordance with at least one embodiment of the present disclosure. [Figure 3D] 1 illustrates an individual marker pattern in accordance with at least one embodiment of the present disclosure. [Figure 4A] 1 illustrates an example of a textured object, in accordance with at least one embodiment of the present disclosure. [Figure 4B] 1 illustrates an example of a binary map corresponding to a textured object, in accordance with at least one embodiment of the present disclosure. [Figure 4C]1 illustrates operations that may be performed to enhance a given image of a textured object, in accordance with at least one embodiment of the present disclosure. [Figure 5] FIG. 1 is a flow diagram of operations associated with a spin estimation system in accordance with at least one embodiment of the present disclosure. [Figure 6] FIG. 1 is a flow diagram of a neural network configured to perform operations associated with a spin estimation system, in accordance with at least one embodiment of the present disclosure. [Figure 7] FIG. 1 is a flow diagram of a refiner algorithm included in a spin estimation system in accordance with at least one embodiment of the present disclosure. [Figure 8] FIG. 1 is a flow diagram of an image enhancement algorithm included in a spin estimation system, in accordance with at least one embodiment of the present disclosure. [Figure 9] 1 is a flowchart of an example method for performing spin estimation, in accordance with at least one embodiment of the present disclosure. [Figure 10] 1 is an exemplary computer system. DETAILED DESCRIPTION OF THE INVENTION

[0007] Measuring the trajectory of an object during flight in three-dimensional space may depend on motion-related characteristics, such as launch velocity, launch angle, spin rate, spin axis, or any other ballistic characteristic, of the object during flight, and / or environmental factors. Forces acting on an object during flight, such as drag or lift, may be affected by the object's motion-related characteristics. Accurate measurement of such motion-related characteristics can therefore facilitate improved estimation and / or modeling of the object's motion. However, objects may move at high speeds, which makes it more difficult to capture accurate details about the object and measure the object's motion-related characteristics in real time.

[0008] The present disclosure relates, among other things, to systems and methods for measuring an object's spin axis and spin rate using images captured of the object during flight. A predetermined marker pattern can be applied to the object so that the object's orientation from any given viewpoint is unambiguous. Because the spin rate and spin axis of a given object depend on analyzing the object from two or more different orientations, measuring the spin rate and spin axis requires at least two images covering different views of the object. A spin estimation system according to the present disclosure may include generating one or more object marker segmentation maps, and pairs of object marker segmentation maps may be used to independently provide spin rate and spin axis estimates in certain instances. The independently provided spin rate and spin axis estimates may be combined to generate a single spin rate-axis estimate. Additionally or alternatively, a three-dimensional environment may be generated to simulate object flight using different launch parameters, and the simulated object flight parameters may be used as a training dataset for a deep learning model configured to perform spin rate-axis estimation. To improve the estimation results by increasing the estimation accuracy, post-processing enhancement algorithms can be applied to the images and / or spin rate-axis estimates of the flying object.

[0009] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0010] 1A and 1B illustrate an exemplary embodiment of an environment 100 including a spin estimation system 110 in accordance with at least one embodiment of the present disclosure, with FIG. 1A illustrating a first view of the environment 100 and FIG. 1B illustrating a second view of the environment 100. The spin estimation system 110 may include a camera 112 or other image capture sensor positioned and configured to capture images within a field of view, such as a field of view bounded by a first line 114 and a second line 116 as shown in FIGS. 1A and 1B.

[0011] In some embodiments, a force may be applied to the object 120 that causes the object 120 to move along the trajectory 122. As shown between FIGS. 1A and 1B , the actor 102 may apply a force to the object 120 by swinging the club 104, causing the object 120 to move along the trajectory 122 toward a destination 124. While the environment 100 is shown as including the actor 102 and the club 104, the environment may or may not include the actor 102 or the club 104. For example, the camera 112 of the spin estimation system 110 may be positioned to capture images of the object 120 during flight. In other words, the camera 112 may be positioned such that the first line 114 and the second line 116 are oriented such that the camera 112 captures images along a portion of the trajectory 122.

[0012] FIG. 2 shows a back view 200a and a front view 200b of a spin estimation system 110 in accordance with at least one embodiment of the present disclosure. In some embodiments, the spin estimation system 110 may include a camera 112, which may be any device, system, component, or collection of components configured to capture an image. The camera 112 may include optical elements, such as lenses, filters, holograms, splitters, or any other components, and an image sensor by which an image may be recorded. Such an image sensor may include any device that converts an image represented by incident light into an electrical signal. The image sensor may include multiple pixel elements, which may be arranged in a pixel array (e.g., a grid of pixel elements). For example, the image sensor may include a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) image sensor. The pixel array may include a two-dimensional array having an aspect ratio such as 1:1, 4:3, 5:4, 3:2, 16:9, 10:7, 6:5, 9:4, 17:6, or any other ratio. The image sensor may be optically aligned with various optical elements, e.g., lenses, that focus light onto a pixel array, which may include any number of pixels, such as 8 megapixels, 15 megapixels, 20 megapixels, 50 megapixels, 100 megapixels, 200 megapixels, 600 megapixels, 1000 megapixels, etc.

[0013] Various other components may also be included in image capture sensors 120 and / or 130. Such components may include one or more lighting features, such as a flash or other light source, a light diffuser, or other components for illuminating an object. In some embodiments, the lighting features may be configured to illuminate a moving object when the moving object is in proximity to the image sensor, for example, when the moving object is within 3 meters of the image sensor.

[0014] In these and other embodiments, camera 112 may be configured to operate at or above a minimum frame rate threshold, with the minimum frame rate threshold set based on the spin rate range of object 120. For example, the spin rate of a golf ball in a typical golf game may range from 500 revolutions per minute (RPM) to 12,000 RPM, and any camera operating at a frame rate below 200 frames per second (FPS) may not be able to properly capture an image of object 120 if object 120 spins at or near the upper end of the spin rate range due to aliasing effects.

[0015] The object 120 may spin at different spin rates depending on the application or setting in which the object 120 and the camera 112 are operating. For example, the object 120 may spin at a higher spin rate if a mechanical device is used to affect the motion of the object 120, which may indicate that the camera used to capture images of the object 120 should operate at a higher frame rate. In situations in which the object 120 rotates at a rate higher than the capture frame rate of the camera 112, aliasing effects may occur if the object 120 rotates more than 360° between image captures by the camera 112. For example, a given object rotated 40° may appear identical to a given object rotated 400°. Therefore, the camera 112 may be selected or configured to capture images at a frame rate faster than the maximum predictable spin rate of the object 120.

[0016] In some embodiments, spin estimation system 110 can include or be communicatively coupled to a computer system configured to process and analyze images captured by spin estimation system 110. For example, the computer system may perform operations such as those described in further detail in connection with FIGS.

[0017] Modifications, additions, or omissions may be made to environment 100 or spin estimation system 110 without departing from the scope of the present disclosure. For example, the designations of different elements in the described methods are meant to help explain the concepts described herein and are not limiting. Furthermore, environment 100 or spin estimation system 110 may include any number of other elements or may be implemented in other systems or contexts other than those described.

[0018] 3A illustrates a three-dimensional model 300 of an object that may be observed by a spin estimation system and a corresponding object mesh model 306 for the object, in accordance with at least one embodiment of the present disclosure. In some embodiments, the three-dimensional model 300 may be generated based on an object typically observed by the spin estimation system based on one or more given environments in which the spin estimation system is likely to be deployed. For example, if the spin estimation system is likely to be deployed to analyze the spin characteristics of a golf ball in a golf environment, the three-dimensional model 300 may be shaped and sized based on a golf ball. As an additional or alternative example, the three-dimensional model 300 may be shaped and sized based on a soccer ball, a baseball, or any other spherical or spheroidal object.

[0019] A 3D cuboid model 302 may be generated based on the 3D model 300, where the 3D cuboid model 302 is sized to enclose the 3D model 300 as an enclosed object model 304. The enclosed object model 304 may be the basis for an object mesh model 306.

[0020] Although three-dimensional model 300 is shown as a spherical model, three-dimensional model 300 may additionally or alternatively be modeled as an oval or other rounded three-dimensional shape. In these and other embodiments, three-dimensional cuboid model 302 may be shaped and sized to correspond to an oval rather than a sphere as shown in FIG. 3A. For example, three-dimensional cuboid model 302 may include one or more sides that are longer or shorter than the other sides.

[0021] 3B shows a textured cuboid 308 based on object mesh model 306 in accordance with at least one embodiment of the present disclosure. In some embodiments, textured cuboid 308 may be generated such that marker patterns 310, 312, and 314 may be printed on the outer surface of object mesh model 306. Additionally or alternatively, textured cuboid 308 may include a fourth marker pattern, a fifth marker pattern, or a sixth marker pattern not shown on textured cuboid 308. Each of marker patterns 310, 312, and 314 may be rotationally asymmetric and may be a different pattern from each of marker patterns 310, 312, and 314, such that marker patterns 310, 312, and 314 are visually distinct from one another regardless of the orientation of textured cuboid 308.

[0022] 3C shows an unfolded marker texture 320 based on a textured cuboid 308, in accordance with at least one embodiment of the present disclosure. The unfolded marker texture 320 may include one or more marker patterns 321-326 that are asymmetrically different from one another. In some embodiments, the unfolded marker texture 320 may be a template for forming the textured cuboid 308, such as by folding the unfolded marker texture 320 to form a cube or cuboid shape.

[0023] In some embodiments, the open marker texture 320 can be resized, or one or more of the marker patterns 321-326 can be modified to adjust the open marker texture 320 to different object shapes and dimensions. Designing, scaling, and applying the marker patterns 321-326 to a given object can be easier and more understandable using the open marker texture 320 than applying one or more of the marker patterns 321-326 directly to the round surface of the object. Additionally or alternatively, generating the marker patterns 321-326 using the open marker texture 320 may be incorporated into existing marker printing processes, which typically use planar printing devices.

[0024] 3D shows individual marker patterns 321-326 in accordance with at least one embodiment of the present disclosure. In some embodiments, each of the marker patterns 321-326 may be formed within a pixel 330 having a unit dimension that may or may not be based on the dimensions of the object on which the marker patterns 321-326 may be printed. To form the marker patterns 321-326, one or more dots 335 may be placed within the pixel 330. The pixel 330 may be rotated to evaluate the rotational symmetry of each of the marker patterns 321-326.

[0025] 4A illustrates an example of a textured object 400 from a first perspective 402, a second perspective 404, a third perspective 406, and a fourth perspective 408, in accordance with at least one embodiment of the present disclosure. The textured object 400 may include the object 400 described in connection with FIGS. 1A and 1B with one or more marker patterns 410 printed on the outer surface of the textured object 120, which may be the same as or similar to the marker patterns 321-326 described in connection with FIGS. 3B-3D. Each of the marker patterns 410 may be rotationally asymmetric and may differ from the other marker patterns 410 printed on the textured object 400, such that any two perspectives, such as the first perspective 402 and the second perspective 404, may be visually distinct from one another.

[0026] 4B illustrates an example of a first perspective 402, a second perspective 404, a third perspective 406, and a fourth perspective 408 of a binary map corresponding to a first perspective 412, a second perspective 414, a third perspective 416, and a fourth perspective 418, respectively, of a textured object 400, in accordance with at least one embodiment of the present disclosure. In some embodiments, the first, second, third, and fourth perspectives 412, 414, 416, and 418 may be negative images of the first, second, third, and fourth perspectives 402, 404, 406, and 408 of the textured object 400. In these and other embodiments, the negative images corresponding to the first, second, third, and fourth perspectives 412, 414, 416, and 418 may represent binary images, such as a binary map 506 output by a binarization algorithm 504 as described in connection with operation 500 of FIG. 5 . Because the textured object 400 may include a substantially light color (e.g., white), the textured object 400 itself may typically include a high pixel density. In comparison, the marker pattern 410 on the surface of the textured object 400 may include a darker color (e.g., black) to more visually contrast the marker pattern 410 from the remainder of the exterior surface of the textured object 400. Performing a negative operation on the textured object 400 to generate the binary maps shown in the first, second, third, and fourth viewpoints 412, 414, 416, and 418 increases pixel values ​​associated with the portion of the surface of the textured object 400 that corresponds to the marker pattern 410, which can facilitate a more accurate analysis of the location of the marker pattern 410 on the surface of the textured object 400.

[0027] 4C illustrates operations that may be performed to enhance an image 420 of a textured object, in accordance with at least one embodiment of the present disclosure. In some embodiments, the image 420 of the textured object may include an image of the first perspective 402, the second perspective 404, the third perspective 406, or the fourth perspective 408 of the textured object 400 described in connection with FIG. 4A . The image 420 may represent the object image 702 as described in connection with operation 700 of FIG. 7 or the object image 802 as described in connection with operation 800 of FIG. 8 . The image 420 may be enlarged to generate a resized image 421 of the textured object, which may correspond to the resized object 806 described in connection with operation 800 of FIG. 8 . Because the textured object is typically flying or otherwise moving relative to the camera that captured the image 400 of the textured object, the size of the textured object between image frames may be inconsistent. Thus, one or more of the captured images of the textured object may be scaled up or down, such as resized image 421, to match the size of the textured object in each of the frames of the image.

[0028] A negation operation may be applied to the resized image 421 to generate a negative image 422 so that marker patterns 430 containing higher pixel values ​​are more easily analyzed. In some embodiments, the negative image 422 may correspond to a negative object 810 as described in connection with operation 800 of FIG. 8 . In some embodiments, a dilation operation may be applied to the negative image 422 to form a stretched image 423, which may correspond to the stretched object 814 described in connection with operation 800 of FIG. 8 that removes distortion of details along the edges of the negative image 422. Because the dilation operation may result in the formation of visual artifacts 432 in the stretched image 423, a circular masking operation may be applied to filter the visual artifacts 432 and generate a masked image 424 in which the image background 434 has been removed without affecting the clarity of the object and marker pattern 430 in the masked image 424. In some embodiments, the masked image 424 may correspond to a masked object 818 as described in connection with operation 800 of FIG. 8 .

[0029] In some embodiments, a contrast operation may be applied to the masked image 424 to increase the contrast of surface features or marker patterns 430 of the object depicted in the masked image 424. In these and other embodiments, a histogram equalization algorithm may be used to generate a histogram equalized image 425, which may correspond to the histogram equalized object 822, as described in connection with operation 800 of Figure 8. Additionally or alternatively, the histogram equalization algorithm may include an adaptive binarization component that removes detail contained in the histogram equalized image 425 to generate a simplified image 426, which may correspond to the output of an adaptive binarization operation 824, as described in connection with operation 800 of Figure 8.

[0030] The simplified image 426 may be enhanced via an enhancement operation as an enhanced image 427. In some embodiments, the enhanced image 427 may correspond to the enhanced object image 708 as described in connection with operation 700 of FIG. 7 or the enhanced object 828 as described in connection with operation 800 of FIG. 8. A polar transform operation, such as the polar transform 710 described in connection with FIG. 7, may be applied to the enhanced image 427 to identify a rotational reference point for the enhanced image 427 and generate a polar image 428 based on the rotational reference point. In some embodiments, the polar image 428 may correspond to the polar image 712 described in connection with operation 700 of FIG. 7.

[0031] 5 is a flow diagram of operations 500 performed by a spin estimation system related to generating an improved spin rate-axis estimate 526 in accordance with at least one embodiment of the present disclosure. The spin estimation system may be configured to acquire object images 502, as described above in connection with FIGS. 1A, 1B, and 2. The operations 500 may be performed to generate an improved spin axis-rate estimate 526 based on two or more of the object images 502.

[0032] The object images 502 captured or otherwise acquired by the spin estimation system may be used as input to a binarization algorithm 504 to generate a corresponding binary map 506 associated with each of the object images 502. In some embodiments, the binarization algorithm 504 may include performing a resizing operation, a negation operation, a stretching operation, a masking operation, or any other operation on each of the object images 502 to convert the acquired object images 502 into the corresponding binary map 506.

[0033] The binary maps 506 may be organized in chronological or sequential order corresponding to the flight of the object in which the object images 502 were captured. In other words, the binary maps 506 may be sorted so that the object can be observed to retrace its flight path through each of the chronologically or sequentially organized binary maps 506. In the organized series of binary maps 506, each pair of consecutive binary maps may be identified as a forward sequential binary map 508 in response to being paired in the forward direction of the organized series (i.e., in response to being viewed from the first binary map corresponding to the first captured object image to the last binary map corresponding to the last captured object image), or as a reverse sequential binary map 510 in response to being paired in the reverse direction of the organized series. For a given set of object images including N object images 502, the number of forward sequential binary maps 508 may be N-1, and the number of reverse sequential binary maps 510 may also be N-1.

[0034] The forward continuous binary maps 508 and the reverse continuous binary maps 510 may be used as inputs to a rotation estimation algorithm 512 to generate a corresponding spin rate estimate and spin axis estimate 514 for each pair of forward continuous binary maps 508 and each pair of reverse continuous binary maps 510. In some embodiments, the rotation estimation algorithm 512 may be performed using operations the same as or similar to the operations 600 described in connection with FIG. 6. Because N−1 forward continuous binary maps 508 and N−1 reverse continuous binary maps 510 may be input to the rotation estimation algorithm 512, a given spin rate estimate and spin axis estimate may be generated for each pair of continuous binary maps, so the number of spin rate estimates and spin axis estimates 514 may be 2(N−1).

[0035] In some embodiments, the spin rate estimates and spin axis estimates 514 may be passed through an outlier removal algorithm 516 that analyzes each of the spin rate estimates and spin axis estimates 514 and removes, from the set of 2(N-1) spin rate estimates and spin axis estimates 514, any outlier estimate that is inconsistent with the rest of the spin rate estimates and spin axis estimates 514. In these and other embodiments, a given spin rate estimate and spin axis estimate may be considered an outlier estimate if the spin rate or spin axis differs from the respective spin rate or spin axis by more than a given threshold. A large difference between successive spin rate estimate or spin axis estimate components may indicate that one or both of the successive estimate components have been incorrectly calculated, such as due to algorithm calculation errors, insufficient image binarization or other image processing issues, or any other reason. Additionally or alternatively, a particular spin rate estimate and spin axis estimate may be between a first spin rate estimate and spin axis estimate and a second spin rate estimate and spin axis estimate, and may be considered an outlier relative to the first spin rate estimate and spin axis estimate and the second spin rate estimate and spin axis estimate, even though the first spin rate estimate and spin axis estimate and the second spin rate estimate and spin axis estimate are consistent with each other. In these and other situations, the particular spin rate estimate and spin axis estimate may be labeled as an outlier estimate by the outlier removal algorithm 516, but the first spin rate estimate and the second spin rate estimate and spin axis estimate are not labeled as outlier estimates.

[0036] In some embodiments, the number of spin rate and spin axis estimates 518 input to the integrating algorithm 520 may be K, where K is less than or equal to 2(N−1) because one or more of the 2(N−1) spin rate and spin axis estimates 514 may be removed from the dataset by the outlier removal algorithm 516. The K spin rate and spin axis estimates 518 output by the outlier removal algorithm 516 may be input to the integrating algorithm 520 to determine a final spin rate-axis estimate 522, which provides an overall spin rate estimate and an overall spin axis estimate for the object for which the object image 502 was captured.

[0037] The final spin rate-axis estimate 522 and one or more object images 502 may be obtained by a refiner algorithm 524 configured to generate an improved spin rate-axis estimate 526 based on the obtained inputs. In some embodiments, determining the improved spin rate-axis estimate 526 using the refiner algorithm 524 may involve enhancing the obtained object images 502 and determining an update to the spin rate-axis estimate using the final spin rate-axis estimate 522 as a baseline or initial value. The enhancement of the object images 502 and finding the improved spin rate-axis using the refiner algorithm 524 may be performed according to operations the same as or similar to operations 700 described in connection with FIG. 7 .

[0038] Modifications, additions, or omissions may be made to operations 500 without departing from the scope of the present disclosure. For example, the designations of different elements in the described methods are meant to help explain the concepts described herein and are not limiting. Furthermore, operations 500 may include any number of other elements or may be implemented in other systems or contexts other than those described.

[0039] 6 is a flow diagram of the operation of a neural network 600 configured to perform operations associated with a spin estimation system, in accordance with at least one embodiment of the present disclosure. In some embodiments, the spin estimation system may be implemented using deep learning, machine learning, neural networks, or any other artificial intelligence system, such as neural network 600. Using neural network 600 or any other artificial intelligence system in conjunction with the spin estimation system may facilitate determining one or more motion parameters associated with the analyzed object, such as spin rate 642 and spin axis 622.

[0040] Operation of neural network 600 may involve obtaining one or more binary maps 602 corresponding to one or more images of an object being analyzed by neural network 600. In some embodiments, binary map 602 may include the negative image depicted in FIG. 4B and may be the same as or similar to binary map 506 described in connection with operation 500 of FIG. 5. The images included in binary map 602 may be concatenated channel-by-channel to generate multi-channel data 604, which may be obtained by feature extractor neural network 606 as a single stream of input data. In some embodiments, multi-channel data 604 may represent two or more views of an object depicted from two or more consecutive frames, which may include information that facilitates determining the amount of rotation about the object, the axis of rotation, other kinematic parameters, or some combination thereof.

[0041] The feature extractor neural network 606 may be configured to generate a feature map 608 by processing the multi-channel data 604 according to a weight matrix, where elements of the weight matrix may indicate the importance of each element of the multi-channel data 604 for determining a given output, such as spin rate 642 and spin axis 622. In some embodiments, the feature map 608 may be output by the feature extractor neural network 606 as a matrix, which may then be flattened into an N-dimensional feature vector 610. The N-dimensional feature vector 610 may be passed through a fully connected layer 612 of the neural network 600, which may be configured to process the elements of the N-dimensional feature vector 610 and output a four-dimensional (4D) vector including a spin axis component 614 and a rotation component 616. For example, the given 4D vector may include three vector elements related to the spin axis of the given object, representing the spin axis component 614, and one vector element related to the rotation of the given object, representing the rotation component 616, as shown in FIG.

[0042] In some embodiments, the spin axis component 614 may be input to a normalization process 618, which may be configured to output a unit vector 622 representing the estimated spin axis vector. Additionally or alternatively, the rotation component 616 may be input to a scaling process 620, which may be configured to output a rotation estimate 628, measured in radians, ranging from 0 radians to 2π radians.

[0043] In some embodiments, neural network 600 may be trained or improved by determining a mean squared error (MSE) loss between unit vector 622 representing estimated spin axis value and rotation estimate 628 and each set of ground truth spin axis values ​​624 and each set of ground truth rotation values ​​626. Spin axis MSE loss 630 may be calculated based on the difference between unit vector 622 and the corresponding spin axis value included in the set of ground truth spin axis values ​​624, and rotation MSE loss 632 may be calculated based on the difference between rotation estimate 628 and the corresponding rotation value included in the set of ground truth rotation values ​​626. Spin axis MSE loss 630 and rotation MSE loss 632 may be processed in a summation process 634 to calculate a final total loss 636, which may represent the total difference between the estimated value corresponding to unit vector 622 and rotation estimate 628 and the ground truth value corresponding to set of ground truth spin axis values ​​624 and set of ground truth rotation values ​​626, respectively. The final total loss 636 may be sent to a weight update optimizer 638, which may be configured to update the weight matrix applied to the feature extractor neural network 606 or / and the fully connected layer 612, so that the subsequently determined unit vector 622 or rotation estimate 628 may be more accurate and closer to the corresponding ground truth value.

[0044] Modifications, additions, or omissions may be made to neural network 600 without departing from the scope of the present disclosure. For example, the designations of different elements in the described methods are meant to help explain the concepts described herein and are not limiting. Furthermore, neural network 600 may include any number of other elements or may be implemented in other systems or contexts other than those described.

[0045] 7 is a flow diagram of a refiner algorithm 700 included in a spin estimation system in accordance with at least one embodiment of the present disclosure. Refiner algorithm 700 may be implemented to generate an improved spin rate-axis estimate based on one or more acquired object image pairs and an initial spin rate-axis estimate, such as with refiner algorithm 524 described in connection with FIG. 5.

[0046] In some embodiments, one or more object images 702 can be used as input to an image enhancement algorithm 706 to output a corresponding number of enhanced object images 708, which can be performed according to the same or similar operations as operation 800 of FIG. 8. The enhanced object images 708 can be the same as or similar to the enhanced image 427 described in connection with FIG. 4C. A polar transform 710 can be performed on the enhanced object image 708 based on the coarse spin rate-axis estimate 704. The coarse spin rate-axis estimate 704 can be used to find an initial guess for the center of the polar transform 710, which can be represented by coordinates (cx, cy), and the center of the polar transform 710 represents a reference point about which the enhanced image 708 can be rotated. Applying the polar transform 710 to the enhanced image 708 can involve shifting the initial guess for the center by a spacing amount represented by (i, j) and converting the Cartesian coordinates of the enhanced image 708 to polar coordinates. Thus, the enhanced image 708 may be translated and transformed into a polar image 712 .

[0047] Identifying a single improved spin rate-axis estimate 730 may involve combining multiple enhanced images 708 into a single spin rate-axis estimate. Accordingly, a cross-correlation process 714 may be applied to the polar images 712 to determine a correlation result 716 between the polar images 712. The correlation result 716 may provide a quantitative indicator of the similarity between the polar images 712, represented by correlation peaks, that indicates how similar the polar images 712 are based on the height of each of the correlation peaks. Based on the correlation result 716, a peak detection algorithm 718 may be applied to identify the highest correlation peak 720.

[0048] In some embodiments, the peak detection algorithm 718 may include an iterative process in which each iteration applies a different spacing amount, represented as a shift pair (i, j) of the polar transform 710. For example, a shift pair, which may represent a translation distance relative to a given center of a given enhanced image 708, may range between lower boundary i, upper boundary i, lower boundary j, and upper boundary j. In this and other examples, the i value may range from −4 to 4, and the j value may range from −3 to 3, such that a total of 63 combinations of (i, j) shift pairs may be possible. An iterative loop 722 may involve applying each of the 63 combinations of shift pairs as inputs to the polar transform 710. During each iterative loop 722, different correlation results 716 and different correlation peaks 720 may be output based on the difference between the i or j values ​​of the different shift pairs. In these and other embodiments, a particular shift pair and the correlation result 716 and correlation peak 720 corresponding to the particular shift pair may be selected based on the correlation peak 720 corresponding to the particular shift pair having the largest value. In these and other embodiments, iteratively identifying the highest correlation peak 720 may improve results, as the coarse spin rate-axis estimate 704 used in a single iteration of the peak detection process may or may not provide the most accurate results.

[0049] In some embodiments, identification of the highest correlation peak 720 from the iteration loop 722 may be facilitated by a buffer 724. The buffer 724 may be configured to obtain and store the correlation peaks 720 from each iteration and the correlation results 716 associated with each of the obtained correlation peaks 720. For example, a value corresponding to the correlation peak 720, a peak location, or a polar transform center associated with the correlation peak 720 may be stored in the buffer 724.

[0050] The highest correlation peak 720 contained in the buffer 724 may be used in a second peak detection process 726, which involves outputting the correlation peak location 720. In some embodiments, the second peak detection process 726 may facilitate determining the center of a polar transform 728 having a Cartesian coordinate form (cx+i, cy+j). In these and other embodiments, the center of the polar transform 728 may be determined by identifying the coordinate corresponding to the highest correlation peak 720 and shifting the polar image 712 a distance corresponding to the coordinate of the highest correlation peak 720.

[0051] The center of the polar transform 728 and the location of the highest correlation peak (the location of the peak in the correlation result 718) may be input into a spin rate-axis transformation process 730 to determine an improved spin rate-axis 732. In some embodiments, the spin rate-axis transformation process 730 may include mapping the center of the polar transform 728 and the location of the highest correlation peak 720 from a polar coordinate system to a Cartesian coordinate system to identify an improved spin rate-axis 732 that corresponds to a given object in the Cartesian coordinate system. In these and other embodiments, the location of the highest correlation peak 730 in the polar coordinate system may correspond to the amount of rotation of the given object in the Cartesian coordinate system, and the center of the polar transform 728 may correspond to the axis of rotation of the given object in the Cartesian coordinate system.

[0052] 8 is a flow diagram of an image enhancement algorithm 800 included in a spin estimation system in accordance with at least one embodiment of the present disclosure. The image enhancement algorithm 700 may be implemented to generate an enhanced object image from one or more acquired object images. In some embodiments, the image enhancement algorithm 700 may facilitate generating one or more of the resized image 421, negative image 422, stretched image 423, masked image 424, histogram equalized image 425, simplified image 426, or enhanced image 427, as described in connection with FIG. 4C . Any reference to an object in the description of the image enhancement algorithm 800 may be interpreted as referring to an image of the object. For example, resized object 806, negative object 810, stretched object 814, masked object 818, histogram equalized object 822, and enhanced object 828 may refer to a resized object image, a negative object image, a stretched object image, a masked object image, a histogram equalized object image, and an enhanced object image, respectively, rather than physical changes made to the object itself from which object image 802 was captured.

[0053] Image enhancement algorithm 800 may take object image 802 as input to resizing algorithm 804, which outputs resized object 806. Object image 802 may be the same as or similar to object image 502 described in connection with Figure 5 or object image 702 described in connection with Figure 7. Additionally or alternatively, object image 802 may be the same as or similar to image 420 described in connection with Figure 4C.

[0054] In some embodiments, the resizing algorithm 804 can scale the object images 802 to the same or similar size according to the reference radius. A first given object image 802 depicting a first object with a radius smaller than the reference radius may be scaled up so that the radius of the first object is the same or similar to the reference radius within a given threshold, while a second given object image 802 depicting a second object with a radius larger than the reference radius may be scaled down accordingly.

[0055] A negative operation 808 can be applied to each of the resized objects 806 to increase pixel values ​​corresponding to one or more marker patterns imprinted on the exterior surface of the resized object 806, thereby more accurately analyzing the marker patterns relative to the object's spin rate and spin axis. In some embodiments, the negative operation 808 can involve subtracting each pixel value of the pixels included in the resized object 806 from a maximum pixel value to generate a corresponding negative image of the object. The negative image of the resized object 806 generated by the negative operation 808 can include low pixel values ​​corresponding to sections of the resized object 806 that included high pixel values, while sections of the negative image include high pixel values ​​where the resized object 806 included low pixel values. Because marker patterns imprinted on the resized object 806 typically include darker colors, and darker colors are typically associated with lower pixel values, the negative operation 808 can increase the pixel values ​​of the marker patterns imprinted on the resized object 806 to facilitate analysis of such marker patterns.

[0056] A stretching operation 812 may be applied to the negative object 810 output by the negative operation 808 to generate a stretched object 814. In some examples, details contained near the edges in the object image 802, such as the overall size of the object image 802, the overall shape of the object image 802, or the location of portions of a marker pattern, may be distorted along the edges of the object image 802 relative to such details appearing near the center of the negative object 810 after processing by the resizing algorithm 804 or the negative operation 808. The stretching operation 812 may be applied to remove or reduce these and other distortion effects caused by the resizing algorithm 804 or the negative operation 808.

[0057] In some embodiments, the decompressed object 814 may be obtained by a circular masking operation 816 and a histogram equalization operation 820 to remove any visual artifacts produced by the decompression operation 812. The circular masking operation 816 may apply a mask to the decompressed object 814 that facilitates filtering background details included in the decompressed object 814 between the decompressed image 423 and the masked image 424, as shown in FIG. 4C . In these and other embodiments, the histogram equalization operation 820 may increase the contrast of the masked object 818 to produce a histogram-equalized object 822 between the masked image 424 and the histogram-equalized image 425, as shown in FIG. 4C . In some embodiments, the histogram equalization operation 820 may be followed by an adaptive binarization operation 824, which includes simplifying the histogram-equalized object 822, as shown between the histogram-equalized image 425 and the simplified image 426.

[0058] The histogram equalized object 822 may have been simplified via an adaptive thresholding operation 824, and one or more of the decompressed objects 814 may be obtained via an AND operation 826 to generate an enhanced object 828. In some embodiments, the AND operation 826 may involve outputting a binary map in which each pixel included in the binary map has a pixel value of “1” or “0.” The AND operation 826 may perform a pixel-by-pixel comparison between the decompressed object 814 and the binary map output by the adaptive thresholding operation 824 and modify the pixel values ​​corresponding to the decompressed object 814 based on the comparison between the decompressed object 814 and the binary map output by the adaptive thresholding operation 824. For example, a given pixel value corresponding to a pixel of a given decompressed object 814 may be converted to a “0” value in response to determining that the pixel value of the corresponding pixel in the binary map output by the adaptive thresholding operation 824 has a pixel value of “0.” As an additional or alternative example, a given pixel value corresponding to a pixel of a given stretched object 814 may remain unmodified from its original pixel value in response to determining that the pixel value of the corresponding pixel in the binary map has a pixel value of "1." Enhanced object 828 may represent a version of object image 802 that is more easily analyzed. For example, enhanced object 828 may be the same as or similar to enhanced image 708 described in connection with operation 700 of FIG. 7 , provided that object image 802 is the same as or similar to object image 702. In some embodiments, enhanced object 828 can be used to estimate the spin rate-axis of the object on which object image 802 is based, as described in connection with improved spin rate-axis estimation 730.

[0059] 9 is a flowchart of an example method 900 for performing spin estimation in accordance with at least one embodiment of the present disclosure. Method 900 may be performed by any suitable system, apparatus, or device. For example, one or more computer systems or software modules corresponding to spin estimation system 110 of FIG. 1 may perform one or more operations related to method 900. Although shown in separate blocks, steps and operations associated with one or more of the blocks of method 900 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0060] Method 900 may begin at block 902, where an image capture frame rate may be set that corresponds to a minimum spin motion of an object. In some embodiments, the image capture frame rate may be configured to operate at or above a minimum frame rate threshold, with the minimum frame rate threshold being set based on the object's spin rate range as determined by the activities in which the object may typically be involved. For example, the spin rate of a golf ball in a typical golf game may range from 500 RPM to 12,000 RPM, and any camera operating at less than a 200 FPS frame rate may not be able to properly capture images of the object if the object spins at or near the upper end of the spin rate range due to aliasing effects. Therefore, the image capture frame rate may be set to at least 200 FPS.

[0061] In block 904, orientation markers may be printed on the outer surface of the object. The orientation markers on the outer surface of the object may be generated according to a three-dimensional rectangular parallelepiped model superimposed on the outer surface of the object, with each face of the three-dimensional rectangular parallelepiped model including a unique marker pattern relative to each other face of the three-dimensional rectangular parallelepiped model, as described in connection with Figures 3A-3D. The marker patterns on each face of the three-dimensional rectangular parallelepiped model may each be rotationally asymmetric so that the orientation of the object during its motion can be determined at any point during the object's translation and rotation.

[0062] In block 906, an image of the object may be captured after the object begins to move. The image may be captured by an image capture sensor having the image capture frame rate set in block 902.

[0063] At block 908, objects may be detected in each of the images captured at block 906 and separated to generate separated object images. In some embodiments, detecting objects in each of the images may involve using a neural network-based object detector to recognize one or more features associated with objects likely to be included in the images, such as a ball object used in a particular sports game, other equipment used in a particular sports game, or any other object. Additionally or alternatively, other digital image processing techniques may be used, such as applying a circular Hough transform to detect circular shapes in the images that likely correspond to the object. Additionally or alternatively, a motion change detection algorithm, such as a three-frame differencing algorithm, may be applied to detect objects from the remainder of the images based on changes corresponding to the object's movement between three or more frames of the images. In these and other embodiments, the detected objects may be separated by cropping the detected objects from the image frames. In some embodiments, the images may be enhanced according to an image enhancement process, such as according to operation 800 of FIG. 8. Each of the images may be resized, processed, stretched, or some combination thereof, to generate standardized images that can be more easily compared to each other. For example, a resizing algorithm 804, a negation operation 808, a stretching operation 812, or some combination thereof, as described in connection with the image enhancement algorithm of Figure 8, may be applied to the image of the object to standardize the image of the object and facilitate comparability between the images. Detecting the object in each image of the plurality of images may involve applying a mask to each of the standardized images that filters background contained within each image of the plurality of images. For example, as described in connection with the image enhancement algorithm of Figure 8, a circular masking operation, a histogram equalization operation, an adaptive thresholding operation, or some combination thereof, such as a circular masking operation 816, a histogram equalization operation 820, an adaptive binarization operation 824, an AND operation 826, or some combination thereof, may be applied to the image of the object to filter background information contained in the object image and facilitate detection of the object in the image.

[0064] At block 910, an object marker segmentation map may be generated based on the separated object images. In some embodiments, generating the object marker segmentation map may involve binarizing each separated object image and generating pairs of successive binarized object maps, where the object marker segmentation map includes all or some of the pairs of successive binarized object maps. In these and other embodiments, the object marker segmentation map may include multiple pairs of successive object images, where the spin rate and spin axis may be determined by comparing the object as depicted in each of the images.

[0065] At block 912, the spin rate and spin axis of the moving object may be estimated based on the object marker segmentation maps by following operation 500 of FIG. 5 . The difference in the orientation, position, any other characteristic, or some combination thereof, of the object shown in any two given consecutive binarized object maps may be used to determine the spin rate and spin axis. Additionally or alternatively, the difference between the estimated spin rate or spin axis between different pairs of consecutive binarized object maps may be statistically analyzed to determine a single estimate of the object's spin rate and spin axis. In some embodiments, determining the spin rate or spin axis between a given pair of consecutive binarized object maps may involve inputting the given pair of consecutive binarized object maps into a deep learning algorithm trained using real-world images of objects annotated according to the object's spin rate and spin axis, such that the deep learning algorithm may estimate the spin rate and spin axis of the object depicted in the given pair of consecutive binarized object maps. Additionally or alternatively, training of deep learning algorithms can be facilitated by synthetic data of successive binarized object maps generated in a simulated environment.

[0066] For example, one or more of the estimated spin rates or one or more of the estimated spin axes associated with a pair of successive binarized object maps may be identified as outlier spin estimates. The estimated spin rates and estimated spin axes that are not identified as outlier spin estimates may be aggregated to determine a final estimated spin rate and final estimated spin axis.

[0067] In some embodiments, estimating the spin rate and spin axis of a given pair of consecutive binarized object maps may involve estimating a first object spin corresponding to each pair of consecutive binarized object maps in a forward direction and estimating a second object spin corresponding to each pair of consecutive binarized object maps in a reverse direction. The first object spin and the second object spin may be compared to each other to determine an estimated spin rate and an estimated spin axis for the given pair of consecutive binarized object maps.

[0068] Modifications, additions, or omissions may be made to method 900 without departing from the scope of the present disclosure. For example, the designations of different elements in the described methods are meant to help explain the concepts described herein and are not limiting. Furthermore, method 900 may include any number of other elements or may be implemented in other systems or contexts other than those described.

[0069] 10 is an exemplary computer system 1000 in accordance with at least one embodiment described in this disclosure. The computing system 1000 may include a processor 1010, a memory 1020, data storage 1030, and / or a communication unit 1040, all of which may be communicatively coupled. Any or all of the spin estimation system 110 of FIG. 1 may be implemented as a computing system consistent with the computing system 1000.

[0070] In general, the processor 1010 may include any suitable special-purpose or general-purpose computer, computing entity, or processing device, including various computer hardware or software modules, and may be configured to execute instructions stored on any applicable computer-readable storage medium. For example, the processor 1010 may include a microprocessor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or any other digital or analog circuitry configured to interpret and / or execute program instructions and / or process data.

[0071] 10 as a single processor, it is understood that processor 1010 may include any number of processors distributed across any number of networks or physical locations configured to individually or collectively perform any number of operations described in this disclosure. In some embodiments, processor 1010 may interpret and / or execute program instructions and / or process data stored in memory 1020, data storage 1030, or memory 1020 and data storage 1030. In some embodiments, processor 1010 may fetch program instructions from data storage 1030 and load program instructions into memory 1020.

[0072] After the program instructions are loaded into memory 1020, processor 1010 may execute the program instructions, such as instructions that cause computing system 1000 to perform the operations of operation 500 in FIG. 5, one or more operations associated with neural network 600 in FIG. 6, one or more operations associated with refiner algorithm 700 in FIG. 7, one or more operations associated with image enhancement algorithm 800 in FIG. 8, or the operations of method 900 in FIG. 9.

[0073] Memory 1020 and data storage 1030 may include a computer-readable storage medium or one or more computer-readable storage media for storing computer-executable instructions or data structures. Such computer-readable storage media may be any available media that can be accessed by a general-purpose computer or a special-purpose computer, such as processor 1010. For example, memory 1020 and / or data storage 1030 may include object image 502, binary map 506, spin rate and spin axis estimate 514, spin rate and spin axis estimate 518, final spin rate-axis estimate 522, or refined spin rate-axis estimate 526, as described in connection with FIG. 5 . In some embodiments, computing system 1000 may or may not include either memory 1020 or data storage 1030.

[0074] By way of example, and not limitation, such computer-readable storage media may include non-transitory computer-readable storage media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid-state memory devices), or any other storage medium that can be used to store desired program code in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer. Combinations of the above may also be included within the scope of computer-readable storage media. Computer-executable instructions may include, for example, instructions and data configured to cause processor 1010 to perform a particular operation or group of operations.

[0075] The communications unit 1040 may include any component, device, system, or combination thereof configured to transmit or receive information over a network. In some embodiments, the communications unit 1040 can communicate with other devices at other locations, the same location, or even other components within the same system. For example, the communications unit 1040 may include a modem, a network card (wireless or wired), an optical communications device, an infrared communications device, a wireless communications device (such as an antenna), and / or a chipset (such as a Bluetooth® device, an 802.6 device (e.g., a metropolitan area network (MAN)), a Wi-Fi® device, a WiMax® device, a cellular communications facility, etc.). The communications unit 1040 may enable data to be exchanged with a network and / or any other device or system described in this disclosure. For example, the communications unit 1040 may enable the system 1000 to communicate with other systems, such as computing devices and / or other networks.

[0076] Those skilled in the art, after reviewing this disclosure, may recognize that modifications, additions, or omissions may be made to system 1000 without departing from the scope of the disclosure. For example, system 1000 may include more or fewer components than explicitly illustrated and described.

[0077] The foregoing disclosure is not intended to limit the present disclosure to the precise form or particular field of use disclosed. Accordingly, various alternative embodiments and / or modifications to the present disclosure, whether expressly described or implied herein, are contemplated as possible in light of the present disclosure. While embodiments of the present disclosure have been thus described, it will be recognized that changes can be made in form and detail without departing from the scope of the present disclosure. Accordingly, the present disclosure is limited only by the scope of the claims.

[0078] In some embodiments, the different components, modules, engines, and services described herein may be implemented as entities or processes (e.g., as separate threads) running on a computing system. Although some of the systems and processes described herein are generally described as being implemented in software (stored on and / or executed by general-purpose hardware), specific hardware implementations or combinations of software and specific hardware implementations are also possible and contemplated.

[0079] The terms used in this disclosure and particularly in the appended claims (e.g., the body of the appended claims) are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to").

[0080] Additionally, if a specific number of introduced claim recitations is intended, such intention will be explicitly stated in the claim; absent such recitation, no such intention exists. For example, as an aid to understanding, the following appended claims may include the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be interpreted as suggesting that when a claim recitation is introduced by the indefinite article "a" or "an," any particular claim including such introduced claim recitation is limited to embodiments including only one of such recitations, even if the same claim contains an introductory phrase such as "one or more" or "at least one" and the indefinite article "a" or "an" (e.g., "a" and / or "an" should be interpreted to mean "at least one" or "one or more"). The same applies when a definite article is used to introduce a claim recitation.

[0081] Furthermore, even when a particular number is explicitly recited in an introduced claim, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., a simple "two items" without other modifiers means at least two items, or more than two items). Furthermore, when conventions similar to "at least one of A, B, and C, etc." or "one or more of A, B, and C, etc." are used, it is generally intended that such configurations include A only, B only, C only, a combination of A and B, a combination of A and C, a combination of B and C, or a combination of A, B, and C, etc.

[0082] Furthermore, any disjunctive word or phrase preceding two or more alternative terms in either the specification, claims, or drawings should be understood to contemplate the possibility of including one of the terms, either of the terms, or both of the terms. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B" or "A and B."

[0083] All examples and conditional language set forth in this disclosure are intended for educational purposes to aid the reader in understanding the disclosure and the concepts contributed by the inventors to the advancement of the art, and should be construed as not being limited to such specifically set forth examples and conditions. Although embodiments of the present disclosure have been described in detail, various changes, substitutions, and alterations can be made therein without departing from the spirit and scope of the present disclosure. The inventions disclosed herein include the following: [Aspect 1] 1. A method comprising: setting an image capture frame rate corresponding to a minimum spin motion of the object; printing an orientation marker on an outer surface of the object; capturing a plurality of images of the object by an image capture sensor having the set image capture frame rate corresponding to the minimum spin motion of the object after initiating the motion of the object; Segmenting the object in each image of the plurality of images to generate a plurality of segmented object images; generating an object marker segmentation map based on the plurality of separated object images; estimating a spin rate and a spin axis based on the object marker segmentation map; A method comprising: [Aspect 2] 2. The method of embodiment 1, wherein the image capture frame rate is set to at least 200 frames per second. [Aspect 3] The method of aspect 1, wherein the orientation markers on the outer surface of the object are generated according to a 3D rectangular parallelepiped model superimposed on the outer surface of the object, and each face of the 3D rectangular parallelepiped model includes a unique marker pattern relative to each other face of the 3D rectangular parallelepiped model. [Aspect 4] The method of aspect 3, wherein each marker pattern included on the surface of the three-dimensional rectangular parallelepiped model is rotationally asymmetric. [Aspect 5] 2. The method of claim 1, wherein separating the object in each image of the plurality of images includes applying a mask to each image that filters background contained within each image of the plurality of images. [Aspect 6] Generating the object marker segmentation map includes: binarizing each separated object image included in the plurality of separated object images; generating a pair of successive binarized object maps. [Aspect 7] estimating the spin rate and the spin axis estimating an object spin corresponding to each pair of successive binarized object maps; identifying one or more of the estimated object spins as outlier spin estimates; Integrating the estimated object spins that are not identified as the outlier spin estimates as the estimated spin rate; 7. The method of claim 6, comprising: estimating the spin axis based on the estimated spin rate. [Aspect 8] The method of aspect 7, wherein estimating the object spin corresponding to each pair of successive binarized object maps includes inputting the pair of successive binarized object maps into a deep learning algorithm trained using a plurality of real-world images or a plurality of object images generated in a simulated environment annotated according to object spin. [Aspect 9] The method of aspect 7, wherein estimating the object spin corresponding to each pair of successive binarized object maps includes estimating a first object spin corresponding to each pair of successive binarized object maps in a forward direction, and estimating a second object spin corresponding to each pair of successive binarized object maps in a reverse direction. [Aspect 10] One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause the system to perform an action, the action including: setting an image capture frame rate corresponding to a minimum spin motion of the object; printing an orientation marker on an outer surface of the object; capturing a plurality of images of the object by an image capture sensor having the set image capture frame rate corresponding to the minimum spin motion of the object after initiating the motion of the object; Segmenting the object in each image of the plurality of images to generate a plurality of segmented object images; generating an object marker segmentation map based on the plurality of separated object images; estimating a spin rate and a spin axis based on the object marker segmentation map; one or more non-transitory computer-readable storage media, [Aspect 11]

[0023] Aspect 11. The one or more non-transitory computer-readable storage media of aspect 10, wherein the image capture frame rate is set to at least 200 frames per second. [Aspect 12] One or more non-transitory computer-readable storage media as described in aspect 10, wherein the orientation markers on the outer surface of the object are generated according to a three-dimensional rectangular parallelepiped model superimposed on the outer surface of the object, and each face of the three-dimensional rectangular parallelepiped model includes a unique marker pattern relative to each other face of the three-dimensional rectangular parallelepiped model. [Aspect 13]

[0023] Aspect 13. The one or more non-transitory computer-readable storage media of aspect 12, wherein each marker pattern included on the face of the three-dimensional rectangular parallelepiped model is rotationally asymmetric. [Aspect 14] One or more non-transitory computer-readable storage media according to aspect 10, wherein separating the object in each image of the plurality of images includes applying a mask to each image that filters background contained within each image of the plurality of images. [Aspect 15] Generating the object marker segmentation map includes: binarizing each separated object image included in the plurality of separated object images; and generating a pair of successive binarized object maps. [Aspect 16] estimating the spin rate and the spin axis estimating an object spin corresponding to each pair of successive binarized object maps; identifying one or more of the estimated object spins as outlier spin estimates; Integrating the estimated object spins that are not identified as the outlier spin estimates as the estimated spin rate; and estimating the spin axis based on the estimated spin rate. [Aspect 17] One or more non-transitory computer-readable storage media according to aspect 16, wherein estimating the object spin corresponding to each pair of successive binarized object maps comprises inputting the pair of successive binarized object maps into a deep learning algorithm trained using a plurality of real-world images or a plurality of object images generated in a simulated environment annotated according to object spin. [Aspect 18] One or more non-transitory computer-readable storage media according to aspect 16, wherein estimating the object spin corresponding to each pair of successive binarized object maps includes estimating a first object spin corresponding to each pair of successive binarized object maps in a forward direction, and estimating a second object spin corresponding to each pair of successive binarized object maps in a reverse direction. [Aspect 19] 1. A spin estimation system comprising: an image capture sensor positioned and configured to capture images of objects within a field of view of the image capture sensor; one or more processors; one or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause the spin estimation system to perform operations, the operations including: setting an image capture frame rate corresponding to a minimum spin motion of the object; printing an orientation marker on an outer surface of the object; capturing a plurality of images of the object by the image capture sensor having the set image capture frame rate corresponding to the minimum spin motion of the object after initiating the motion of the object; Segmenting the object in each image of the plurality of images to generate a plurality of segmented object images; generating an object marker segmentation map based on the plurality of separated object images; estimating a spin rate and a spin axis based on the object marker segmentation map; a spin estimation system, [Aspect 20] the orientation markers on the outer surface of the object are generated according to a three-dimensional rectangular parallelepiped model superimposed on the outer surface of the object, each face of the three-dimensional rectangular parallelepiped model including a unique marker pattern relative to each other face of the three-dimensional rectangular parallelepiped model; Each marker pattern included on the surface of the three-dimensional rectangular parallelepiped model is rotationally asymmetric. 20. A spin estimation system according to embodiment 19.

Claims

1. 1. A method comprising: setting an image capture frame rate corresponding to a minimum spin motion of an object, the object having an orientation marker printed on an exterior surface of the object; capturing a plurality of images of the object by an image capture sensor having the set image capture frame rate corresponding to the minimum spin motion of the object after initiating the motion of the object; Segmenting the object in each image of the plurality of images to generate a plurality of segmented object images; generating an object marker segmentation map based on the plurality of separated object images; estimating a spin rate and a spin axis based on the object marker segmentation map; Including, the orientation markers on the outer surface of the object are generated according to a three-dimensional cuboid model superimposed on the outer surface of the object, each face of the three-dimensional cuboid model including a unique marker pattern relative to each other face of the three-dimensional cuboid model.

2. The method of claim 1 , wherein the image capture frame rate is set to at least 200 frames per second.

3. The method of claim 1 , wherein each marker pattern included on the surface of the three-dimensional rectangular parallelepiped model is rotationally asymmetric.

4. The method of claim 1 , wherein isolating the object in each image of the plurality of images comprises applying a mask to each image that filters background contained within each image of the plurality of images.

5. Generating the object marker segmentation map includes: binarizing each separated object image included in the plurality of separated object images; and generating a pair of successive binarized object maps.

6. estimating the spin rate and the spin axis estimating an object spin corresponding to each pair of successive binarized object maps; identifying one or more of the estimated object spins as outlier spin estimates; integrating the object spins not identified as the outlier spin estimates as an estimated spin rate; and estimating the spin axis based on the estimated spin rate.

7. 7. The method of claim 6, wherein estimating the object spin corresponding to each pair of successive binarized object maps comprises inputting the pair of successive binarized object maps to a deep learning algorithm trained using a plurality of real-world images or a plurality of object images generated in a simulated environment annotated according to object spin.

8. 7. The method of claim 6, wherein estimating the object spin corresponding to each pair of successive binarized object maps includes estimating a first object spin corresponding to each pair of successive binarized object maps in a forward direction, and estimating a second object spin corresponding to each pair of successive binarized object maps in a reverse direction.

9. One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause a device to perform an action, the action including: setting an image capture frame rate corresponding to a minimum spin motion of an object, the object having an orientation marker printed on an exterior surface of the object; capturing a plurality of images of the object by an image capture sensor having the set image capture frame rate corresponding to the minimum spin motion of the object after initiating the motion of the object; Segmenting the object in each image of the plurality of images to generate a plurality of segmented object images; generating an object marker segmentation map based on the plurality of separated object images; estimating a spin rate and a spin axis based on the object marker segmentation map; Including, one or more non-transitory computer-readable storage media, wherein the orientation markers on the outer surface of the object are generated according to a three-dimensional cuboid model superimposed on the outer surface of the object, each face of the three-dimensional cuboid model including a unique marker pattern relative to each other face of the three-dimensional cuboid model.

10. The one or more non-transitory computer-readable storage media of claim 9 , wherein the image capture frame rate is set to at least 200 frames per second.

11. The one or more non-transitory computer-readable storage media of claim 10 , wherein each marker pattern included on the face of the three-dimensional rectangular parallelepiped model is rotationally asymmetric.

12. 10. The one or more non-transitory computer-readable storage media of claim 9, wherein separating the object in each image of the plurality of images comprises applying a mask to each image that filters background contained within each image of the plurality of images.

13. Generating the object marker segmentation map includes: binarizing each separated object image included in the plurality of separated object images; and generating a pair of successive binarized object maps.

14. estimating the spin rate and the spin axis estimating an object spin corresponding to each pair of successive binarized object maps; identifying one or more of the estimated object spins as outlier spin estimates; integrating the object spins not identified as the outlier spin estimates as an estimated spin rate; and estimating the spin axis based on the estimated spin rate.

15. 15. The one or more non-transitory computer-readable storage media of claim 14, wherein estimating the object spin corresponding to each pair of successive binarized object maps comprises inputting the pair of successive binarized object maps to a deep learning algorithm trained using a plurality of real-world images or a plurality of object images generated in a simulated environment annotated according to object spin.

16. 15. The one or more non-transitory computer-readable storage media of claim 14, wherein estimating the object spin corresponding to each pair of successive binarized object maps includes estimating a first object spin corresponding to each pair of successive binarized object maps in a forward direction, and estimating a second object spin corresponding to each pair of successive binarized object maps in a reverse direction.

17. 1. A spin estimation device, comprising: an image capture sensor positioned and configured to capture images of objects within a field of view of the image capture sensor; one or more processors; one or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause the spin estimation device to perform an operation, the operation comprising: setting an image capture frame rate corresponding to a minimum spin motion of the object, the object having an orientation marker printed on an exterior surface of the object; capturing a plurality of images of the object by the image capture sensor having the set image capture frame rate corresponding to the minimum spin motion of the object after initiating the motion of the object; Segmenting the object in each image of the plurality of images to generate a plurality of segmented object images; generating an object marker segmentation map based on the plurality of separated object images; estimating a spin rate and a spin axis based on the object marker segmentation map; Including, the orientation markers on the outer surface of the object are generated according to a three-dimensional rectangular parallelepiped model superimposed on the outer surface of the object, each face of the three-dimensional rectangular parallelepiped model including a unique marker pattern relative to each other face of the three-dimensional rectangular parallelepiped model.

18. each marker pattern included on the surface of the three-dimensional rectangular parallelepiped model is rotationally asymmetric; 18. The spin estimation device of claim 17.

19. A spin estimation system comprising the spin estimation device of claim 17 or 18.

Images