System and method for measuring 3D properties using computer vision

Abstract

The present invention discloses a system including a computing device and a camera; the system is configured to measure three-dimensional properties and related performance measurements of a mechanical device. Some embodiments include a camera configured to capture an image of the mechanical device and a computing device in communication with the camera. In some embodiments, the computing device is configured to acquire a first set of pixels associated with a plurality of first fiducials to calibrate the spatial resolution of the camera. A second image from the camera can be converted into a second set of pixels associated with each of a plurality of fiducials attached to the mechanical device. The computing device can be further configured to compare the first set of pixels and the second set of pixels to determine the position of the plurality of fiducials on the mechanical device.

Description

[0001] This application is a divisional application. The national application number of the parent application is: 201980052786.2 (international application number is PCT / US2019 / 036839), the date of entry into the Chinese national phase is: February 8, 2021 (international application date is June 12, 2019), and the name of the invention is: System and method for measuring 3D properties using computer vision.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application is a U.S. non-provisional patent application claiming priority to U.S. provisional application serial number 62 / 684,119, filed on June 12, 2018, the entire contents of which are incorporated herein by reference. Technical Field

[0004] The present disclosure generally relates to a computerized system comprising at least one computing device and a camera, the at least one computing device and the camera being collectively configured to measure (in real time) predetermined physical properties associated with a mechanical device and allow adjustments thereto by predicting three-dimensional (3D) characteristics of the mechanical device based on processed two-dimensional (2D) data associated with the mechanical device. Background Art

[0005] Mechanical devices often undergo adjustment phases during or after device manufacturing, for example, to configure angles and other dimensions associated with components of the device as needed. For example, the angle of an arm or elongated member relative to the main body of the device or relative to a selected surface can be adjusted or modified according to predetermined specifications suitable for a particular application. However, this adjustment phase may involve connecting and disconnecting the mechanical device with various measuring and adjustment tools, which can cause wear and tear on the device.

[0006] As a specific, non-limiting example, adjusting the properties of a golf club (e.g., loft) during the manufacturing process or otherwise requires careful structural modifications to the club head and shaft based on precise measurements. In particular, measurements must be taken after each adjustment of a golf club property (e.g., loft, head angle, length, lift, or roll) to ensure that the adjustment is appropriate. However, the tools for measuring golf club properties typically require physical contact with the golf club. As such, the golf club is typically transferred back and forth between the measuring tool and the adjustment tool multiple times. Therefore, the golf club may have to be repositioned within the adjustment tool before each adjustment. Transferring the golf club multiple times in this manner results in increased manufacturing work time and may also result in scratches and wear on the finished golf club product.

[0007] It is with these considerations in mind, among other things, that the various aspects of the present disclosure have been conceived and developed. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1A is a simplified block diagram illustrating a system configured to allow measurement and possible adjustment of a mechanical device according to desired target performance specifications.

[0009] Figure 1B is shown with Figure 1A A flowchart of the flow logic and relationships between possible modules of an application related to a system for evaluating and possibly adjusting mechanical equipment.

[0010] Figure 2A is an exemplary calibration patch that includes an array of fiducials for tracking image features of a mechanical device.

[0011] Figure 2B is with Figure 2A The calibration sheet is associated with a set of fiducials that can be positioned along a mechanical device to measure various aspects of the device as described herein.

[0012] Figure 3A is a simplified illustration of an exemplary mechanical device that can be measured and adjusted using the systems described herein.

[0013] Figure 3B is a simplified illustration of the application of the system described herein to an actual golfer to enhance swing analysis.

[0014] Figure 4 Is shown using Figure 1A A flow chart of the flow logic and relationships between possible modules of an application for a system for evaluating and possibly adjusting a particular mechanical device (e.g., a golf club).

[0015] Figure 5 yes Figure 4 Perspective view of a golf club.

[0016] Figure 6A yes Figure 4 A perspective view of a golf club in FIG. 1 and a benchmark positioned along the golf club.

[0017] Figure 6B yes Figure 4 Another perspective view of a golf club in FIG. 1 and a fiducial positioned along the golf club.

[0018] Figure 6C yes Figure 6B An enlarged detail view of a golf club head and fiducials located along the face of the golf club.

[0019] Figure 7 yes Figure 6BSide view of a golf club during measurement and analysis.

[0020] Figures 8A-8B is a diagram illustrating test data associated with the inventive concepts described herein.

[0021] Figure 9 is a simplified block diagram illustrating an example of a computing system that may implement the various services, systems, and methods discussed herein.

[0022] Corresponding reference numerals indicate corresponding elements throughout the views of the drawings. The headings used in the drawings do not limit the scope of the claims. DETAILED DESCRIPTION

[0023] In view of the foregoing, there is a need for improvements in the art related to systems for measuring three-dimensional (3D) properties associated with mechanical equipment. Accordingly, aspects of the present disclosure relate to an inventive solution in the form of a system comprising a computing device and / or computer program product configured via an application or otherwise, and at least one camera (integrated with the computing device or implemented separately), wherein the system is generally configured to measure 3D properties associated with predefined performance measurements of the mechanical equipment based on computing or processing image data representing changes in 2D image features associated with the mechanical equipment (understood as real features defined along the equipment and / or along a reference positioned along the equipment).

[0024] The system of the present invention can improve various aspects of mechanical equipment manufacturing and measurement. More specifically, during the production process, the system enables manufacturers to read the 3D properties of mechanical equipment in real time, allowing adjustments to be made after each measurement without repositioning the equipment within the adjustment mechanism. Consequently, the system can reduce work time, the number of adjustments, and the wear and tear on finished products. Furthermore, the system can improve equipment quality, reduce operating costs, and alleviate technician fatigue. Referring to the accompanying drawings, an embodiment of a system for real-time measurement of mechanical equipment properties is shown, generally designated 100 and 200 in Figures 1-9.

[0025] Reference Figure 1A, a system 100 for measuring and adjusting properties of a mechanical device is shown. System 100 generally may include a measurement and adjustment application 102 (hereinafter referred to as "application 102") executed by a computing device 104, a camera 106 (or multiple cameras 106) in operative communication with computing device 104, and a fiducial 108 (or multiple fiducials 108) defined or positioned along a mechanical device 110. Fiducials 108 may include any object or marker placed in the field of view of camera 106 or other imaging system that appears as a reference point or measurement in a subsequently generated image or image data (or as an object or set of markers in an optical instrument reticle). In some embodiments, fiducials 108 may include a discrete component that can be attached to a portion of mechanical device 110, such as a barcode, a 2D barcode, a solid geometric shape (e.g., a green triangle), or any such component that can be captured or interpreted by camera 106 and represented by pixels of an image generated by camera 106. Alternatively, the fiducials 108 may include real features defined along the mechanical device 110, such as grooves, ridges, protrusions, printed characters, etc., that are used to define a plane, vector, or other attribute.

[0026] In some embodiments, the system 100 may further include a fixing tool 112, such as a vise, for holding the mechanical device 110 in position relative to the camera 106, and may also include an adjustment tool 114 (e.g., a bender, a clamp, a wrench, etc.) for manipulating, bending, or otherwise changing a physical property of the mechanical device 110. In general, the system 100 is configured to allow for measurement and adjustment of predetermined properties of the mechanical device 110, as further described herein.

[0027] As shown, computing device 104 can be in operable wireless communication with camera 106 via network 116. In other embodiments, computing device 104 can be in operable communication with camera 106 via a wired connection, or camera 106 can be integrated with computing device 104, as further described herein. Computing device 104 can include one or more of the following: a server, a controller, a personal computer, a terminal, a workstation, a laptop computer, a mobile device, a smart phone, a tablet computer, a mainframe, or other such computing devices configured with application 102 or configured to implement the functions described herein. Application 102 can be created using different software packages such as Open CV and ArUco toolsets in C++, but the present invention is not limited thereto. Various aspects of system 100 and / or application 102 can be set as mobile applications using platform as a service (PaaS) and / or software as a service (SaaS) using, for example, Amazon Web Services (Amazon Web Services) or other distributed or decentralized systems. Network 116 can include the Internet, an intranet, a virtual private network (VPN), a local area network (LAN), a wide area network (LAN), a peer-to-peer network, the cloud, etc. In some embodiments, a cloud (not shown) may be used to execute one or more components of system 100 .

[0028] As further noted, the computing device 104 can be operably connected to or have access to a database 118. The database 118 can store information about predetermined or desired characteristics of the mechanical device 110, virtual objects, and other associated information described herein. For example, the database 118 can store information about predetermined loft angles and head angles (lie) suitable for different types of golf clubs. In addition, at least some features of the application 102 can be provided to a plurality of user devices 120 in communication with the computing device 104 via the network 116. The plurality of user devices 120 can include, but are not limited to, at least one of the following: a controller, a personal computer, a terminal, a workstation, a portable computer, a laptop computer, a mobile device, a tablet computer, a phone, a pager, or a multimedia console. Any of the plurality of user devices 120 can be used, for example, to submit a request or information to the computing device 104, such as requesting that the mechanical device 110 be adjusted in some form.

[0029] In reference Figure 1A Based on the reference Figure 1B, a flowchart 150 is shown that illustrates the creation, processing, and flow of data using a computing device 104 and a camera 106 as the data travels and through a number of possible modules associated with the application 102. Utilizing the modules shown, the application 102 is configured to output physical 3-D physical properties or other information (e.g., the relative angle between any surface and a given axis of the mechanical device 110) that can be used to generate performance measurements or information that can be used to evaluate or adjust the mechanical device. In general, the camera 106 can be used to generate calibration image frames 152 associated with a "scene calibration" module 154 to facilitate evaluating the pre-calibrated 3D position (156) of the fiducial 108. The fiducial 108 can then be deployed along the mechanical device 110, image frames 158 of the mechanical device 110 can be generated, and a "fiducial tracking" module 160 and / or a "real feature tracking" module 164 can be used (in conjunction or separately) with the camera 106 to record or identify 2D image coordinates 162 or other 2D image features associated with the fiducial 108. Using the 2D image coordinates 162, a "3D Position Estimation" module 170 associated with the application 102 is configured to subsequently estimate or generate a set of three-dimensional (3D) points / coordinates 172 corresponding to the real features of the fiducial 108 and the mechanical device 110. Once such 3D points 172 are created, a "Define Physical Properties" module 174 can be implemented, which is configured to apply a series of predetermined linear algebraic operations to the 3D points 172 in order to define certain physical properties 176 of the mechanical device 110. A "Performance Measurement" module 178 can then utilize data associated with the physical properties 176 of the mechanical device 110 to output any number or type of performance measurements, as further described herein. In some embodiments, the performance measurements can also be further checked against predetermined target specifications (not shown) via a "Virtual Object Overlay" module 180 with reference to a database 182 of predetermined virtual objects.

[0030] More specifically, refer to Figure 1B The scene calibration module 154 may first calibrate the spatial resolution of the camera 106 using the reference 108. In some embodiments, this may first include using the reference 108 or a reference array (in Figure 2A The reference array is shown as 204) to place the calibration piece ( Figure 2A The calibration sheet 202 is positioned in front of the camera 106 so that the calibration sheet 202 is oriented toward the camera 106. Figures 2A-2BAs shown, fiducial 108 may be a 2D image pixelation that defines a generally square shape and a unique geometric shape of a pixelated black and white pattern. In other embodiments, fiducial 108 may define any type or specific shape (e.g., a circle, rectangle, triangle, pentagon, octagon, or any other shape) or some type of feature suitable for tracking changes in image data associated with fiducial 108. Fiducial 108 may further define a surface area. For example, the surface area of ​​fiducial 108 may include any shape (e.g., a square, rectangle, triangle, circle, pentagon, octagon, or any other shape) and may be, for example, one square inch. In other examples, the surface area of ​​fiducial 108 may be scaled up. For example, the surface area of ​​fiducial 108 may be at least 1 square inch, at least 1.5 square inches, at least 1.75 square inches, at least 2 square inches, at least 2.25 square inches, or at least 2.5 square inches. In other examples, fiducial 108 may be further scaled down. For example, the surface area of ​​the reference may be less than 0.75 square inches, less than 0.50 square inches, less than 0.25 square inches, or less than 0.15 square inches. In some embodiments, the reference 108 may take the form of a two-dimensional (2D) quick response (QR) code (in Figures 2A-2B ), or may take the form of a natural feature present or naturally defined along the mechanical device 110 (not shown).

[0031] Reference Figure 2B Further possible features of the fiducials 108 are illustrated by depicted fiducials 108A, 108B, and 108C, which may be three examples of fiducials 108 that may be removed or copied from the calibration patch 202. For example, as shown for fiducial 108A, the fiducial 108 may include an outline 210 (shown in green) defining the perimeter of the shape of the fiducial 108 and a dot 212 (shown in red) located at a point on the outline. The outline 210 indicates the plane coordinates of the fiducial 108A, while the dot 212 indicates the origin of the plane coordinates. Additionally, the unique geometric shapes of the pixelated black and white pattern defined by fiducial 108A and the other fiducials 108 may represent different binary codes. The camera 106 may recognize the different binary codes and may allow identification and / or determination of the different locations along the machine 110 at which a particular fiducial 108 is located. In some embodiments, each fiducial 108 may be associated with a different identifier. For example, reference 108A can be associated with or defined as identifier "10" (based on a QR code or other), reference 108B can be associated with or defined as identifier "40" (based on a QR code or other); and reference 108C can be associated with or defined as identifier "41" (based on a QR code or other).

[0032] Return Reference Figure 1A and Figure 1B Once calibration patch 202, including fiducials 108, is positioned in front of camera 106, the position and orientation of calibration patch 202 relative to camera 106 can be varied over a predetermined period of time while capturing images or image frames using camera 106 to generate calibration image frames 152. In this manner, the actual dimensions of each fiducial 108 on calibration patch 202 can be used to calibrate camera 106. Calibration image frames 152 can then be fed to scene calibration module 154 to facilitate pre-calibration 156 of the 3D positions of fiducials 108. In other words, calibration patch 202 can be used as a reference when calculating an initial pose of camera 106 relative to fiducials 108 (and / or a reference position of fiducials 108 relative to camera 106). Overall, scene calibration module 154 automatically calibrates camera 106 and lens (not shown) and / or computing device 104 using calibration image frames 152, thereby allowing subsequent modules to calculate 3D coordinates for all points of interest on mechanical device 110. As described above, scene calibration using scene calibration module 154 allows computing device 104 and / or camera 106 to identify fiducials 108 in subsequent image frames that are within the field of view of camera 106 , regardless of distance from camera 106 .

[0033] It should be understood that the camera 106 used may include a single camera or a group of cameras or multiple camera devices operating together or independently, such that the camera 106 is not limited to any specific number of cameras or camera devices capable of capturing image data. The camera 106 may further be any electronic device having a camera (e.g., a mobile device, tablet, or laptop) or any camera connected to a computer. Furthermore, the camera 106 may be a stereo camera (not shown) configured to capture 3D images. In embodiments utilizing such a stereo camera, the stereo camera may be used independently to generate property measurements without fiducials 108 as corresponding points. In some embodiments, the camera 106 is configured to have a resolution sufficient to capture selected real features or fiducials 108 for marking the mechanical device 110. Multiple cameras may be used to increase the field of view or provide a stereoscopic view of the mechanical device 110. In one embodiment, the positioning and pose determination of the camera 106 is performed prior to executing the scene calibration module 154. The real features and / or fiducials 108 used to mark the mechanical device 110 may be represented by pixels in 2D images, and the application 102 utilizes these 2D images (and observed changes in the 2D images) as described herein. In one embodiment, the use of a camera 106 capable of capturing high-resolution (1920p×1080p) images can improve the functionality and output of the application 102, since the fiducials 108 or real features can be represented by a greater number of pixels. In some embodiments, when the camera 106 is implemented as a real-time camera (the camera continuously feeds information to the application 102), the image frames 158 are refreshed at a given rate between 0.1 and 144 Hz. The application 102 can operate on one or more files associated with the image frames 158 to convert the image frames 158 to binary (i.e., black and white), sharpen edge resolution, thicken lines, thin lines, etc., to facilitate performing subsequent measurements or locating certain features / fiducials 108 within the image frames 158.

[0034] Once the scene calibration is complete, the fiducials 108 can be deployed or positioned as needed along the desired location of the mechanical device 110. Figure 3AIn the diagram 300 , fiducial 108A can be deployed along the top of body 302 of mechanical device 110, and fiducials 108B and 108C can be positioned along an elongated member 304 extending from body 302 of mechanical device 110, as shown. Additionally, at least one fiducial 108 can be positioned along surface 306, such as fiducial 108D shown. In some embodiments, the fiducials 108 shown can be cut from calibration sheet 202 and placed at different predetermined locations along mechanical device 110. The fiducials 108 positioned along mechanical device 110 can also be duplicated or repeated from calibration sheet 202. The fiducials 108 can be temporarily attached to the surface of mechanical device 110 using tape, a weak adhesive, or magnetically applied. In examples where the fiducials are scaled down, the fiducials 108 can be embedded within a coating of mechanical device 110 or manufactured such that the fiducials are integrated within or onto mechanical device 110. The illustrated locations of fiducials 108A, 108B, and 108C are exemplary only and may be repositioned depending on desired measurements of mechanical device 110 .

[0035] In some embodiments, Figure 3A The illustrated datums 108 allow for measurement of the relative angle between a surface 306 (which may be any flat surface) and a given axis defined along the mechanical device 110. For example, datums 108A and 108D may be used to measure the relative angle between an axis 310 (X1) defined by the surface 306 and an axis 312 (X2) defined by the body 302 of the mechanical device 110. Similarly, datums 108D, 108B, and / or 108C may be used to measure the relative angle between an axis 310 defined by the surface 306 and an axis 312 (X3) defined by the elongated member 304 of the mechanical device 110.

[0036] Return Reference Figure 1B Once fiducial 108 is deployed, camera 106 may be used to acquire or capture one or more images or image frames 158 of fiducial 108 and mechanical device 110. Image frames 158 may define two-dimensional image features of fiducial 108 and mechanical device 110 and / or two-dimensional image features associated with natural or real features of mechanical device 110 (at predetermined locations).

[0037] Using the fiducial tracking module 160, the 2D position of the fiducial 108 in the image frame 158 can be recorded or identified for use in the 3D position estimation module 170. Similarly, in other embodiments, the real feature tracking module 164 can be used to record or identify the 2D image position of a predetermined real feature of the mechanical device 110 in the image frame 158. At this stage, the camera 106 can continuously track the fiducial 108 as long as the mechanical device 110 remains in the image frame. In other words, the pose of the camera 106 calculated by the scene calibration module 154 can be used to calibrate the fiducial 108 and / or the real feature within the natural scene and track them over an extended set of frames. The position and rotation of the fiducial 108 in the camera coordinate frame of the image frame 158 is reported.

[0038] The 3D position estimation module 170 can then be configured to import Figure 1B The data generated in the previous modules of the camera 106 are used to estimate the 3D positions / coordinates 172 of all fiducials 108 (and real features). A recursive filter, such as a Kalman filter, can be used to estimate the 3D positions 172 of new points on the mechanical device 110 based on the pose of the camera 106 and the measurements of the 2D image coordinates 162 of the fiducials / features.

[0039] Once the 3D points 172 are generated, a "Define Physical Properties" module 174 may be implemented that is configured to apply a series of predetermined linear algebraic operations to the 3D points 172 in order to define or identify certain physical properties 176 of the mechanical device 110. A "Performance Measurement" module 178 may then be implemented to process or apply data associated with the physical properties 176 of the mechanical device 110 using additional predefined functions to output any number or type of performance measurements, as further described herein. As an example, a Figure 1B The function of θ determines the relative angle of the elongated member 304 of the mechanical device 110 relative to the surface 306. In this example, where the mechanical device 110 is a gun and the elongated member 304 is a gun barrel, this angle can be used as an input to another function for determining, for example, the distance a bullet or other projectile is expected to travel when released through the elongated member 304. In either case, using Figure 1B , the application 102 is configured to output physical 3-D physical properties or other information that can be used to generate performance measurement results 178 (such as the relative angle between any surface and a given axis of the mechanical device 110) or information useful for evaluating or adjusting the mechanical device 110.

[0040] Many other applications of the system 100 are envisioned. In some embodiments, a set of benchmarks can be placed as stickers on various parts of a golf club, racket, hockey stick, or other sporting equipment for use in swing analysis and fitting. The benchmarks can be read by a camera, and identifiers assigned to the benchmarks can be fed into the system 100 as input. The system 100 can then be used to analyze the initial body motion during the downswing, the club head orientation during different swing segments, and the overall consistency of the swing. During the analysis, the system 100 can assign a value during fitting or teaching that the athlete can see and implement to improve performance.

[0041] More specifically, refer to Figure 3B In the diagram 400, any number of benchmarks 108 can be set along various parts of the user 410 and / or golf club 412 to facilitate analysis of the initial body motion during the downswing, the club head orientation at different swing segments, and the overall consistency of the swing of the golf club 412, which generally defines a shaft 414 and a club head 416. For example, in the diagram 400, benchmark 108E can be set along the upper part of the user 410 (e.g., the shoulder). In this example, benchmark 108E can be attached to or integrated / sewn into the clothing worn by the user 410. In some embodiments, benchmark 108F can be set along one or more arms 420 of the user (left arm 420B as shown). One or more benchmarks 108G can also be set along one or more legs 422 of the user 410. Any number of benchmarks 108 can be used to perform a swing analysis depending on the desired output.

[0042] Additionally, any number of references 108 may be provided along the golf club 412. Figure 3B As shown, the reference 108H can be located along the shaft 414 and the reference 108I can be located along the club head 416, but the present disclosure is not limited to this configuration. In some embodiments, a smartwatch 430 or other wearable device can be used with the system 100. For example, the smartwatch 430 can be used to digitally display one or more references 108 along the wrist 432 of the user 410, which are captured by the camera 106 and displayed by the user 410 as described herein. Figure 1A 4. In conjunction with the functions described herein, the smartwatch 430 may also be used to provide the computing device 104 with additional input data for use in swing analysis or modification of the physical attributes of the club 412. In some embodiments, the smartwatch 430 may include an APPLE 1000 from Apple Inc. in Cupertino, California equipped with multiple sensors (e.g., accelerometers, gyroscopes, magnetometers). The sensors are configured to measure backswing time, downswing time, tempo (ratio of backswing time to downswing time), and other characteristics of a user 410 swinging a golf club 412 .

[0043] In some embodiments, while performing a swing analysis, the system 100 may output one or more swing analysis input values ​​that may be utilized during the fitting of the club 412. Specifically, the system 100 may utilize the swing analysis input values ​​as input to a set of predetermined functions comprising a series of predetermined linear algebraic operations. As further described herein (e.g., Figure 4 ), the linear algebraic operations can, in turn, provide one or more output values ​​corresponding to measurements of physical properties of the golf club 412 that are optimal for the user 410, such as the relative angle between the shaft 414 and the club head 416. In some embodiments, the system 100 applies a three-dimensional function that takes as input a set of coordinates representing a predetermined origin of a local coordinate system (derived from a set of datums 108 or otherwise) and a second set of coordinates representing points on the club 412. The second set of coordinates can correspond to the positions of the datums along the shaft 414 of the club 412. The three-dimensional function calculates the distance between each corresponding coordinate, thereby calculating the distance between the coordinates of the origin representing a horizontal origin and the coordinates representing a horizontal point on the shaft 414, calculating the distance between the coordinates of the origin representing a vertical origin and the coordinates representing a vertical point on the shaft 414, and calculating the distance between the coordinates of the origin representing a point on an axis perpendicular to the horizontal and vertical axes at the origin and the coordinates representing a point on the shaft 414 at the point perpendicular to the horizontal and vertical axes. The output of the three-dimensional function is a set of coordinates representing the axis of the shaft of the club 412. Another function may be applied that performs a cross product operation on an input comprising a set of points defining a plane on the face of club 412 or club head 416, and may generate a normal vector to the plane as an output. Ultimately, the face angle and head angle of club 412 may be calculated by inputting the ratio of the vectors corresponding to the points on club 412 into an inverse trigonometric function (e.g., an inverse tangent function).

[0044] For example, the set of points representing the predetermined origin of the local coordinate system may be (0, 0, 0). The set of points representing the points on club 412 may be selected as (10, 20, 30). Applying the three-dimensional function, the set of coordinates representing the shaft axis vector of club 412 is found to be (10, 20, 30) - (0, 0, 0) = (10, 20, 30).

[0045] If the plane on the face of club 412 is defined by points A=(2, 2, 3), B=(1, 0, 1), and C=(-1, 3, 4), then the coordinates of the normal vector to the plane represented by variable F can be calculated as follows:

[0046] AB x AC = F

[0047] ((1,0,1)-(2,2,3))X((-1,3,4)-(1,0,1))=F

[0048] (-1,-2,-2)X(-2,3,3)=F

[0049] (0,7,-7)=F

[0050] Therefore, the coordinates of the normal vector at the selected point on the face of club 412 are (0, 7, -7).

[0051] Finally, if the ratio of the coordinates used as input for calculating the head angle is 0.88, the head angle may be arctan(0.88), which is equal to 0.73 radians, or approximately 41.63 degrees. Similarly, if the ratio of the coordinates used as input for calculating the face angle is 0.45, the face angle may be arctan(0.45), which is equal to 0.42 radians, or approximately 24.23 degrees.

[0052] System 100 can also be used to improve grip during the manufacture of golf clubs or other sports equipment such as clubs, rackets, or hockey sticks. Currently, club gripping is often performed by eye. This approach results in inconsistent products from different grippers. By applying system 100 not only to the face angle / head angle calculation process but also to the gripping process, assembly time can be reduced and product-to-product consistency can be significantly increased. Other golf-related applications are contemplated.

[0053] In some embodiments, another application of system 100 may be in the construction field. Specifically, system 100 can be used by manufacturers to calculate the angle of one stud relative to another and the distance between studs to ensure more consistent spacing between studs and ensure that the studs are level with each other. For a typical manufacturer, who often must build hundreds of houses in a short period of time, the speed with which system 100 can calculate the angles and distances between individual studs can result in significant savings in labor time and costs. When applied to robotics in the manufacturing process, system 100 can also be used to reduce labor time and costs. A datum 180 can be placed or set on an assembly so that a robot in a manufacturing environment can pick up a sheet of material or a finished part and, using the input provided by the datum, correctly orient it for assembly or another task. In this application, datum 180 may include a QR code that defines an identifier associated with a set of instructions instructing the robot on how to correctly orient the relevant sheet of material or finished part.

[0054] As another example, system 100 can be used to calculate the angle between a gun barrel or arrow shaft and a flat target placed at a predetermined distance. In this way, gun / arrow enthusiasts may benefit from the ability to better position their weapons. As another example, the angle between a spray gun and a selected surface can be calculated. Because car manufacturers use robotic arms to paint car bodies, this can provide closed-loop feedback on the spray angle between the spray gun and the car body surface. Furthermore, the disclosed computer vision may be suitable for detecting gaps in the opaque paint layer of a car body. As another example, the angle and distance between the landing gear on a helicopter or airplane's support column and the approaching helipad or runway can be calculated. This can provide closed-loop automatic control during landing to protect passengers and can be used to provide warnings when the helipad or runway is tilted (for example, when landing a helicopter on a cruise ship during stormy weather and the ship's deck is buckling).

[0055] In addition, the system 100 can also be used for data collection in motorsports. Stickers or other forms of benchmarks 180 can be placed on key components of the vehicle. The benchmarks 180 can be scanned by a camera in communication with the system 100. The system 100 can then use any identifiers associated with the benchmarks 180 as input to analyze important measurements and determine whether the car complies with the rules and regulations associated with its respective racing. The system 100 can use the input to analyze vehicle dynamic suspension angles, among other things. The system 100 can also record suspension travel under various cornering conditions. With respect to any of the above possible applications of the system 100, the benchmarks 108 can be replaced in whole or in part by real features present along the device or object being analyzed, and the camera 106 or other means can be used as described herein to interpret changes in the position, orientation, and other properties of the real features.

[0056] Reference Figure 4 , shows a flowchart 450 (similar to flowchart 150) illustrating a number of other possible modules associated with application 102 and for data creation, processing, and flow (using computing device 104 and camera 106) to measure golf clubs ( Figure 5 510) in the form of a specific non-limiting embodiment of the mechanical device 110. As shown in the following description of the flowchart 450, the golf club 510 can be used to Figure 1A aspects of the system 100 and Figures 2A-2B The calibration sheet 202 and the reference 108 are used to measure and adjust the face angle and the head angle of the golf club 510, while reducing working time and reducing wear and damage to the golf club 510 during measurement and adjustment.

[0057] Generally, the camera 106 can be used to generate calibration image frames 452 associated with a "scene calibration" module 454 to facilitate evaluating a pre-calibrated 3D position (456) of the fiducial 108. The fiducial 108 can then be deployed at a predetermined location along the golf club 510, an image frame 458 of the golf club 510 can be generated, and then a "fiducial tracking" module 460 and / or a "real feature tracking" module 464 (either collaboratively or individually) can be used with the camera 106 to record or identify 2D image coordinates 462 or other 2D image features associated with the fiducial 108. Utilizing the 2D image coordinates 462, a "3D position estimation" module 470 associated with the application 102 is configured to subsequently estimate or generate a set of three-dimensional (3D) points / coordinates 472 corresponding to the fiducial 108 and the real features of the golf club 510. Once such 3D points 472 are created, a "Define Shaft Axis + Face Placement Normal Vector" module 474 may be used, which is configured to apply a series of predetermined linear algebraic operations (represented in 474 as "Define Shaft Axis + Face Placement Normal Vector") to the 3D points 472 in order to define the central axis of the golf club shaft, a plane parallel to the face of the golf club 510, and a normal vector to the plane 476. The "Face Angle + Head Angle Calculation" module 478 may then utilize this data to output the face angle and head angle associated with the golf club 510. In some embodiments, the face angle and head angle associated with the golf club 510 may also be checked against predetermined target specifications by a "Virtual Object Overlay" module 480 against a database 482 of predetermined virtual objects associated with golf clubs and golf accessories.

[0058] More specifically, referring to the scene calibration module 454 of FIG2 , the spatial resolution of the camera 106 may first be calibrated using the fiducial 108. In some embodiments, this may first include calibrating the spatial resolution of the camera 106 using the fiducial 108 or a fiducial array ( Figure 2A 2 (shown as 204 in FIG. 2 ) to position the calibration patch (202 in FIG. 2 ) in front of the camera 106 so that the calibration patch 202 is oriented toward the camera 106. The fiducials 108 may take a variety of forms and shapes as described herein, and in some embodiments, the fiducials 108 may take the form of a two-dimensional (2D) quick response (QR) code ( Figures 2A-2B ), or may take the form of a natural feature existing along the golf club 510 or a naturally defined natural feature (not shown).

[0059] return Figure 4Once the calibration patch 202, including the fiducials 108, is positioned in front of the camera 106, the position and orientation of the calibration patch 202 relative to the camera 106 can be varied over a predetermined period of time while capturing images using the camera 106 to generate calibration image frames 452. In this manner, the actual dimensions of each fiducial 108 on the calibration patch 202 can be used to calibrate the camera 106. The calibration image frames 152 can then be fed to the scene calibration module 454 to pre-calibrate 456 the 3D positions of the fiducials 108. In other words, the calibration patch 202 can be used as a reference for calculating the initial pose of the camera 106 relative to the fiducials 108 (and / or the reference position of the fiducials 108 relative to the camera 106). In general, the scene calibration module 454 automatically calibrates the camera 106 and lens (not shown) and / or the computing device 104 using the calibration image frames 452, thereby allowing subsequent modules to calculate 3D coordinates for all points of interest on the golf club 510, as further described herein. Performing scene calibration using the described scene calibration module 454 allows the computing device 104 and / or camera 106 to identify fiducials 108 in subsequent image frames within the field of view of the camera 106, regardless of the distance from the camera 106. The camera 106 can take many different forms, can capture images, image frames, or real-time streaming data (defined as image frames), and can include one or more cameras, as further described herein.

[0060] Once scene calibration has been completed, the golf club 510 is then ready for deployment and measurement. Figure 5 , a golf club 510 may generally include a golf club head 512 (which may be an iron-type) and a golf club shaft 514. The golf club head 512 is connected to one end of the golf club shaft 514, and a grip 516 is connected to the opposite end of the golf club shaft 514. Suitable materials for the golf club shaft 514 may include steel and graphite. Although the golf club head 512 is shown as an iron-type golf club head, in other embodiments, the golf club head 512 may also be a putter or wood-type club head without departing from the scope of the inventive concepts described herein.

[0061] Golf club head 512 includes a body 518 and a hosel 520 having a cylindrical bore 522 for receiving one end of golf club shaft 514. Body 518 defines a heel end 524 spaced apart from a toe end 526. Figure 5The main body 518 includes a rear surface 532 extending along the back or rear portion of the main body 518 between the heel end 524 and the toe end 526. The main body 518 also includes a front surface 534 extending between the heel end 524 and the toe end 526. The hosel 520 includes a neck 521 connected to the heel end 524 of the main body 518. The neck 521 has a notch (not shown) in its lower surface. The golf club head 512 may also include a golf club face plate 540 having a front surface 542 and a rear surface 544. The club head 512 may be formed from a solid casting, such as by casting or machining. Suitable materials for the club head 512 include, but are not limited to, stainless steel, titanium, aluminum, nickel, titanium alloys, aluminum alloys, nickel alloys, and the like.

[0062] Fiducials 108 (which may be the same fiducials used for scene calibration) may be placed, engaged, or otherwise positioned at various predetermined locations along the golf club 510 at any time prior to generating the image frame 458 of the golf club 510 or prior to placement into the adjustment tool 114. Figures 6A-6B In one example, at least three datums 108 may be placed in at least three different locations on the golf club 510. The first datum (e.g., Figure 2B The datum 108B shown in FIG. 108B is placed on the golf club face plate 540 of the golf club head 512. A second datum (e.g., Figure 2B The datum 108B shown is placed on the first end 550 of the golf club shaft 514 near the golf club head 512. A third datum (e.g., Figure 2B The datum 108C shown is placed on the second end 552 of the shaft 514 opposite the first end 550. The first datum (108A) indicates the face plane of the face plate 540 of the golf club 510. The second and third datums (108B and 108C) indicate the centerline axis of the shaft 514 of the golf club 510. Identifying or observing the face plane with reference to the centerline axis of the shaft 514 allows the application 102 to determine the face angle and head angle of the golf club head, as further described herein. The second and third datums (108B and 108C) further allow the application 102 to determine the length of the shaft 514.

[0063] In other examples, any number of fiducials 108 may be included (e.g., at least four fiducials, at least five fiducials, at least six fiducials, at least seven fiducials, at least eight fiducials, at least nine fiducials, at least 10 fiducials, at least 11 fiducials, at least 12 fiducials, or at least 13 fiducials). Increasing the number of fiducials 108 deployed may improve the accuracy with which application 102 measures various aspects of golf club 510. Furthermore, increasing the number of fiducials 108 may allow application 102 to measure other attributes, such as the roll and lift of a golf club. In some embodiments, the orientation of a fiducial 108 on face plate 540 may be aligned with an edge of the fiducial 108 that is parallel to a plurality of grooves located on face plate 540 of golf club head 512. In other examples, the fiducials 108 on face plate 540 need not be aligned with the plurality of grooves of golf club head 512. The orientation of the fiducials 108 on shaft 514 may be any orientation, as long as there are at least two fiducials 108 on shaft 514.

[0064] In some embodiments, the illustrated fiducial 108 can be cut from the calibration sheet 202 and placed at various predetermined locations along the illustrated golf club 510. The fiducial 108 positioned along the golf club 510 can also be copied or repeated from the calibration sheet 202. The fiducial 108 can be temporarily attached to the surface of the golf club 510 using tape, adhesive, or magnetic application. In examples where the fiducial is scaled down, the fiducial 108 can be embedded within the coating of the golf club 510 or manufactured so that the fiducial is integrated within or onto the golf club 510. The fiducial 108 can also be placed onto the shaft 514 of the golf club 510 by adhering it to a flat surface with an attachment mechanism (e.g., a snap-on clip) that attaches to the shaft 514. The fiducial 108 can also be placed onto the shaft 514 of the golf club head 512 using magnets. The locations of fiducials 108A, 108B, and 108C shown are exemplary only and may be repositioned if different measurements are desired for golf club 510. In one embodiment, fiducials 108 may be printed and measured prior to being applied or positioned along golf club 510.

[0065] Reference Figure 6C , fiducial 108A (and other fiducials 108) may generally define a unique geometric shape of a square and a pixelated black and white pattern. In other embodiments, fiducial 108 may include any shape (e.g., a circle, rectangle, triangle, pentagon, octagon, or any other shape). In some embodiments, fiducial 108 also includes an outline 210 (previously described in Figure 2B) and a dot 212 located at a point along outline 210. Outline 210 specifies the plane coordinates of fiducial 108, while dot 212 indicates the origin of the plane coordinates. The unique geometric shapes of the pixelated black and white pattern represent different binary codes. After scene calibration, the different binary codes are recognized by camera 106, so that computing device 104 and camera 106 are configured to identify at which different locations on golf club 510 a particular fiducial 108 is located.

[0066] Return to reference Figure 4 Once the fiducial 108 is deployed, the two-dimensional position of the fiducial 108 can be recorded, and / or one or more images or image frames 458 of the fiducial 108 and the golf club 510 can be acquired or captured using a camera. The image frames 458 can define two-dimensional image features or image coordinates 462 of the fiducial 108 and the golf club 510, and / or two-dimensional image features associated with natural or real features of the golf club 510 (at predetermined locations). The image frames 458 can further define the 2D image positions of the grooves of the club face 540 and points along the perimeter of the golf club 510 within the image frames. As long as the club 510 remains within the image frames, the camera 106 can continuously track the fiducial 108 or the natural features of the golf club 510. In other words, using the fiducial tracking module 460, the 2D position of the fiducial 108 in the image frames 458 can be recorded or otherwise identified for use in the 3D position estimation module 470. Similarly, in other embodiments, real feature tracking module 464 can be used to record or identify the 2D image location of a predetermined real feature of golf club 510 within image frame 458. The position and rotation of fiducial 108 in the camera coordinate frame of image frame 458 is recorded.

[0067] The 3D position estimation module 470 can then be configured to import Figure 4 The data generated in the previous modules of the algorithm are combined to estimate the 3D positions / coordinates 472 of all fiducials 108 (and real features). A recursive filter, such as a Kalman filter, can be used to estimate the 3D position 472 of a new point along the golf club 510 based on the pose of the camera 106 and the measurements of the 2D image coordinates 462 of the fiducials / features.

[0068] Once the 3D point 472 is generated or otherwise determined, a "Define Shaft Axis + Face Plane Normal Vector" module 474 may be implemented that is configured to apply a series of predetermined linear algebraic operations to the 3D point 472 in order to estimate the shaft axis, face plane, and normal vector (shown as 476). In one embodiment, the vector of the shaft axis ( Figure 6A The S) in can be calculated as:

[0069] S=(xi,yj,zk)-(x′,y′,z′),

[0070] Where (x', y', z') is defined as the origin of the local coordinate system. Figure 6A 560, 562 and 564) are used to define planes on the club face.

[0071] The normal vector F to this plane can be calculated as follows:

[0072] AB×AC=F,

[0073] Where AB and AC are vectors pointing from point A to point B and C respectively.

[0074] Finally, the “Loft Angle + Head Angle Calculation” module 478 may be configured to calculate the head angle of the club as follows:

[0075]

[0076] And calculate the face angle as follows:

[0077]

[0078] In this manner, the application 102 is configured to calculate the face angle and head angle (as vector projections) over a large range of club orientations and club positions relative to the camera 106. As an additional step, the application 102 may check the validity of the face angle and head angle measurements against a 3D model of a target golf club (i.e., having the specified face angle and head angle measurements) within the database 482 of virtual objects.

[0079] In other words, the application 102 is configured to implement Figure 4 The modules and functionality of the present invention allow calibration of the camera 106 and the scene, positioning of the fiducials 108 along the golf club 510, and capturing images of the fiducials 108 and the golf club 510 using the camera 106. The positions and rotations of these fiducials 108 can be reported to determine the centerline shaft vector, with reference to data defining image features associated with the fiducials 108B-108C. Furthermore, the plane of the face 540 of the golf club 510 can be determined with reference to data defining image features associated with the fiducial 108A. The face angle and head angle of the golf club 510 can then be measured based on the centerline shaft vector of the golf club 510 and the plane of the face 540 using the equations described above.

[0080] like Figure 7As shown in , in some embodiments, the golf club 510 being measured and adjusted can remain engaged to the adjustment tool 114 (e.g., a pneumatic vise) throughout the adjustment process. The camera 106 can be positioned so that the camera 106 has a clear line of sight (LOS) to the face and shaft 514 of the golf club 510 being adjusted.

[0081] Test and sample results

[0082] like Figures 8A-8B As indicated, utilizing system 100, application 102 and Figure 4 The system's features (collectively referred to as "non-contact PC vision") were found to significantly reduce the time required to measure the face angle and head angle of a golf club 510 compared to other methods and systems, and the calculations resulted in a suitable error rate. The system's accuracy can exceed .0004" (within a 4" field of view), with typical angular accuracy exceeding 0.05°.

[0083] Importantly, the methods described herein allow for efficient measurement of the face angle and head angle of a golf club 510, and can be accomplished without tools or fixtures. When utilizing a fixture tool 114, aspects of the system 100 can be implemented as a mobile application (app) or computer program that measures golf club 510 attributes without removing the golf club 510 from the fitting tool 114. The camera 106 can feed image data to the application 102 continuously or at intervals during scene calibration and measurement without undesirably removing and reattaching the golf club 510 to the fitting tool 114.

[0084] In other embodiments, inherent features of the golf club 510 can be used for measurement and self-calibration (i.e., the groove spacing and diameter of the shaft 514 at the tip and grip of the golf club 510, not shown). In other embodiments, multiple cameras / detectors using imaging techniques for high-resolution point clouds or meshes can be implemented. All of these techniques can be used with both desktop and mobile implementations of the system 100 described herein.

[0085] Exemplary Computing Components

[0086] Figure 9is an example diagram of a computing device 700 that can implement the various methods discussed herein. For example, computing device 700 can include computing device 104 that executes or obtains functionality and / or aspects of application 102. Computing device 700 includes a bus 701 (i.e., interconnect), at least one processor 702 or other computing element, at least one communication port 703, main memory 704, removable storage media 705, read-only memory 706, and mass storage device 707. Processor 702 can be any known processor, such as, but not limited to, or processor, or Athlon processor, or series processor. Communication port 703 can be any of an RS-232 port for use with a modem-based dial-up connection, a 10 / 100 Ethernet port, a Gigabit port using copper or fiber optic cables, or a USB port. Communication port 703 can be selected based on the network (e.g., a local area network (LAN), a wide area network (WAN), or any network to which the computer device 700 is connected). The computing device may also include a transport and / or transmission network 755, a display screen 760, an I / O port 740, and an input device 745 (e.g., a mouse or keyboard).

[0087] The main memory 704 may be a random access memory (RAM) or any other dynamic storage device known in the art. The read-only memory 706 may be any static storage device, such as a programmable read-only memory (PROM) chip for storing static information such as instructions for the processor 702. The mass storage device 707 may be used to store information and instructions. For example, a hard disk (e.g., series of Small Computer Serial Interface (SCSI) drives), optical disks, disk arrays (such as Redundant Array of Independent Disks (RAID), such as series of RAID drives), or any other mass storage device.

[0088] Bus 701 communicatively connects processor 702 to other memory, storage devices, and communication blocks. Bus 701 can be a system bus based on PCI / PCI-X, SCSI, or Universal Serial Bus (USB) (or other), depending on the storage devices used. Removable storage media 705 can be any type of external hard drive, thumb drive, compact disc – read only memory (CD-ROM), compact disc – rewritable (CD-RW), digital video disk – read only memory (DVD-ROM), etc.

[0089] The embodiments herein can be provided as a computer program product, which can include a machine-readable medium having an instruction stored thereon, which can be used to program a computer (or other electronic device) to perform a method. The machine-readable medium can include, but is not limited to, an optical disc, a CD-ROM, a magneto-optical disc, a ROM, a RAM, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or other types of media / machine-readable media suitable for storing electronic instructions. In addition, the embodiments herein can also be downloaded as a computer program product, wherein the program can be transmitted from a remote computer to a requesting computer via a communication link (e.g., a modem or a network connection) by a data signal contained in a carrier wave or other propagation medium.

[0090] As shown, the main memory 704 may be encoded with the application 102 that supports the functionality discussed above. In other words, aspects of the application 102 (and / or other resources described herein) may be embodied as software code that supports processing functionality according to the various embodiments described herein, such as data and / or logic instructions (e.g., code stored in memory or another computer-readable medium (e.g., disk). During operation of one embodiment, the processor 702 accesses the main memory 704 using the bus 701 to facilitate, for example, launching, running, executing, interpreting, or otherwise performing processing based on the application 102 stored in the main memory or tangibly stored by executing logic instructions on the processor 702.

[0091] The above description includes example systems, methods, techniques, instruction sequences and / or computer program products that embody the techniques of the present disclosure. However, it will be understood that the described disclosure may be practiced without these specific details. In the present disclosure, the disclosed methods may be implemented as device-readable instruction sets or software. Furthermore, it will be understood that the specific order or hierarchy of steps in the disclosed methods are examples of example methods. Based on design preferences, it will be understood that the specific order or hierarchy of steps in the method may be rearranged while remaining within the disclosed subject matter. The accompanying method claims present the elements of the various steps in an example order and are not necessarily meant to be limited to the specific order or hierarchy presented.

[0092] The described disclosure may be provided as a computer program product or software, which may include a machine-readable medium having stored thereon instructions that may be used to program a computer system (or other electronic device) to perform methods according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form (e.g., software, a processing application) readable by a machine (e.g., a computer). Machine-readable media may include, but are not limited to, optical storage media (e.g., CD-ROM); magneto-optical storage media, read-only memory (ROM); random access memory (RAM); erasable programmable memory (e.g., EPROM and EEPROM); flash memory; or other types of media suitable for storing electronic instructions.

[0093] Certain embodiments are described herein as including one or more modules. Such modules are hardware-implemented and therefore include at least one tangible unit capable of performing certain operations and may be configured or arranged in some manner. For example, a hardware-implemented module may include dedicated circuits that are permanently configured to perform certain operations (e.g., as a dedicated processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)). A hardware-implemented module may also include programmable circuits (e.g., as contained in a general-purpose processor or other programmable processor) that are temporarily configured by software or firmware to perform certain operations. In some example embodiments, one or more computer systems (e.g., stand-alone systems, client and / or server computer systems, or peer computer systems) or one or more processors may be configured by software (e.g., an application or application portion) as hardware-implemented modules for performing certain operations described herein.

[0094] Accordingly, the term "hardware-implemented module" or "module" encompasses a tangible entity, i.e., an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner and / or perform certain operations described herein. Considering embodiments in which the hardware-implemented modules are temporarily configured (e.g., programmed), each hardware-implemented module need not be configured or instantiated at any one instance in time. For example, where the hardware-implemented modules include a general-purpose processor configured using software, the general-purpose processor can be configured as various different hardware-implemented modules at different times. The software can thus configure the processor, for example, to constitute a particular hardware-implemented module at one instance in time and to constitute a different hardware-implemented module at a different instance in time.

[0095] A hardware-implemented module can provide information to other hardware-implemented modules and / or receive information from other hardware-implemented modules. Therefore, the described hardware-implemented modules can be considered to be communicatively coupled. In the case where there are multiple such hardware-implemented modules at the same time, communication can be achieved by signal transmission (e.g., through appropriate circuits and buses) connecting the hardware-implemented modules. In embodiments where multiple hardware-implemented modules are configured or instantiated at different times, communication between such hardware-implemented modules can be achieved, for example, by storing and retrieving information in a memory structure accessible to multiple hardware-implemented modules. For example, a hardware-implemented module can perform an operation and the output of the operation can be stored in a storage device to which it is communicatively coupled. Then, another hardware-implemented module can access the storage device at a later time to retrieve and process the stored output. A hardware-implemented module can also initiate communication with an input or output device.

[0096] It is believed that the present disclosure and its attendant advantages will be understood from the foregoing description, and it should be apparent that various changes can be made in the form, construction, and arrangement of components without departing from the subject matter disclosed herein or without sacrificing all of its substantial advantages. The forms described are illustrative only, and the appended claims are intended to encompass and include such changes.

[0097] Although the present disclosure has been described with reference to various embodiments, it should be understood that these embodiments are illustrative and the scope of the present disclosure is not limited to them. Many variations, modifications, additions and improvements are possible. More generally, embodiments according to the present disclosure have been described in the context of specific embodiments. Functions may be separated or combined in a manner different from that of the various embodiments of the present disclosure, or functions may be described in different terms. These and other variations, modifications, additions and improvements may fall within the scope of the present disclosure as defined by the appended claims.

[0098] Terms

[0099] Item 1: A system for measuring properties of a golf club, comprising: a camera configured to capture an image of a golf club and a plurality of benchmarks; and a computing device in operative communication with the camera, the computing device configured to: acquire a first set of pixels associated with each of a plurality of benchmarks on the first image to calibrate a spatial resolution of the camera, acquire a second image from the camera, the second image comprising the golf club and a plurality of benchmarks positioned at predetermined positions along the golf club, convert the second image into a second set of pixels associated with each of the plurality of benchmarks, generate a set of points corresponding to the plurality of benchmarks based on a comparison between the first set of pixels and the second set of pixels, the set of points defining estimated three-dimensional features of the plurality of benchmarks in the second image, apply the set of points to a series of predetermined functions to generate club characteristic data, the club characteristic data defining a central axis of the shaft of the golf club, a plane parallel to the face of the golf club, and a normal vector to the plane, and output, based on the club characteristic data, a measurement result including a face angle and a head angle associated with the golf club.

[0100] Clause 2: The system of clause 1, wherein the plurality of fiducials comprises discrete fiducials that are removable from the golf club.

[0101] Clause 3: The system of clause 2, wherein the discrete references comprise a set of two-dimensional Quick Response (QR) codes.

[0102] Clause 4: The system of clause 1, wherein the plurality of benchmarks comprises natural pre-selected characteristics of the golf club.

[0103] Clause 5: A system according to clause 1, wherein calibrating the spatial resolution of the camera comprises: providing a sheet, the plurality of fiducials being defined as an array along a side of the sheet, positioning the sheet in front of the camera so that the side of the sheet is oriented toward the lens of the camera, and while capturing one or more images, changing the position and orientation of the sheet relative to the camera to generate calibration data, wherein the calibration data is used by the computing device to determine a set of points corresponding to the plurality of fiducials.

[0104] Clause 6: The system of clause 5, wherein the sheet is used as a reference for computing an initial pose of the camera.

[0105] Clause 7: The system of clause 1, wherein the second set of pixels is a portion of an array of pixels defining the entire second image, wherein the second set of pixels defines positions and sizes for a plurality of fiducials in the second image and further defines an array of integers.

[0106] Clause 8: The system of clause 1, wherein the computing device generates the point set using a recursive filter based on the pose of the camera and the second set of pixels.

[0107] Clause 9: The system of clause 1, wherein a portion of the plurality of fiducials is assigned to a face of the golf club and a second portion of the plurality of fiducials is assigned to a shaft of the golf club.

[0108] Clause 10. A system according to clause 1, wherein the predetermined function set includes: a series of one or more predetermined linear algebraic operations including the following functions: a three-dimensional (3D) l function, the 3D function including an input including a set of coordinates representing the origin of a local coordinate system and a second coordinate representing a predetermined position along the golf club, the 3D function being configured to generate an output including a third set of coordinates including a vector representing the shaft axis of the golf club; a first function, the first function performing a cross product operation on a first function input, the first function input including a set of points defining a plane on the face of the golf club, the first function being configured to generate a first function output defining a normal vector of the plane; a second function, the second function including a second function input including a ratio of vectors, the second function being configured to generate a second function output defining an angle representing the head angle of the golf club; and a third function, the third function including a third function input including a ratio of vectors, the third function being configured to generate a third function output defining an angle representing the face angle of the golf club.

[0109] Item 11: A method comprising: providing a processor in operative communication with a camera assembly, the processor being configured to: acquire a first image from the camera assembly, the first image comprising a plurality of fiducials; convert the first image into a plurality of first picture elements associated with each of the plurality of fiducials; acquire a second image from the camera assembly, the second image comprising a plurality of fiducials positioned at predetermined positions along a golf club; convert the second image into a plurality of second picture elements associated with each of the plurality of fiducials; generate a set of points defining virtual three-dimensional aspects of the plurality of fiducials based on differences between the plurality of first picture elements and the plurality of second picture elements; apply the set of points as input to a series of predefined linear algebraic operations to generate golf club characteristic data; and generate output measurement values ​​defining a face angle and a head angle associated with the golf club based on the golf club characteristic data.

[0110] Clause 12: The method of clause 11, wherein the camera assembly comprises one or more cameras.

[0111] Clause 13: The method of clause 11, wherein the plurality of first picture elements and the plurality of second picture elements comprise pixels represented as a two-dimensional array of integers.

[0112] Clause 14: The method of clause 11, further comprising continuously tracking, with the camera assembly, the plurality of fiducials while positioning the golf club within an image frame associated with the camera assembly.

[0113] Clause 15: The method of clause 11, further comprising: engaging the golf club with the pneumatic vise of the adjustment clamp to position the golf club within a clear line of sight of the camera assembly.

[0114] Clause 16: The method of clause 11, further comprising, prior to the calibrating step, positioning the camera assembly in a predetermined position relative to a plurality of fiducials.

[0115] Clause 17: The method of clause 11, wherein the plurality of fiducials include features of the golf club including grooves, logos, and the face of the golf club.

[0116] Clause 18: The method of clause 11, wherein the first and second datums of the plurality of datums are defined along a sole groove of the golf club, and wherein a third datum of the plurality of datums is defined along a face of the golf club.

[0117] Clause 19: An apparatus comprising: a camera; and a computing device configured to: obtain a plurality of picture elements associated with a plurality of benchmarks from a first image generated by the camera, identify a series of changes in the plurality of picture elements from a second image depicting the plurality of benchmarks positioned along a golf club, generate a 3D point set based on the series of changes in the plurality of picture elements associated with the plurality of benchmarks, apply the 3D point set as input to one or more predetermined linear algebraic operations to generate golf club characteristic data, and output measurement results defining a face angle and a head angle associated with the golf club based on the golf club characteristic data.

[0118] Clause 20: Apparatus according to clause 19, wherein the golf club characteristic data comprises a central axis of the shaft of the golf club, a plane parallel to the face of the golf club, and a normal vector to the plane, and wherein the outputted measurement results are checked against predetermined target specifications associated with the golf club.

[0119] Clause 21: The apparatus of clause 19, wherein the plurality of picture elements associated with the plurality of fiducials comprise pixels of a two-dimensional image, and the series of changes in the picture elements associated with the plurality of fiducials comprise changes in pixels reflected by the second image.

[0120] Item 22: An apparatus comprising: a camera; and a computing device configured to obtain a plurality of two-dimensional image features associated with a plurality of references from a first image generated by the camera, identify changes in position and rotation of the plurality of references based on one or more changes in the plurality of two-dimensional image features observed via a second image depicting the plurality of references positioned along a mechanical device, generate a 3D point set based on the one or more changes in the plurality of two-dimensional image features associated with the plurality of references, apply the 3D point set as input to one or more predetermined linear algebraic operations to generate physical property data associated with the mechanical device, and output a performance measurement associated with the mechanical device based on the physical property data.

[0121] Clause 23: The apparatus of clause 22, wherein the plurality of fiducials comprises discrete QR codes positioned at predetermined locations along the mechanical device.

[0122] Clause 24: The apparatus of clause 22, wherein the plurality of fiducials comprises natural or real features associated with predetermined positions of the mechanical device.

Claims

1. A system for analyzing a golf swing, the system comprising: a processor in operable communication with at least one camera, the processor being configured to: acquiring one or more images of a golf swing captured by the camera, the one or more images being associated with a plurality of references; generating a first set of coordinates defining virtual three-dimensional aspects of the plurality of fiducials; obtaining data associated with a physical property of the golf club based on the first set of coordinates; inputting the data associated with the physical attributes of the golf club as one or more swing analysis input values ​​into a set of predetermined linear algebraic operations; outputting one or more values ​​corresponding to optimal measurements of a physical property of the golf club specific to the user swinging the golf club; The multiple benchmarks include: a first datum disposed along the shaft of the golf club; a second datum disposed along the head of the golf club; and A third reference is positioned along the user's body.

2. The system according to claim 1, further comprising: a wearable device for use with the system; wherein the wearable device includes one or more sensors; The wearable device is configured to provide additional input data to the processor for inclusion in the set of predetermined linear algebraic operations.

3. The system according to claim 2, wherein: The wearable device is configured to digitally display one of the plurality of benchmarks.

4. The system according to claim 2, wherein: The one or more sensors are selected from the group consisting of an accelerometer, a gyroscope, and a magnetometer.

5. The system according to claim 4, wherein: The one or more sensors are configured to measure one or more characteristics of a golf swing.

6. The system according to claim 5, wherein: The one or more characteristics of the golf swing include downswing timing.

7. The system according to claim 1, wherein: The third fiducial is attached to a garment worn by the user.

8. The system according to claim 1, wherein: The optimal measurement includes the relative angle between the shaft and the club head.

9. The system according to claim 1, wherein: The first benchmark comprises a natural preselected characteristic of the golf club.

10. The system according to claim 1, wherein: The second benchmark comprises a natural preselected characteristic of the golf club.

11. The system according to claim 10, wherein: The second datum is defined along a sole groove of the golf club.

12. The system according to claim 1, wherein: At least one of the plurality of fiducials includes a set of 2D Quick Response (QR) codes.

13. The system of claim 1, wherein: The calibration of the spatial resolution of the at least one camera comprises: providing a sheet, the plurality of fiducials being defined as an array along a side of the sheet, positioning the sheet in front of the at least one camera so that the side of the sheet is oriented toward the lens of the at least one camera, and When capturing one or more images, calibration data is generated, The calibration data is used by the processor to determine the first set of coordinates corresponding to the plurality of references.

14. The system according to claim 13, wherein: The sheet is used as a reference for calculating an initial pose of the at least one camera.

15. The system of claim 1, wherein: The at least one camera is integrated with the computing device.

16. The system of claim 1, wherein: The at least one camera is a stereo camera.

17. The system of claim 1, comprising a plurality of cameras configured to provide a stereoscopic view of the golf club.

18. The system of claim 1, wherein: The third reference is located along the user's arm.

19. The system of claim 1, wherein: The third reference is located along a leg of the user.

20. The system of claim 1, wherein: The plurality of references include a plurality of references disposed along a body of the user.

Images