Device, system, and method for measuring performance data and pairing the data with a performance video

Abstract

A measuring device having one more electronic components, including one or more sensors located in a housing. The measuring device can be used to measure motion or movement of a sport equipment, such as a surfboard or a jet ski, which is controlled or operated by a user. The measuring device can record timestamped measured motion data and upload data to a computer to process, along with a performance video clip of a user using the sport equipment having the measuring device mounted thereto. Software running on the computer can generate a paired video clip showing both the performance video clip and the measured timestamped measured motion data shown as one or more measured usable parameters.

Description

Atty Dkt No. 2161-001.502Via Patent CenterDEVICE, SYSTEM, AND METHOD FOR MEASURING PERFORMANCE DATA AND PAIRING THE DATA WITH A PERFORMANCE VIDEOCROSS-RELATED TO APPLICATION(S)

[0001] This PCT application claims priority to U.S. Provisional Application No. 63 / 701,651, filed on October 1, 2024, the contents of which are expressly incorporated herein by reference as if set forth in full.FIELD OF ART

[0002] The present disclosure is generally related to a device, system, and method for measuring performance data, such as a surfing performance by a surfer, and then paring the measured data with a video of the performance to provide objective indicia of the performance.BACKGROUND

[0003] Water sports, such as surfing, boogie boarding, water skiing, and jet ski racing are hugely popular among amateurs and professionals alike. While professionals typically have teams, resources, and coaches with training tools to help them analyze and improve in their water sports, amateurs are typically relegated to trial and errors when trying to improve. Having usable performance data can help to improve performance. The situation is similar for land-based sports, such as snow skiing and motorcycle racing.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] These and other features and advantages of the present devices, systems, and methods will become appreciated as the same becomes better understood with reference to the specification, claims and appended drawings wherein:

[0005] FIG. 1 is a top view of a measuring device in accordance with aspects of the invention, shown next to a quarter for reference purposes.

[0006] FIG. 2 is a schematic view- of exemplary7components of the measuring device of FIG. 1.

[0007] FIG. 3 is a traction pad having the measuring device located in a holding cavity, the traction pad is configured for use with a surfboard.

[0008] FIG. 3A is an expanded view of the holding cavity and the measuring device of FIG. 3.Atty Dkt No. 2161-001.502Via Patent Center

[0009] FIG. 4 is a performance measuring system in which various example embodiments described in the present disclosure may be implemented.

[0010] FIG. 5 is an exemplary user interface showing the measured performance data of a surfer displayed with a video of the surfer's performance in the ocean.

[0011] FIG. 6 is an exemplary user interface showing the measured performance data of a surfer (not shown) displayed with a video of the surfer's performance in a surf pool.

[0012] FIG. 7 is a paired video showing a surfer’s performance displayed with the surfer’s performance data.

[0013] FIG. 8 is a block diagram of a computer system used to enable a user to interact with the measuring device of FIG. 1 for the purpose of pairing the data with the performance video.

[0014] FIG. 9 is a flow diagram illustrating how data from a sports activity, received from the measuring device of FIG. 1, is processed to select sport runs, and to determine various measurements of performance data during each sport run.

[0015] FIG. 10 is a flow diagram illustrating data intake, processing, and storage of a sport session.

[0016] FIG. 11 is a flow diagram illustrating the function of a wave detection algorithm.DETAILED DESCRIPTION

[0017] The detailed description set forth below in connection with the appended drawings is intended as a description of the presently preferred embodiments of devices, systems, and methods for measuring performance data, such as a surfing performance by a surfer, and then paring the performance data with the video of the performance to provide subjective indicia of the performance. The exemplary embodiments described herein are not intended to represent the only forms in which the present devices, systems, and methods may be constructed or utilized. The description sets forth the features and the steps for constructing and using the embodiments of the present devices, systems, and methods in connection with the illustrated embodiments. It is to be understood, however, that the same or equivalent functions and structures may be accomplished by different embodiments that are also intended to be encompassed within the spirit and scope of the present disclosure. As denoted elsewhere herein, like element numbers are intended to indicate like or similar elements or features.Atty Dkt No. 2161-001.502Via Patent Center

[0018] Descriptions of technical features or aspects of an exemplary configuration of the disclosure should typically be considered as available and applicable to other similar features or aspects in another exemplary configuration of the disclosure. Accordingly, technical features described herein according to one exemplary configuration of the disclosure may be applicable to other exemplary configurations of the disclosure, and thus duplicative descriptions may be omitted herein.

[0019] With reference now to FIG. 1, a top view of a measuring device 100 in accordance with aspects of the invention is shown next to a quarter coin. The current view shows relative sizes between the measuring device 100 and the quarter. The measuring device has a housing 102 having various electronic components enclosed therein, as further discussed below. The housing maybe sealed and water-tight for embodiments that are used in water, but this is not required if used in a land based sport. The housing may be generally cuboid in shape, but a wide range of shapes and sizes may be used, depending upon the needs of the user. However, the measuring device 100 can be practiced with a different size and shaped housing, such as round, a differently shaped polygon, or an irregular shaped housing. The housing material may be made from any suitable material, including but not limited to a plastic polymer material, a rubber material, an elastomeric material, or a thermoplastic elastomer (TPE) material. An optional opening may be incorporated with the housing to enable maintenance or facilitate assembly of electronic components thereinto, which can subsequently be enclosed with a cap or a door. The housing includes a mounting component for mounting the housing on the sport equipment, or otherwise associates the housing with the user. In one embodiment, the mounting component is in the form of an adhesive layer for bonding the housing to a surfboard, or a part mounted on the surfboard.

[0020] FIG. 2 is a schematic view of the measuring device 100 of FIG. 1. In an example, the measuring device 100 contains or is embedded with hardware and / or software components that enable the device 100 to measure and / or detect the performance data and other parameters. In an example, the performance data may include acceleration and linear velocity in all directions, and may include other parameters which can include water temperature, humidity, and other data determined useful by those skilled in the art. In some embodiments, the measuring device 100 may include one or more sensors, such as an accelerometer, a gyroscope, a gravimeter, a magnetometer, an altimeter, and / or other like sensors. Data collected by the one or more sensors may be configured to determine a magnitude and direction of a velocity,Atty Dkt No. 2161-001.502Via Patent Center acceleration, and / or motion of the measuring device 100 when the device is mounted to a sport equipment, such as a surfboard, a jet ski, or snow skis.

[0021] The one or more sensors may be configured to detect the motion of the surfboard as sensor data. The sensor data may then be passed to one or more processors and / or a sensor hub located in the housing of the measuring device 100 or in an external computer so that the sensor data can be processed and displayed as flow, linear velocity, acceleration, vertical drop, and / or any other t pe of format of analyzed, processed, or formatted data. Furthermore, the measuring device 100 may track the performance by timestamping the sensor data during data collection and / or when processing the data.

[0022] The measuring device 100 may also include hardware and / or software components that enable the measuring device 100 to communicate with one or more other devices, such as a laptop, a gateway, or another computer device over a network. In this regard, the measuring device 100 may include a transceiver, which is broadly defined to include any form of transmitter / receiver or other form of communications technology. To communicate with other devices, the measuring device 100 may transmit and / or receive signals according to a wireless communications protocol, such as to update software and firmware. The wireless communications protocols may be any suitable set of standardized rules or instructions implemented by the measuring device 100 to communicate with other devices. Examples of such wireless communications protocols may include, cellular communications protocols (e.g., Long Term Evolution (LTE), Universal Mobile Telecommunications System (UMTS). Global System for Mobile Communications (GSM) and / or Enhanced Data GSM Environment (EDGE), Wi-MAX or IEEE 802.16 protocols, and the like), Local Area Network (LAN), Wide Area Network (WAN), or wide LAN (WLAN) protocols (e.g., Wi-Fi-based protocols or Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocols), and person-to- person (P2P) or personal are network (PAN) protocols (e.g., IEEE 802.15.4 based protocols including ZigBee, IPv6 over Low power Wireless Personal Area Networks (6L0WPAN), WirelessHART, MiWi, Thread, and the like; WiFi-direct; Bluetooth or Bluetooth Low Energy (BLE) protocols; ANT protocols; Z-Wave; LTE device-to-device (D2D) or Proximity Services (ProSe); Universal Plug and Play (UPnP); and the like). The measuring device 100 may be a telemetry device or system.

[0023] As further discussed below, the measuring device 100 is configured to capture and / or record data associated with a sports activity, such as surfing, boogie boarding, jet skiAtty Dkt No. 2161-001.502Via Patent Center racing, water skiing, and snow skiing, to name a few non-limiting examples. Sport activities typically involve the use of sport equipment, such as a surfboard or a jet ski. and the measuring device 100 can be mounted to the sport equipment for recording motion of the sport equipment, as manipulated or maneuvered by the user of the equipment. Thus, the measuring device 100 is configured to measure action or movement involving the sport equipment, as used or manipulated by the user, to indirectly measure the performance of the user. The measured data can include at least one of velocity and acceleration measurement of the sport equipment, a change in direction of a motion, such as a vertical drop, a change in speed / velocity. a state / position / orientation change of the computer device and therefore the sport equipment, a temperature change, and flow.

[0024] As used herein, flow is a unitless measurement that considers the change in speed or velocity relative to the threshold or preferred baseline. For example, if a surfer rides a wave at a constant velocity, then the flow indicator is a flatline. If the surfer speeds up or slows down, then the indicator will deviate from the flat line. In an example, the measurement device 100 is configured to measure rail-to-rail movement, which is the side-to-side tilting of the surfboard by the surfer to generate speed. A flow indicator can provide feedback to the surfer that reflects the surfer’s rail-to-rail movement when catching a wave. Thus, as an example, a flow indicator can show a flat line when the surfer initially catches a w ave and can change as the surfer speeds his rail-to-rail movement near the end of the ride.

[0025] In various embodiments, an event may be measured by the measuring device 100 after a certain velocity threshold is reached and the measuring device may stop measuring when the velocity falls below' the threshold. For example, when used with a surfboard, the surfboard can bobble up and down in the ocean as the surfer waits for the ideal wave. During this waiting period, the measuring device 100 does not measure and / or store motion data. Once data associated with an event is captured and recorded by the measuring device 100, with a timestamp, the captured data may be uploaded to a local computer device via a direct wireless connection, such as via Bluetooth connection. In some embodiments, the captured data maybe relayed through a gateway and / or network and reported to a coach or a trainer for processing and / or analysis, as further discussed below.

[0026] FIG. 2 shows exemplary components of an embodiment of the measuring device 100 in accordance with aspects of the invention. In an example, the measuring device 100 may comprise a transceiver 105, a power source 110 such as a rechargeable power supply and powerAtty Dkt No. 2161-001.502Via Patent Center management circuitry, a processor circuitry' 115, a computer memory' 120, which can be a computer readable media, such as a solid state drive, a network interface circuitry’ 130. an input / output (I / O) interface 135, one or more sensors 150, and a sensor hub 155. The various components are coupled with one another via a bus 145. In other examples, fewer or additional components may be incorporated with the measuring device 100.

[0027] As the housing 102 is sealed, the power supply can be wirelessly recharged or a sealed cap can be removed to expose the battery for charging. A magnetic switch can be incorporated for powering up the measuring device 100. An external magnet can be placed next to the housing 102 to turn on the measuring device and the magnetic pole can be terminated, such as by moving the magnet away from the housing, to turn off the measuring device.

[0028] FIG 3 is a perspective rear view of a traction pad 200 provided in accordance with aspects of the invention for use with a surfboard. In this embodiment, the mounting component is in the form of a traction pad 200. The traction pad 200 can be made from a foam material, such as EVA foam, and can include a plurality of gripping surfaces 202, such as projections or bumps, and a stepped or shoulder 204 to support a surfer’s foot. Adhesive can be provided on the underside surfaces of the traction pad for adhering the traction pad to the surfboard.

[0029] In an example, the traction pad 200 is made up of two or more pad sections. As shown, the traction pad 200 has a central pad section 210 and two side pad sections 212, 214. The two or more pad sections allow the traction pad 200 to be placed on different sized surfboards, by spacing the multiple pad section 210, 212, 214 as desired for coverage of the tail end of the surfboard 302 (FIG. 4).

[0030] A holding cavity 220 can be provided at one end of the traction pad 200. In the example shown, the holding cavity 220 is formed in the stepped section 204 of the traction pad 200. In an example, each of the two side pad sections 212, 214 can have a cut-out so that the two cut-outs from the two side pad sections can define the holding cavity 220 for holding the measuring device 100. A retaining lip 222 can be incorporated with a base to retain the measuring device 100 against the wall surfaces of the holding cavity 220. In some examples, straps or buckles, such as a VELCRO fastener, can be used to more firmly secure the measuring device 100 to the traction pad.

[0031] FIG. 3A is an enlarged view of the holding cavity7220 and the measuring device 100 of FIG. 3.Atty Dkt No. 2161-001.502Via Patent Center

[0032] FIG. 4 shows a measuring system 300 provided in accordance with aspects of the invention. The measuring system 300 can comprise the measuring device 100. alocal computer 380 for receiving and processing data measured by the measuring device 100, a gateway 382 to allow the local computer 380 to connect to a network 384, such as the Cloud, to enable one or more remote computers 386 to access the measured data. For example, an expert, a coach, a trainer, a sports equipment manufacturer, or other users can use the remote computer 386 to review uploaded data to then provide input or recommendations to the performer to help the performer improve, based at least in part on measurements captured by the measuring device 100. A sports equipment manufacturer can also use the uploaded data as input for improving or modifying the sports equipment. The uploaded data can be measured data taken directly from the measuring device 100 or can be processed data that used the measured data, as further discussed below.

[0033] The local computer 380 and the one or more remote computers 386 can be a laptop, a desktop, a touchscreen tablet, a smartphone, or any computing devices with a processor that can process data from the measuring device 100. The measuring device 100 can upload data to the local computer 380 via a wireless connection, such as a Bluetooth connection, or a wired connection, which is less preferred.

[0034] With reference now to FIG. 5, a computer display 400 of a software application 402 is shown, which has the measured data 404 arranged on the left side of the screen and a video clip 406 of the performer’s performance for which the measuring device 100 was used to measure motion data is shown on the right side of the screen.

[0035] After uploading measured data to a computer, software 402 running on the computer can be used to process the measured data. As shown, the software processed the measured data to display four different parameters, with fewer or additional parameters contemplated. For example, water temperature readings can also be displayed. As shown, the parameters include flow measurement 410, linear velocity 412, acceleration 414, and vertical drop 416. In an example, quaternions are used to describe orientation or rotations in 3D space using an ordered set of four numbers. Quaternion functions are a unique way to describe any three-dimensional rotation about an arbitrary axis. The measurement is timestamped and the software displays a time line 418 of the measured event. The time line 418 can be incrementally sped up or reversed to help sync the measured event with the video clip 406 of the performance.Atty Dkt No. 2161-001.502Via Patent Center

[0036] The video clip 406 of the performance for which the measuring device 100 is used to measure motion data is also timestamped. For example, during a surfing competition, the measuring device 100 can be mounted to a surfboard to measure motion of the surfboard used by the surfer 500 (FIG. 5). The surfing event also video records the surfer 500 as the surfer rides the various waves during the surfing competition, such as to provide contents for televisions. The video recording is also timestamped. In other embodiments, video recordings or clips can be taken from beach mounted cameras, e.g., beach cams, which are generally available year round, such as those offered by SURFLINE and SURFERSWAREHOUSE, which may require video subscriptions to access. In still other instances, beach cams at known public beaches and hotel resorts can be accessed by surfers for use with the measuring device 100 and system in accordance with aspects of the invention.

[0037] After the video clip 406 is uploaded to the computer, the software 402 can display the video clip with a timeline 422, which can include incremental frame adjustments by allowing the user to speed up or reverse the video frame in one to five second increments, similar to the measured data. Other incremental changes can be programmed into the software. The software provides a side-by-side view of the measured data and the video clip, as shown in FIG. 5. The computer user can then adjust the timeline of the measure data and of the video clip, using the timestamps, to sync the two.

[0038] Thus, the present measuring system 300 (FIG. 4) is configured to display various measured performance data with a video clip of the performance. In an example, the measured data can include one or more of flow, vertical drop, linear velocity, acceleration, and water temperatures. If measured, water temperature data can provide feedback on whether the surfer’s performance is affected by the warm or cold temperatures.

[0039] As further discussed below, the software 402 is further programmed to overlay the motion data measurements over the video clip, and then capturing the overlayed clip with the measured data to generate a paired video clip 510, as shown with reference to FIG. 7. The paired video clip 510 is a single video showing both the performance and the measured data. The paired video clip 510 allows the video to be shared as a single file without requiring a second user of the shared file to re-process the video clip and the measured data and then syncing the two to generate usable video, as discussed above with reference to FIG. 5.Atty Dkt No. 2161-001.502Via Patent Center

[0040] FIG. 6 is similar to FIG. 5 but shows the video clip captured at a surf generating pool rather than the ocean. Indeed, the present measuring device and system is usable in various applications for tracking various sports equipment aside from surfing.

[0041] Methods of making and of using the measuring device and components thereof, such as the traction pad, are within the scope of the present invention.

[0042] FIG. 8 is a block diagram of a computer system 800 used to enable a user to interact with the measuring device 100 of FIG. 1 for the purpose of pairing the performance data with the performance video. As shown in FIG. 8, in this embodiment the computer system 800 is implemented on a cloud computing web sendee 802, with this exemplary embodiment being Amazon Web Services® (AWS), although equivalent systems may alternatively be used.

[0043] For purposes of this application, the terms “computer;’ “computer device,” “server,” and similar terms, refer to a device and / or system of devices that include at least one computer processor, and some form of computer memory having a capability to store data. The computer may comprise hardware, software, and firmware for receiving, storing, and / or processing data as described below. For example, a computer may comprise any of a wide range of digital electronic devices, including, but not limited to, a server, a desktop computer, a laptop, a smart phone, a tablet, or any form of electronic device capable of functioning as described herein.

[0044] The term “computer processor” as used herein refers to an electrical component that performs operations on an external data source, such as a computer memory, typically in the form of a microprocessor, although any equivalent structure may be used.

[0045] The term “computer memory” as used herein refers to any tangible, non-lransitoiy storage that participates in providing instructions to a processor for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and any equivalent media known in the art. Non-volatile media includes, for example, ROM, magnetic media, and optical storage media. Volatile media includes, for example, DRAM, which typically sen es as main memory. Common forms of computer memory' include, for example, hard drives and other forms of magnetic media, optical media such as CD-ROM disks, as well as various forms of RAM, ROM, PROM, EPROM, FLASH-EPROM, solid state media such as memory cards, and any other form of memory chip or cartridge, or any other medium from which a computer can read. While several examples are provided above, these examples areAtty Dkt No. 2161-001.502Via Patent Center not meant to be limiting, but illustrative of several common examples, and any similar or equivalent devices or systems may be used that are known to those skilled in the art.

[0046] The term "database" as used herein, refers to any form of one or more (or combination of) relational databases, object-oriented databases, hierarchical databases, network databases, non-relational (e.g. NoSQL) databases, document store databases, inmemory databases, programs, tables, files, lists, or any form of programming structure or structures that function to store data as described herein.

[0047] As shown in Fig. 8, in this embodiment, a user 804 may engage with the cloud computing web service 802 via multiple options, such as a personal electronic device 806 (e.g., a smartphone, desktop, laptop, tablet, etc.) and / or a central computer 808, operating, for example, a web site that is configured for accessing the cloud computing web service 802. The personal electronic device 806 and / or the central computer 808, may access static web assets, API calls, and / or any other methods known in the art.

[0048] In one embodiment, cloud computing w eb service 802 includes a front-end web application 810, such as an online dashboard, which includes a content delivery network 812 and a cloud storage service 814. As shown in FIG. 8, the content delivery network 812 directs data flow- into the cloud storage service 814. In one example, the content delivery network 812 is Amazon Cloudfront®, and the cloud storage service 814 is Amazon S3®.

[0049] As illustrated, back-end APIs 816 may include an application load balancer 818 of an elastic container service 820 (e.g., Amazon ECS®, or a similar or equivalent software solution).

[0050] As shown in FIG. 8, a data fde upload 821 is directed to data fde processing 822 that includes a cloud storage service 824 that transfers the data file to an event-driven serverless function as a service 826 (e.g., AWS Lambda®), which transfers to a serverless database 828 (e.g., Amazon DynamoDB®). The data file upload can be data recorded by the measuring device 100 during a sports activity participated by the user, such as during a surf outing or a ski outing.

[0051] A video upload 830, such as a video clip of the sports activity, is similarly received in a video processing component 832 that includes a cloud storage 834 for receiving the video, a serverless function as a service 836 for providing metadata to the data file processing which directs metadata back to the serverless database of the data file processing block, and stores theAtty Dkt No. 2161-001.502Via Patent Center final video in the cloud storage service 838 of the video processing block. That video may be provided for public video hosting 840.

[0052] In an embodiment, external data sources 850 (e.g., weather, tide, etc.) may be retrieved periodically via a data enrichment component 852. The data enrichment component 852 includes a serverless function as a service that transmits to a serverless database, and also through a raw data store into a cloud storage service. Information from these sources can be synced with the data file and video file to help determine whether external conditions, such as weather and tides, affects the user’s performance.

[0053] As illustrated, the cloud computing web service may further include a cloud security posture management (CSPM) service 860 (in this embodiment. AWS Security Hub®), a web services monitor 862 (Amazon Cloudwatch®), and an activity tracker 864 (AWS Cloudtrail®).

[0054] FIG. 9 is a flow diagram of a data processing and overlay generation flow 900, illustrating how data from a sport session or sports activity (e.g., surfing session, or any other sports, or other activities that a user might want to video and analyze) is downloaded in step 902, and subsequently processed to determine and define sport runs, and to determine various measurements of performance data during each sport run. For exemplary purposes, much of the discussions are particular to surf sessions (i.e., a surf outing with many individual surfing actions or surf sessions) and waves (i.e., a surf outing that takes place over multiple sport runs or waves with each wave clip having the surfer riding a wave and the measuring device measuring performance data of that particular ride), but this may also be applied to any other sports and data files gathered during those other sport sessions.

[0055] As shown in FIG. 9, the data from the measuring device 100 of FIG. 1 is first subjected to a Whittaker Shannon Interpolation 904 to construct a continuous-time bandwidth function from a sequence of real numbers (e.g., from the data acquired by the measuring device). Next, the downloaded data is sent through a process waves step 906 in which the function is processed. A select wave step 908 is where individual and distinct waves (runs) are identified.

[0056] Once the particular waves are determined, the raw sensor readings are processed in step 910, so that all of the sensor data 912 may be determined. The particular sensor data may include, for example, forward acceleration 914, force magnitude 916, orientation 918, directional force 920, flow' 922, speed 924, and vertical drop 926. Obviously, alternative or additional measurements may be determined, and the given examples should not be consideredAtty Dkt No. 2161-001.502Via Patent Center limiting. For example, the measurements may be completely different for other sports and activities, and the specifics may be determined by one skilled in the art.

[0057] Once these measurements are determined, a preprocess overlay data and downsample to 3hz step is executed in step 928. Once the data is in a more usable form, at a next step 930, the data processing module interpolates between datapoints to smooth the animation. At a final step 932, overlays are generated from data with a rendering algorithm and a video editing library (e.g., MoviePy®).

[0058] FIG. 10 is a flow diagram 1000 illustrating data intake, processing, and storage of one or more sport sessions, including steps that move from raw data storage 1002, via a bus 1004 (e.g., a pub-sub event bus), to data processing 1006 and a processed data storage 1008. First, datafiles are received, such as via an event planning software (e.g., Iconl®) device 1010, or via a mobile app. 1012, or any other method known in the art. The data is uploaded and stored 1014, such as in a cloud storage service bucket.

[0059] This triggers an event being sent via the bus 1004, a new data file 1016 for sport sessions is sent for data processing 1006. The data file for the new sport sessions is processed 1018 and saved 1020, wherein the sport sessions are saved in a serverless database of a processed data storage module. An algorithm function 1022 is also triggered, to identify sport runs (e.g., waves), and the individual runs are then saved 1024 and stored in step 1026 so that the sport runs are saved to a cloud storage service bucket of the raw data storage module.

[0060] Each of the sport runs 1028 is then processed in data processing to save the new sport runs 1030, calculate speed 1032, calculate vertical drop 1034, and calculate flow 1036 (and / or calculating any other performance qualities desired by one skilled in the art). These are all saved in a suitable database 1038 (e.g., Amazon DynamoDB®).

[0061] FIG. 11 is a flow diagram illustrating the function of a wave detection algorithm 1 100. As noted above, the wave detection algorithm 1100 functions to determine which sections of data are of a wave (or other sporting event), and which are not (and which generally may be discarded). The system detects waves and selects portions of the data that correspond to the wave, so that this data may be overlaid on the corresponding video of the wave, as noted above.

[0062] As shown in FIG. 11, in this embodiment, a dataframe is parsed in step 1102. The system is then able to begin iterating through the parsed data, in step 1104, so that the algorithm is able to search for and identify flagged data points 1106 that indicate the start of each sportingAtty Dkt No. 2161-001.502Via Patent Center run. In this case, in the surfing data, the algorithm searches for instance(s) of GPS speed being greater than 4mph. as an example. While this is one example of flagged data to search for and one example of a velocity threshold, those skilled in the art may select other forms of data that indicate the start of a sport run (e.g., a wave), and such alternative flags should be considered within the scope of the present invention.

[0063] Once an instance of the flagged data point has been found, a next step 1108 is to save a start point, which may be adjusted based upon the specific needs of the system. In an example, the start point is indexed a time period (e.g., 2.5 seconds earlier, or the beginning of data file), as the start point of the wave. It is desirable to capture details at least a couple of seconds before the acceleration of actually catching the wave, as this data may be helpful, and more likely to match the video, which would also preferably start before the actual drop in on the wave.

[0064] Then the algorithm moves forward through the data, in step 1110, until another flagged data point is reached that indicates the end of the wave. In this case, the flagged data point is set or considered when speed drops to less than Imph, indicating the wave has ended, or until the end of the file. If a speed less than Imph is found before the end of the file, the algorithm marks the actual end of the wave data at a later time (e.g., 2.5 seconds later) as the end of wave. The first and last index of the wave are then recorded as a tuple, in step 1114, and this process may repeat, again searching for the start of another wave. This continues until the end of the file is reached, in step 1116.

[0065] At this point, the algorithm executes a step 1 118 of iterating through the array of wave start and end tuples, to create a truncated copy of original dataframe in step 1120. The index is then reset in step 1122, and the wave(s) are saved.

[0066] Thus, an aspect of the invention is a method of simultaneously acquiring a video clip or recording of a sport run and performance data using a measuring device in accordance with aspects of the invention of that same sport run, said performance data comprising at least two or more of forward acceleration data, force magnitude data, orientation data, directional force data, flow data, speed data, and vertical drop data. The method can further comprise processing the video clip and the performance data to generate a paired video clip having both the video clip of the sport run and the performance data of the same sport run.

[0067] The video clip can contain multiple sport runs and the performance data can include data for the multiple sport runs.Atty Dkt No. 2161-001.502Via Patent Center

[0068] The measuring device can comprise a water-tight housing having an interior containing one or more sensors, a battery, a communications module, and a memory.

[0069] The video clip can be acquired from a televised sporting event, a feed from a beach cam, a feed from a hotel resort video cam, or a feed from a video cam located on a ski slope, to name a few non-limiting examples.

[0070] Both the video clip and the performance data can be timestamped.

[0071] The measuring device can be mounted directly to a sport equipment and spaced from the user. This mounting location can isolate measured data for a movement or motion for that of the sport equipment and not of the user of the sport equipment.

[0072] In instances where performance data is of the user, such as a to monitor the ready position, the starting position, the acceleration, and the body position of a track sprinter or a swimmer, as examples, then the measuring device can be mounted directly on the user. For example, cameras used to air track and field events at stadiums can be used to synchronize with the performance data gather from the measuring device.

[0073] In an example, the video clip is processed using a computer system implemented on a cloud computing web service.

[0074] The method can further comprise the step of accessing the computer system with a personal electronic device and a central computer. The personal electronic device can comprise a smartphone, a desktop, a laptop, a tablet, or equivalents thereof.

[0075] The cloud computing web service can comprise a front-end web application that includes a content delivery network and a cloud storage service. The front-end web application can comprise an online dashboard.

[0076] The method can further comprise a step of uploading a data file, such as captured performance data of sport runs, and directing the data file to a data file processing unit that includes a cloud storage service that transfers the data file to an event-driven serverless function as a service, which can then transfer to a serverless database.

[0077] The method can further comprise uploading a video clip containing images of one or more sport runs in a video processing component that includes a cloud storage for receiving the video clip. The video clip of one or more sport runs can include a video of a user using a sport equipment to participate in the one or more sport runs, and wherein the measuring device in accordance with aspects of the invention is mounted on the sport equipment used by the user in the video clip.Atty Dkt No. 2161-001.502Via Patent Center

[0078] The method further comprises the step of providing metadata to the data file processing which directs metadata back to the serverless database of the data file processing block.

[0079] The method further comprises the step of storing the final video in the cloud storage service of the video processing block.

[0080] The method further comprises gathering data from external data sources via a data enrichment component.

[0081] The external data sources comprise at least one weather data, tide data, and temperature data.

[0082] The data enrichment component can comprise a serverless function as a service that transmits to a serverless database. The method can further optionally comprise storing raw data in a cloud storage service.

[0083] The method can further comprise data processing and overlay generation so that data from a sport session or sporting event is downloaded and subsequently processed to determine and define sport runs, and to determine various measurements of performance data during each sport run.

[0084] The method can further comprise a step of subjecting data from the measuring device to a Whittaker Shannon Interpolation to construct a continuous-time bandwidth function from a sequence of real numbers.

[0085] The method can further comprise processing the downloaded data to identify individual or distinct waves (runs) so that performance data of each distinct wave can be analyzed and synced with a video clip of that same wave.

[0086] The method can further comprise processing sensor data acquired by the measuring device, which comprises one or more sensors mounted therein. The sensor data comprises at least one of forward acceleration data, force magnitude data, orientation data, directional force data, flow data, speed data, and vertical drop data.

[0087] The method can further comprise processing the sensor data to generate overlaydata and downsample to 3hz steps to produce usable data form.

[0088] The method can further comprise the step of interpolating between datapoints to smooth the animation of the video clip and / or of the sensor data.

[0089] The method can further comprise the step of generating overlays from the data with a rendering algorithm and a video editing library.Atty Dkt No. 2161-001.502Via Patent Center

[0090] The method can further comprise applying an algorithm function to identify individual or discrete sport runs and then saving the individual sport runs so that the individual sport runs are saved to a cloud storage, in a raw data storage module.

[0091] The method further comprises processing each sport run, calculating speed, calculating vertical drop, and calculating flow and then saving each processed sport run in a database.

[0092] The method further comprises processing additional performance parameters. Wherein the additional performance parameters comprise at least one of weather data, tide data, and temperature data.

[0093] The method can further comprise parsing a dataframe and using an algorithm to search for and identify flagged data points that indicate a start point of each sport run.

[0094] The method wherein the algorithm searches for one or more instances of GPS speed being greater than 4mph and labeling each such flagged data point as a start point of a sport run.

[0095] The method can further comprise saving the start points in memory. Optionally adjusting each start point based upon the specific needs of the system.

[0096] The method can further comprise adjusting one or more start points by indexing the one or more start points with an earlier fixed time period. In an example, each earlier fixed time period is at least 2 seconds or earlier, such as 2.5 seconds or earlier. In another example, at least one of the earlier fixed time period is the beginning of the data file, which is indicative of the start point of the wave.

[0097] The method can further comprise the step of using an algorithm to locate one or more end points of one or more sport runs captured by the performance data.

[0098] The method can further comprise using the algorithm to flag each end point in the data file to define a plurality of flagged end data points. Wherein each flagged data end point is one or more instances of GPS speed being less than Imph, indicating the wave has ended, or until the end of the file.

[0099] The method can further comprise using the algorithm to flag an end point as the actual end of the wave data plus a fixed time period later as the end of wave. Wherein the fixed time period later is 2.5 seconds later as the end of wave.Atty Dkt No. 2161-001.502Via Patent Center

[0100] The method can further comprise the step of recording the first and the last index of the wave as a tuple and then repeating this process to search for the start and end points of another wave.

[0101] The method can further comprise the step of applying the algorithm to iterate through the array of wave start and end tuples to create a truncated copy of original dataframe.

[0102] Although limited embodiments of the measuring device, measuring system, and traction pad and their components have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. Accordingly, it is to be understood that the device and system and their components constructed according to principles of the disclosed device, system, and method may be embodied other than as specifically described herein. The disclosure is also defined in the following claims.

Claims

Atty Dkt No. 2161-001.502Via Patent CenterCLAIMSWhat is claimed is:1 . A system for capturing performance data relating to a user using a sport equipment in a sports activity, and for pairing the performance data with a video of the user engaged in the sports activity7, the system comprising: a measuring device having a housing having a mounting component for mounting the housing of the measuring device to the sport equipment, the housing containing one or more sensors, a processor circuitry, a computer memory, a power source, and a transceiver for transmitting the performance data collected by the one or more sensors; one or more computer devices having a computer processor and computer memory, the computer memory storing executable code that, when executed by the computer processor, enables the computer system to perform a process that comprises: receiving performance data from the measuring device, the performance data being timestamped; receiving a video clip showing the sports activity, the video clip being timestamped; processing the performance data to determine one or more measured usable parameters; displaying the one or more measured usable parameters on or adjacent the video clip; using the timestamped information on the performance data and the video clip to sync the performance data and the video clip by7time; and pairing the performance data and the video clip to generate a paired video clip having both the one or more measured usable parameters and the video clip.

2. The system of claim 1, wherein the mounting component is in the form of a traction pad.

3. The system of claim 1, wherein the one or more sensors includes an accelerometer.

4. The system of claim 1, wherein the video clip is taken from one or more beach cams.

5. The system of claim 4, wherein the video clip is taken from a subscription service.

6. The system of claim 1, wherein the video clip is taken from one or more cams mounted at a surf pool.

7. The system of claim 2, wherein the traction pad is secured to a surface of a surfboard.

8. The system of claim 7, wherein the measuring device is mechanically engaged to the traction pad.Atty Dkt No. 2161-001.502Via Patent Center9. The system of claim 1, wherein the video clip comprises multiple sport runs and the performance data include data for the multiple sport runs.

10. The system of claim 1, wherein the measuring device comprises a water-tight housing having an interior containing one or more sensors, a battery, a communications module, and a memory.

11. A method for generating a paired video performed by one or more computers, the method comprising the steps of: receiving measured motion data from a measuring device having a water-tight housing having one or more sensors enclosed therein, the measured motion data being timestamped and the measuring device being mounted to a sport equipment; receiving a video clip showing the sport equipment in use while having the measuring device mounted thereto, the video clip being timestamped; processing the measured motion data to display one or more measured usable parameters in a side-by-side view with the video clip; using the timestamped information on the measured motion data and the video clip to sync the measured motion data and the video clip by time; and pairing the motion data and the video clip to generate a paired video clip having both the one or more measured usable parameters and the video of the sport equipment in use.

12. The method of claim 11, further comprising a step of accessing a computer system with a personal electronic device and a central computer.

13. The method of claim 1 1 , wherein the personal electronic device comprises a smartphone, a desktop, a laptop, a tablet, or equivalents thereof.

14. The method of claim 11, further comprising a step of uploading a data fde comprising the motion data and directing the data fde to a data fde processing unit that includes a cloud storage service that transfers the data fde to an event-driven serverless function, which then transfers the data fde to a serverless database.

15. The method of claim 14, further comprising the step of providing metadata to the data fde, which directs metadata back to the serverless database of a data fde processing block.

Images