Providing motion dynamics estimation

Abstract

The invention relates to providing motion dynamics estimation. Aspects of the subject technology obtain sensor data from microphones and / or motion sensors in devices disposed on each side of a user's body. The sensor data is used to provide running dynamics estimates including estimates relating to differences between the motion characteristics of the right side of the user's body and the left side of the user's body. These estimates may also be used to provide feedback to the user regarding running style, running route, and injury information.

Description

TECHNICAL FIELD

[0001] This specification generally relates to electronic devices, including, for example, analyzing motion characteristics using sensors in a portable electronic device. BACKGROUND

[0002] A headset is an audio device that includes a pair of speakers, each of which is placed proximate to an ear of a user. The pair of speakers of a headset are typically connected (e.g., wired or wireless connection) to a separate playback device (such as a media playback device) that drives each of the speakers with an audio signal in order to produce sound (e.g., music). Some headsets can include, for example, an audio input sensor (e.g., a microphone) so that speech of a user can be captured and relayed back to the playback device. A headset can include other sensors. BRIEF DESCRIPTION OF DRAWINGS

[0003] Particular features of the subject technology are set forth in the appended claims. However, for purpose of explanation, several implementations of the subject technology are listed in the following drawings.

[0004] Figure 1 FIGS. illustrate example electronic devices and systems in which aspects of the subject technology can be implemented, in accordance with one or more implementations.

[0005] Figure 2 FIG. illustrates a block diagram of an audio signal processing architecture for an electronic device, in accordance with one or more implementations.

[0006] Figure 3 FIG. illustrates a data flow diagram of a block that performs motion dynamic estimation, in accordance with one or more implementations.

[0007] Figure 4 FIG. illustrates a data flow diagram of a block that performs motion dynamic estimation, in accordance with one or more implementations.

[0008] Figure 5 FIG. illustrates a data flow diagram of a block that performs sensor weighting and / or encoding based on sensing input, in accordance with one or more implementations.

[0009] Figure 6 FIG. illustrates a flow diagram of an example process for performing motion dynamic estimation, in accordance with one or more implementations.

[0010] Figure 7 FIG. illustrates an example user interface for providing information about various motion dynamics estimates over various time periods, in accordance with one or more implementations.

[0011] Figure 8Example user interfaces for providing information regarding various motion dynamics estimates within a fitness session are illustrated in accordance with one or more embodiments.

[0012] Figure 9 Example electronic systems that can be used to implement aspects of the subject technology in accordance with one or more implementations are illustrated. DETAILED DESCRIPTION

[0013] The detailed description set forth below is intended as a description of various configurations of the subject technology and is not intended to represent the only configurations in which the subject technology can be practiced. The appended drawings are incorporated herein and constitute a part of the detailed description. The detailed description includes specific details for the purpose of providing a thorough understanding of the subject technology. However, the subject technology is not limited to the specific details set forth herein and can be practiced using one or more other implementations. In one or more implementations, structures and components are shown in block diagram form in order to avoid obscuring the concepts of the subject technology.

[0014] Some accessory devices, such as headphones, include components designed to be disposed on opposite sides of a user’s body. In addition to speakers, headphones can also include microphones, inertial measurement units (IMUs) or other motion sensors, bone conduction devices, and the like. Other accessory devices, such as eyeglasses, head-mounted displays, headbands, wristbands, or specialized devices, can similarly have components disposed on opposite sides of a user’s body. The subject technology utilizes sensor input from sensors disposed on opposite sides of a user’s body, analyzes the sensor input, and provides information related to a user’s self-propelled motion, such as walking, jogging, or running. Because the information is provided from opposite sides of a user’s body, the results of such analysis can include information about stride length, gait, step frequency, ground contact time, vertical oscillation, and the like. Because sensors are utilized on opposite sides of a user’s body, these motion characteristics can be predicted and further analyzed for left-right imbalances. For example, such imbalances can be due to injury or poor running / walking style / habits. Regardless of the cause of the imbalance, because they are measurable, these issues can be addressed for correction.

[0015] For simplicity, the measurements collected from the dual-zone sensors described herein are generally referred to as running dynamics estimates. It should be understood that these measurements (or similar measurements taken from opposite sides of the body for other activities) can be useful in contexts other than running, such as jogging, walking, cross-country skiing, sprinting, rollerblading, ice skating, swimming, cycling, and the like. Thus, references to running and / or running dynamics herein should be understood as not being limited to being related to running activities, but rather to any suitable motion characteristics. Thus, the term running dynamics estimates as used herein can be understood to include any motion dynamics estimates, unless otherwise indicated.

[0016] Performance can be measured using running dynamics. For participants, running dynamics can be particularly valuable for helping track performance, improve performance, and prepare for specific events and / or races. These metrics can also be useful for recreational exercisers trying to stay active, as they can point to issues with gait, issues with one foot versus the other, issues with walking stability, and so on. For example, for older people, tracking such metrics can provide insight into measuring a decline in stability, or help pick the right shoes for better support. For experienced performers, these metrics can tell where to improve, and for occasional performers, these metrics can tell where to improve to prevent injury. Training for long runs can leverage such metrics to an advantage, as subtle imbalances in these metrics can accumulate over time if not addressed, resulting in larger biomechanical issues.

[0017] By way of explanation and not limitation, some of the running dynamics contemplated herein include step frequency (the total number of steps taken by both feet per minute), ground contact time (the amount of time both feet spend on the ground per step), ground contact time balance (the symmetry or asymmetry of ground contact time between the right foot and the left foot), stride length (the length of a half-step), stride length imbalance (the symmetry or asymmetry of stride length for steps of the right foot versus the left foot), vertical oscillation (the distance a foot is lifted per step), and vertical oscillation imbalance (the symmetry or asymmetry of vertical oscillation between the right foot and the left foot).

[0018] While some devices (e.g., smartwatches) can predict or estimate some of these metrics, using sensor input from both sides of the body can provide improved quality, alternative estimates that can be used when sensor data on other devices is noisy, and additional metrics that a single-sided sensor cannot provide. For example, sensor signals used in the subject devices can be used to provide running dynamics estimates that can be combined with similar estimates from other devices, such as smartwatches, to provide better overall estimates. Further, for example, sensor signals on the subject devices can include microphone sensor input that is located in or near the ear canal and focused into the ear when worn by a user. These sensor signals can be less susceptible to noise or drift issues than other signals, such as accelerometer signals used on smartwatches. Still further, for example, sensor signals of the subject devices are disposed on both sides of the body, which provides different data points for each side that can be used to predict running dynamics that a sensor device worn on one side of the body, such as a smartwatch or phone, cannot predict because the missing signal cannot be fully or reliably approximated by a sensor device worn on one side of the body.

[0019] Aspects of the subject technology utilize a headset device (e.g., earbuds) that includes sensor devices disposed therein as a platform for obtaining signals used to estimate running dynamics. In some implementations, the earbuds can include accelerometer sensors, e.g., an inertial measurement unit (IMU) in each earbud, such that when worn by a user, the accelerometer sensors are disposed at each ear. In some implementations, the earbuds can include microphone sensors, such as an in-ear microphone sensor disposed in each earbud, such that when worn by a user, the microphone sensors are disposed at each ear, either in the ear canal or proximate to the ear canal, in both cases likely within the air space or acoustic cavity of the ear canal. In some implementations, the earbuds can include both accelerometer sensors and microphone sensors.

[0020] Aspects of the subject technology compute running dynamics based on sensor input and machine learning models trained to predict various running dynamics based on such sensor data. As part of computing running dynamics, a footstep sound detection process can be used, and then the footstep sounds can be separated into signals of different footstep sounds. These signals can be provided to a running dynamics estimation engine, which subjects the signals to various processes, such as digital signal processing algorithms, and / or machine learning models, such as classification and / or prediction models. These machine learning models can provide various estimates of running dynamics over time as output. In some aspects, data received as sensor signals can be processed entirely on the earbuds, in other aspects, data received as sensor signals can be partially processed on the earbuds and partially processed on one or more devices connected to the earbuds, and in yet other aspects, data received as sensor signals can be received on the earbuds and transmitted to one or more devices connected to the earbuds for processing.

[0021] Aspects of the subject technology can also utilize the determined running dynamics to provide feedback to the user. The feedback can include, for example, user interfaces that depict plots and values of the running dynamics over time. The feedback can indicate predictions for characteristics of the user’s running style. For example, in addition to the running dynamics described above, the feedback can include providing the user with a prediction of whether the user is a heel striker, midfoot striker, or forefoot striker; a prediction of the user’s pronation (under-pronation, over-pronation, or normal pronation); or a prediction of an injury associated with the user (e.g., by comparing prior information with updated information). The feedback can also include feedback about the running surfaces and routes taken by the user. For example, the feedback can include an estimate of the types of surfaces encountered while running, such as an indication that a running route includes 30% of the run on dirt or grass and 70% of the run on pavement. Depending on the training program selected, the user can be warned that they are running too much on pavement or too much on dirt or grass (or vice versa). In some aspects, the user can provide a desired distribution of grass / dirt and pavement running, a desired route length, and a starting and ending point as input, and a route can be created and suggested by the device. In some aspects, the user can input historical running events, such as their past running cycles, and compare current performance to past performance and receive feedback on performance aspects. In some aspects, the user can select from provided historical running events, such as past marathon races, and the user can run along these routes.

[0022] Aspects of the subject technology can provide real-time or near real-time feedback for some events, such as providing a real-time warning when a running posture is detected to be declining. Other feedback can be determined and provided later.

[0023] The subject technology advantageously analyzes and provides information to the user about running dynamics and other metrics derived from running sessions. The subject technology can also provide running resources to the user to enhance running sessions, such as mapping, running surface information, personal historical comparisons, and historical running events. The subject technology can also provide running style recommendations and running style-based product recommendations, such as recommendations for footwear that support the user’s pronation and foot strike characteristics.

[0024] Figure 1 Example electronic devices and systems are illustrated in accordance with one or more implementations. However, not all of the depicted components can be used in all implementations, and one or more implementations can include additional or different components than those shown in the figures. Variations in the arrangement and type of the components can be made without departing from the spirit or scope of the claims as set forth herein. Additional components, different components, or fewer components can be provided.

[0025] In Figure 1In the example of FIG. 1, the electronic device 100 has been implemented using a housing 106 that is small enough to be portable and carried or worn by a user. In particular, Figure 1 An example is shown in which the electronic device 100 is implemented as an earbud or a pair of earbuds. In this example, the housing 106 is shaped to seat in a user’s outer ear and / or ear canal opening and to interface with the user’s ear canal. An optional sealing member 120 can be engaged with the housing 106 to provide a sealing effect between the electronic device 100 and the user’s ear canal. The sealing member 120 can be made of any suitable material, such as silicone, plastic, rubber, foam, polymer, a composite thereof, or a combination thereof.

[0026] In one or more other implementations, Figure 1 The earbud of FIG. 1 can be used in conjunction with another electronic device 130 or 132, such as a smartphone or tablet computer or smartwatch, to which the microphone signals received by the microphones 114 and / or 116 and / or the sensors 118 and / or the sensors 121 are transmitted. The other electronic device 130 or 132 can also provide the audio output signals for the speaker 112 to the electronic device 100. In one or more implementations, Figure 1 The earbud of FIG. 1 can include processing circuitry that performs one or more of the operations described herein.

[0027] In Figure 1 In the example of FIG. 1, the housing 106 includes an opening 108. For example, the opening 108 can form one or more respective ports for one or more respective audio components. In Figure 1 In the example of FIG. 1, the electronic device 100 includes an opening 108 that forms a speaker port for the speaker 112, which is disposed within the housing 106. Other openings can include vent or port openings for the microphones 114 and / or 116. The opening for the microphone 114 can be disposed on an outer surface of the housing 106, and the opening for the microphone 116 can be disposed on an inner surface of the housing 106 such that it is disposed within the opening 108 for the speaker 112. The microphone 116 can be positioned such that it is placed at the interior or tip of the user’s ear canal, or in an acoustic chamber or acoustic sealed chamber that includes the user’s ear canal. Although two microphones are depicted, it will be appreciated that the electronic device 100 can include additional microphones. For example, in some implementations, the headset can also provide active noise cancellation (ANC) that can also utilize a reference microphone disposed on an outward-facing surface of the electronic device 100 to measure ambient noise to provide an anti-noise signal or a direct signal in accordance with the user’s settings when the headset is worn by the user.

[0028] In various implementations, the housing can also include fewer or additional openings. The additional openings can include openings for one or more additional audio input devices (e.g., microphones), one or more pressure sensors, one or more light sources, one or more light sensors, or other components that receive signals from or provide signals to the environment external to the housing 106. An opening such as the opening 108 can be an open port, or can be completely or partially covered with a permeable membrane or mesh structure that allows air and / or sound to pass through the opening.

[0029] The housing 106, which can sometimes be referred to as a case, can be formed from plastic, glass, ceramic, fiber composite, metal (e.g., stainless steel, aluminum, etc.), other suitable materials, or a combination of any two or more of these materials. The electronic device 100 also includes additional components such as processing circuitry (e.g., one or more processors), memory, a power source such as a battery, communication circuitry, etc.

[0030] In one or more use cases, one or more of the speakers 112 can generate speaker output based on, for example, downlink communication signals or device-generated or streamed audio signals. In one or more implementations, the speakers 112 can be driven by output downlink signals that include far-end acoustic signal components from a remote device such as the electronic devices 130 or 132.

[0031] Upon receiving microphone signals and / or other signals from a wired or wireless data communication link 135 with the electronic device 100 and / or each other, aspects of the subject technology described herein can be executed by one or more processors of the electronic device 100 and / or can be executed by processors internal to the electronic devices 130 and / or 132. Figure 1 The electronic device 100 of the example of FIG. 1 can include communication circuitry for communicating with one or more other electronic devices 130 and / or 132 via a wired or wireless data communication link 135. Figure 1

[0032] In use, the microphones 116, the sensors 118, and / or the sensors 121 can receive sensor inputs. The microphones 116 can receive inputs that propagate through the body from the user’s feet to the ears, for example. The sensors 118 can receive inputs related to motion characteristics of the electronic device 100. The sensors 121 can receive inputs related to motion characteristics associated with the user speaking, such as vocal cord motion and / or jaw motion. The earbuds of the electronic device 100 of Figure 1 The earbuds of the electronic device 100 of FIG. 1 can include a left earbud L and a right earbud R for the user’s left and right ears, respectively.

[0033] In the example of FIG. 1, the electronic device 100 can include a housing 106, a left earbud L, a right earbud R, a left microphone 116, a right microphone 116, a left sensor 118, a right sensor 118, a left sensor 121, a right sensor 121, a left speaker 112, a right speaker 112, and a communication link 135. Figure 1 ​In the example of FIG. 1, the electronic device 100 includes a speaker 112, a bottom microphone (e.g., microphone 114) that is located in or near an end portion of the housing 106 of the earbud at which the bottom microphone is the microphone closest to the user’s mouth, and an error microphone 116 that senses sound at or near the user’s eardrum (e.g., in the user’s ear canal). In Figure 1 In the example of FIG. 1, the microphone 116 can be in a position and orientation to receive output from the speaker 112 and / or one or more environmental sounds (e.g., environmental noise, speech of a person other than the user of the electronic device 100, speech of the user of the electronic device 100, etc.) that leak through the housing 106 and / or the sealing member 120 to the microphone 116. Although not shown, a top microphone can be provided whose sound sensitive surface faces in an opposite direction from the user’s eardrum when the earbud is worn.

[0034] In one or more implementations, when the electronic device 100 is implemented as an earbud as in Figure 1 FIG. 1, the electronic device 100 can include a battery device, one or more processors, and / or a communication interface (see FIG. 1). Figure 3 The processor of the earbud can be a digital signal processing chip that processes audio signals (e.g., microphone signals) from at least one of the microphones 114 and / or 116 and signals (e.g., IMU signals) from the sensor 118 and signals (e.g., voice accelerometer signals) from the sensor 121. The communication interface can include a Bluetooth ™ receiver and transmitter to communicate signals from the microphones 114 and 116 and / or the sensors 118 and / or 121 with external electronic devices 130 and / or 132 in both directions (uplink and downlink). In addition, the earbud can be a wired or cordless wireless earbud that communicates with each other and with the external electronic devices 130 and / or 132 via Bluetooth ™ signals.

[0035] In some aspects, the electronic device 100 can be communicatively coupled to the electronic devices 130 and / or 132 via a wireless connection, such as shown above. In other aspects, the electronic device 100 can be communicatively coupled with the electronic devices 130 and / or 132 via other methods. For example, both devices can be coupled via a wired connection. In such a case, one end of the wired connection can be (e.g., fixedly) connected to the electronic device 100, while the other end can have a connector that plugs into a jack of the electronic devices 130 and / or 132, such as a media jack or a Universal Serial Bus (USB) connector. Once connected, the electronic devices 130 and / or 132 can be configured to drive one or more speakers of the electronic device 100 with one or more audio signals. For example, the electronic devices 130 and / or 132 can transmit the audio signals as digital audio (e.g., PCM digital audio). In another aspect, the audio can be transmitted in analog format.

[0036] The electronic devices 130 and / or 132 can be any electronic device (e.g., having electronic components such as one or more processors, memory, etc.) capable of streaming audio content in any format, such as a stereo audio signal, for playback (e.g., via one or more speakers integrated within the source device and / or via one or more output devices, as described herein). For example, the source device can be a desktop computer, a laptop computer, a digital media player, etc. In one aspect, the device can be a portable electronic device (e.g., operable by hand), such as a tablet computer, a smartphone, etc. In another aspect, the source device can be a wearable device (e.g., a device designed to be worn on (e.g., attached to) a user's clothing and / or body, such as a smartwatch). In some aspects, the electronic device 130 can include an interface and programming contained in memory that, when executed by one or more processors, performs a hearing test with the electronic device 100.

[0037] For simplicity, the discussion herein describes the electronic device 100 as a pair of earbuds in use cases. However, it should be understood that the electronic device can be any suitable device that is provided with sensors on opposite sides of the body, such as any suitable wearable device that is worn or attached on opposite sides of a moving body. This can include two separate devices that are each worn with sensors on each side, which can communicate with each other and / or with a third device (such as a smartphone or smartwatch) for storing and / or processing data using the processes and techniques discussed herein, or a single device that is provided with sensors on each side of the body when worn, which can communicate with another device (such as a smartphone or smartwatch) for storing and / or processing data. For example, in some embodiments, the electronic device 100 can be implemented as an over-ear or on-ear headphone, eyewear, head-mounted display, headband, wristband, or a dedicated device, each of which is provided with sensor components on both sides of the user’s body. Further, other sensors can be suitable for detecting one or more similar signals for detecting and processing running dynamics.

[0038] Figure 2 A block diagram of the electronic device 130 and / or 132 and the electronic device 100 that are wirelessly coupled together according to one aspect is shown. The electronic device 130 and / or 132 includes a controller 140, a memory 141, a network interface 142, a speaker 143, a display 144, a microphone 147, a camera 148, and a volume control 146. In one aspect, the source device can include more or fewer elements, as described herein. For example, the device can include several speakers and / or microphones, or can not include a camera or display.

[0039] According to one aspect, the electronic device 100 includes a controller 124, a network interface 125, an audio output device (such as a speaker 112), microphones 114 and 116, a sensor 118 (e.g., an IMU configured to detect motion of the electronic device 100), a sensor 121 (e.g., an accelerometer configured to detect voice input from a user of the electronic device 100), one or more (other) sensors 126, a volume control 119, and a memory 127. In one aspect, the device can include more or fewer elements. For example, the electronic device 100 can include one or more additional speakers, and / or can include one or more additional microphones.

[0040] Microphone 114 can be an input microphone (such as a voice microphone), and microphone 116 can be an error microphone, whose primary function is for ANC, but can also be used for the processes described herein. A reference microphone (not shown) for ANC can also be included in electronic device 100. In the case of an earbud or headphone device, microphone 116 can sense sounds inside the user's ear when electronic device 100 is positioned on (or in) the user's ear.

[0041] Sensor 118 (e.g., an IMU) can be designed to produce motion data indicative of (changes in) a position and / or orientation of the output device. In another example, the sensor can include an accelerometer that can be arranged and configured to receive (detect or sense) vibrations produced when a user (e.g., a user that can be wearing the output device) is in motion, and produce an accelerometer signal representative of (or containing) the motion vibrations. In particular, the accelerometer is configured to sense bone conduction vibrations transmitted through the body when running or walking. For example, when the electronic device is a wireless headphone, the accelerometer can be located on or within the headphone anywhere that can contact a portion of the user's body in order to sense the vibrations. Sensor 121 (e.g., an accelerometer) can be designed to produce motion data indicative of a user of electronic device 100 speaking. In particular, this motion data can be tuned to detect movement of the user's jaw and / or vibrations of the user's vocal cords. Sensor 126 can include one or more other sensors designed to produce sensor data. For example, the sensor can include a proximity sensor or a light sensor for determining whether a user is wearing electronic device 100.

[0042] The controller 124 can be a special purpose processor such as an application specific integrated circuit (ASIC), a general purpose microprocessor, a field programmable gate array (FPGA), a digital signal controller, or a set of hardware logic structures (e.g., filters, arithmetic logic units, and dedicated state machines). In one aspect, the controller 124 can be configured to perform digital signal processing associated with a run dynamics estimation (RDE). Microphone signals and accelerometer output signals, and the controller 124 can obtain those signals and perform one or more steps that result in performing an RDE. For example, digital signal processing can be used to pre-process the signals to remove noise and isolate user footstep sounds picked up by each sensor. The sensors (such as the microphones 116, the sensor 118, and the sensor 121) can be affected by various sources of interference that can cause noise or errors in the signals. The sources of interference can include, for example, breathing (breathing sounds), playback sounds (e.g., from the speakers 112), fitment changes while running or exercising, processing noise, sensor drift issues, wind noise, heartbeat noise, external or environmental sounds, and so on. Footstep sound information can be extracted from the sensor inputs and isolated for analysis. Prior to and / or after performing isolation of the footstep sound data, audio processing filters (e.g., high pass, low pass, band pass, etc.) and / or other digital signal processing to clean up the signals can be applied. Further, as discussed in more detail below, a weighting scheme can be used to place more emphasis on input from one sensor than another sensor when certain sources of interference are more prevalent than others.

[0043] For example, the footstep sound data can include sensor input information of a user's left foot contacting the ground, which is sensed by the user's left side sensors, and the user's right side sensors sense at a lower intensity than the left side sensors, or vice versa. The isolated footstep sound data can be further processed to separate the footstep sounds into right foot footstep sounds and left foot footstep sounds. The isolated footstep sound data can be provided to the local RDE processing block 129 to perform some or all of the RDE processing. In some cases, some or all of the processing of the sensor signals can be transferred to another device (such as the electronic devices 130 and / or 132) for processing at an RDE processing block on one or more of such devices. This can be done, for example, at least in part, to conserve processing and power (battery) resources on the electronic device 100.

[0044] The controller 124 can also be configured to perform other functions, such as (other) audio signal processing operations and / or networking operations. For example, the controller can be configured to obtain (or receive) audio data (as analog or digital audio signals) including audio content, such as music for playback through the speakers 112. In some aspects, the controller can obtain the audio data from the memory 127, or the controller can obtain the audio data from another device (such as a source device) via the network interface 125. For example, the output device can stream the audio signals from the source device (e.g., via Bluetooth ™ connection) for playback through the speakers 112. The audio signals can be signal input audio channels (e.g., mono). In another aspect, the controller can obtain two or more input audio channels (e.g., stereo channels) for output through two or more speakers.

[0045] The volume control 119 can be configured to adjust the volume level of the sound output by the electronic device 100 in response to receiving a user adjustment (e.g., user input) at the control. For example, upon receiving the user input, the control 119 can adjust the (e.g., total) sound volume output by the electronic device 100 (e.g., the speakers 112 of the electronic device). In some aspects, the volume control 119 can be used to adjust the volume setting at the electronic device 130. In one aspect, the volume control can be a “physical” volume control, which can be a dedicated volume input control, such as one or more buttons, a rotatable knob, or a physical slider. For example, the volume control 146 can include at least two buttons: a “volume up” button, which can be configured to perform a step increase in volume each time it is pressed by a user, and a “volume down” button, which can be configured to perform a step decrease in volume each time it is pressed by a user. In some aspects, the volume control can be any type of physical input device capable of adjusting the total volume level.

[0046] In the electronic device 130 and / or 132, the controller 140 can be a special purpose processor, such as an application specific integrated circuit (ASIC), a general purpose microprocessor, a field programmable gate array (FPGA), a digital signal controller, or a set of hardware logic structures (e.g., filters, arithmetic logic units, and dedicated state machines). The controller 140 is configured to perform operations for performing a run dynamic estimation (RDE) and processes associated with RDE estimation, such as described in further detail below. The controller 140 can also be configured to perform other processing, such as audio signal processing operations and / or networking operations. The controller 140 can have disposed within it hardware, logic blocks, instructions, etc. that provide aspects of a run dynamic estimation (RDE), which are collectively referred to as RDE block 149.

[0047] The memory 141 can be any type of non-transitory machine-readable storage medium. Examples can include read-only memory, random access memory, CD-ROMS, DVDs, magnetic tape, optical data storage equipment, flash memory equipment, and phase change memory.

[0048] The camera 148 can be a complementary metal-oxide-semiconductor (CMOS) image sensor capable of capturing digital images that include image data representing a field of view of the camera, where the field of view includes a scene of an environment in which the electronic device 130 is located. In some aspects, the camera can be a charge-coupled device (CCD) camera type. The camera is configured to capture still digital images and / or video represented by a series of digital images. In one aspect, the camera can be positioned anywhere near or on the electronic device 130. In some aspects, the electronic device 130 can include multiple cameras (e.g., where each camera can have a different field of view).

[0049] The microphone 147 can be any type of microphone configured to convert acoustic energy caused by sound waves propagating in an acoustic environment into a microphone signal (e.g., a differential pressure gradient microelectromechanical system (MEMS) microphone). In some aspects, the microphone can be an “external” (or reference) microphone arranged to capture sound from the acoustic environment.

[0050] The speaker 143 may, for example, be an electrically driven speaker that can be specifically designed for a particular frequency band, such as a subwoofer, a tweeter, or a midrange driver. In one aspect, the speaker 143 can be a “full-range” (or “full-frequency”) electrically driven speaker that reproduces as much as possible of the audible frequency range.

[0051] The display 144 is designed to present (or display) video of digital image or video (or image) data. In one aspect, the display can use liquid crystal display (LCD) technology, light-emitting polymer display (LPD) technology, or light-emitting diode (LED) technology, although other display technologies can be used in other aspects. In some aspects, the display can be a touch-sensitive display screen configured to sense user input as a touch or tap on the screen and, in response, produce one or more control signals. In some aspects, the display can use any touch-sensing technology, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies.

[0052] The volume control 146 is configured to adjust a volume level of a sound output of the electronic device 130 and / or 132 in response to receiving a user adjustment (e.g., a user input) at the control. In one aspect, the volume control can be a “master” volume control configured to control a total volume level (e.g., a sound output level of the speaker 143) of the electronic device 130 and / or 132. In another aspect, the volume control can also control a total volume level of the electronic device 100 when the electronic device 130 and / or 132 is communicatively coupled with the electronic device 100 to stream audio content to the electronic device 100 for playback. In one aspect, the volume level output on the electronic device 100 for a hearing test process or application can override settings provided by a user through the volume control 146, for example. In one aspect, the volume control can be a “physical” volume control, which can be a dedicated volume input control such as one or more buttons, a rotatable knob, or a physical slider. For example, the volume control 146 can include at least two buttons: a “volume up” button, which can be configured to perform a step increase in volume each time it is pressed by a user, and a “volume down” button, which can be configured to perform a step decrease in volume each time it is pressed by a user. In some aspects, the volume control can be any type of physical input device capable of adjusting a total volume level.

[0053] Figure 3 A data flow diagram 300 between sensors of the electronic device 100 and the run dynamics estimation (RDE) block 129 and / or 149 is illustrated. While the following discussion of the data flow diagram 300 continues to describe the data flow from block to block, it should be understood that the flow can be executed in parallel as appropriate, or can not proceed in the order described, with the associated processes permitting the data flow to differ.

[0054] As described below, the RDE block 129 and / or 149 can include a run dynamics estimation function that utilizes digital signal processing techniques, such as filtering, Fourier transforms, and wave analysis, to analyze sensor data inputs to estimate run metrics associated with a user sensed by the sensors. The RDE block 129 and / or 149 can utilize digital signal processing techniques to provide an output based on the sensor inputs, for example, including a characterization of a user’s run activity. With respect to Figure 4 These outputs are described in further detail. These techniques can be understood to correspond to non-AI-based processes to obtain digital and / or analog sensor data from the microphones, IMUs, and accelerometers of each earbud to indicate usage and environmental conditions.

[0055] Sensor inputs from the sensors (including left error microphone 116L, right error microphone 116R, left IMU 118L, right IMU 118R, left voice accelerometer 121L, and right voice accelerometer 121R) can be fed to RDE block 129 and / or 149 for processing and providing a run dynamics estimate. RDE block 129 and / or 149 can include processes that implement one or more learning-based and / or non-learning-based models for perceiving, synthesizing, and inferring information. Those skilled in the art will appreciate that RDE block 129 and / or 149 can include any suitable number of RDE processes to generate output characterizations (e.g., step frequency, ground contact time, ground contact time balance, vertical oscillation, gait, etc.) based on the sensor inputs. In some embodiments, RDE block 129 and / or 149 can optionally select to use a lightweight engine instead of a multi-faceted process at RDE block 129 and / or 149 to process data in the case of processing only one data modality. Lightweight engine 108 may, for example, be a non-learning network.

[0056] Those of ordinary skill in the art will appreciate that the processes implemented in RDE block 129 and / or 149 can include any suitable machine learning model that is well-known or widely available, such as regression techniques, classification techniques, neural networks, and deep learning networks. For example, RDE block 129 and / or 149 can include neural networks such as artificial neural networks (ANN), convolutional neural networks (CNN), recurrent neural networks (RNN), adversarial networks (GAN), reinforcement learning models (RLM), encoder / decoder networks, and / or transformer-based models (e.g., Bidirectional Encoder Representations from Transformers (BERT), Generative Pretrained Transformers (GPT), and / or Multimodal Large Language Models (LLM)). Additionally or alternatively, those of ordinary skill in the art will appreciate that RDE block 129 and / or 149 can include any suitable non-learning processes, such as rule-based systems, heuristic methods, decision trees, knowledge-based systems, statistical or stochastic systems, and expert systems.

[0057] Where RDE block 129 and / or 149 implements machine learning-based models, the processes of RDE block 129 and / or 149 can be trained to disambiguate between footfalls and characterize run dynamics using one or more well-known or widely available training techniques, such as supervised learning, semi-supervised learning, unsupervised learning, and / or reinforcement learning techniques. Training data can include known course and / or surface information, known physical characteristics of training user data, and location information related to the footfall information. Additional training techniques and consideration data are described further below.

[0058] In the case where RDE boxes 129 and / or 149 are non-learning-based systems, RDE boxes 129 and / or 149 can use predefined rule sets or predefined structures to make decisions based on the input seen by the box. For example, RDE boxes 129 and / or 149 can process each signal in parallel as data is streamed via a signal source and compare interaction points between signals to determine, for example, patterns indicating stride frequency or footstep sounds, patterns indicating whether a runner is running on a hard surface or a natural or elastic surface, or patterns estimating vertical oscillation metrics associated with running. The RDE box can then use the various signals and their interactions to prepare a characterization.

[0059] Sensor inputs from electronic device 100 may include a left error microphone 116L, a right error microphone 116R, a left IMU 118L, a right IMU 118R, a left voice accelerometer 121L, and a right voice accelerometer 121R. In some embodiments, one or more IMUs may be omitted (or only accelerometer sensor types may be included), one or more error microphones may be omitted, and / or one or more sound accelerometers may be omitted. Sensor inputs may each be provided to running dynamics estimation (RDE) blocks 129 and / or 149. For example, sensor inputs may be provided to RDE block 129 of electronic device 100 and / or RDE block 149 of electronic devices 130 and / or 132. In some embodiments, some of these sensor inputs may be provided to RDE block 129, while others may be provided to RDE block 149. For example, the left error microphone 116L signal and the right error microphone 116R signal may be provided to the RDE box 129 of electronic device 100, and the left IMU 118L signal and the right IMU signal 118R signal may be provided to the RDE box 149 of electronic devices 130 and / or 132, and vice versa. In some embodiments, sensor inputs may be provided to both RDE boxes 129 and RDE boxes 149. Sensor inputs may be transmitted at different times. For example, sensor inputs may initially be transmitted to electronic device 100 in real time, and sensor inputs may be transmitted periodically (e.g., every 0.5 minutes to 15 minutes or every 15 minutes to 60 minutes) to electronic devices 130 and / or 132, or at specified times or under specified conditions, such as at the end of a running session, at a time of day, during device status (e.g., in a case and charging), or when the battery power of electronic device 100 meets a threshold.

[0060] like Figure 3 As shown, the sensor input is provided by electronic device 100. RDE 129 and / or 149 may be provided by electronic device 100, electronic device 130 or electronic device 132 (or a combination thereof).

[0061] The RDE block 129 and / or 149 can receive the signals and perform RDE on the signals. For example, the RDE can supply the signals to a machine learning model or perform digital signal processing (or a combination of both) to determine measurements indicative of various characteristics of the user’s gait.

[0062] In one aspect, user running attributes such as step frequency, ground contact time imbalance, ground contact time, and vertical oscillation can be determined. No other input from the user can be required to obtain the data, however, if the user desires, a target range of settings can be performed manually or automatically based on historical data, and if the user is outside of these ranges, the user can be notified, for example, through audio output of the electronic device 100 and / or through audio, vibration, or visual output on the electronic devices 130 and / or 132.

[0063] In another aspect, user gait attributes such as overstriding, heel striking, forefoot striking, or midfoot striking can be determined. For example, the RDE block 129 and / or 149 can use the user running attributes along with the signals (which can be denoised or weighted as discussed below) to predict whether the user is overstriding, to predict whether the user is heel striking or forefoot or midfoot striking. In this aspect, if the user provides input that they are heel striking and want to transition to forefoot / midfoot striking, the RDE block 129 and / or 149 can first be trained using the user’s natural gait (e.g., heel striking), and then when the user attempts to transition to a different kind of striking, the system can provide feedback to the user based on the current gait attribute predictions. For example, the system can provide feedback via audio output to the electronic device 100 and / or through audio, vibration, or visual output on the electronic devices 130 and / or 132 that indicates the progress and / or status of the user’s transition.

[0064] In another aspect, surface detection can be performed by RDE blocks 129 and / or 149. This aspect can inform the user whether the user is running too much on one detected surface relative to another detected surface, which can lead to any further injuries or issues. The user can provide input regarding past injuries or current gait type, and / or surface goals the user wants to run on. In such an aspect, a training period can be used to train a model based on the user and the user's equipment, such as shoe type. For example, the user can start a session and provide input to the system indicating whether the run is an asphalt run or a dirt trail run. Over time, the system can then derive potential spatial information / embeddings about the surface using the left / right sensors. Once trained over one to two running sessions, this model can be used to predict the time and / or distance spent on asphalt or dirt trail surfaces in a running session. This model can also be used to predict transitions to different surfaces during a given run and correlate this information with GPS data to determine map information for the user or for the system, which is typically used by the system to suggest running routes with specific surface types for other users.

[0065] In this aspect, the system can also notify the user that, given the user's current gait analysis and past injuries for a given detected surface, it can be harmful to recovery or can lead to further injuries. For example, the system can provide a notification via audio output to electronic device 100 and / or through audio, vibration, or visual output on electronic devices 130 and / or 132 indicating a warning regarding the current gait and / or surface detection.

[0066] In a use case, electronic device 100 can receive one or more signal pairs and provide the signal inputs to RDE block 129. RDE block 129 can perform a first analysis on the signals to determine one or more characteristics of the user's gait and trigger a response when one or more of these characteristics is outside of an acceptable value bound for that respective characteristic (e.g., exceeds an acceptable threshold). For example, if the user's stride length is too short or too long (compared to previous sessions), the user can be notified during the user's running session in order to suggest corrective measures in real-time or near real-time (e.g., during the session). The notification can be issued by electronic device 100 or electronic devices 130 and / or 132. The one or more characteristics of the first analysis can be a subset of the available characteristics. Then, in a second analysis of the signals that occurs at a later time (after the signals are provided to RDE block 149), the signals can be analyzed again at RDE block 149 and one or more characteristics are determined again, but more detailed and / or by using a different (e.g., more complex) machine learning model. Additionally, the second analysis of the signals can include applying additional machine learning models and / or digital signal processing to estimate additional characteristics of the running session.

[0067] In another use case, a similar process to that described in the previous paragraph can be performed, except that the signal can be provided to the RDE block 149 of the electronic device 130 and / or 132, and the first analysis can be performed in near real-time (e.g., depending on the frequency at which signal data is provided from the electronic device 100). Processing can then be performed on the electronic device 130 and / or 132.

[0068] Figure 4 A data flow diagram 400 between the sensors of the electronic device 100 and the running dynamics estimation (RDE) blocks 129 and / or 149 is illustrated. Although the discussion of the data flow diagram 400 below continues to describe the data flow from block to block, it should be understood that the flow can be performed in parallel as appropriate, or can not proceed in the order described, with the associated processes allowing the data flow to differ.

[0069] The sensor data from the electronic device 100 can be the same as discussed above with respect to Figure 3 The RDE blocks 129 and / or 149 can likewise be the same as discussed above with respect to Figure 3 The signal processing data flow between the signals and the RDE blocks 129 and / or 149 can be used to reduce the data of the input signals and pre-process the signals to provide an abstraction of the signal data to the RDE blocks 129 and / or 149.

[0070] For the left side, the signal can be provided to a footfall detection block 160L, which can reduce the noise on the left error microphone 116L signal, the left IMU 118L signal, and / or the left voice accelerometer 121L signal. For example, IMU drift can cause two identical footfalls at two different points in a running session to have different measurements. Reducing the noise can help reduce the impact of IMU drift. Furthermore, since the error microphone can also be used by the ANC, reducing the noise in the error microphone signal can help remove some of the noise associated with the ANC process. The footfall detection block 160L can extract the user’s footfall information, for example, by applying filtering to target certain frequency ranges (e.g., where footfall information will be detected). These frequency ranges can vary depending on which surface is being run on. Thus, footfall detection can be a dynamic process that can analyze the signal to pattern match footfall characteristics across available frequency ranges, and then apply a filter based on the frequency at which the pattern occurs. The footfall detection block 160L can also be used to detect that the user has started running to initiate the digital signal processing associated with performing the running dynamics estimation described herein.

[0071] The footstep detection block 160L can also take as input various sensor information from the opposite side device (e.g., in this case, the electronic device 100 disposed on the right side of the user). The sensor information can be encoded sensor information, that is, the sensor information can be processed using suitable encoding and / or sampling techniques to simplify the representation of the sensor information. Likewise, the footstep detection block 160L can provide similar sensor information to the opposite electronic device 100 (in this case, the electronic device 100 disposed on the right side of the user). The footstep detection block 160L can use the sensor information from the opposite side device to help correlate footstep information from the opposite side with footstep information from the current side to provide a more accurate extrapolation of footstep detection.

[0072] After the footstep detection block 160L, the resulting footstep data can be provided to the footstep separation block 162L in addition to the raw (and optionally noise reduced) signal. The footstep separation block 162L can analyze the data to separate the footstep into right and left footstep. For example, for the footstep separation block 162L, since it is used to process the left side data, the footstep detected for the left foot can be more prominent in the sensor data than the footstep for the right foot.

[0073] At the same time, the footstep detection block 160R and the footstep separation block 162R can be performed in a similar manner as the footstep detection block 160L and the footstep separation block 162L to provide footstep separation information to the RDE block 129 and / or 149.

[0074] The signal processing flow including the footstep detection blocks 160L and 160R and the footstep separation blocks 162L and 162R can be performed by the electronic device 100 and / or 130 and / or 132. For example, the signal processing flow can also reduce the data size of the signal data so that the transfer to the electronic device 130 and / or 132 can be more efficient and use less power than the pre-processed signal from the sensor.

[0075] Figure 4 The RDE block 129 and / or 149 illustrated in FIG. 1 illustrates some of the running dynamics characteristics that can be estimated. As described above, these running dynamics characteristics can include one or more of, for example, step frequency, ground contact time, ground contact time balance, stride length, stride length imbalance, vertical oscillation, or vertical oscillation imbalance.

[0076] Figure 5A data flow diagram 500 between the sensors of the electronic device 100 and the sensor weighting block 190 is illustrated. Although the discussion of the data flow diagram 500 continues below describing the data flow from block to block, it should be understood that the flow can be executed in parallel as appropriate, or can not proceed in the order described, with the associated processes allowing the data flow to differ. The flow of the data flow diagram 500 can apply to each of the sensor data discussed above with respect to Figure 3 and Figure 4 the sensor data discussed below with respect to Figure 6 .

[0077] The sensor data from the electronic device 100 can be the same as discussed above with respect to Figure 3 . The sensor weighting block 190 can take as input the sensor data and the sensing inputs 180, and provide weighted and / or encoded outputs corresponding to each of the same sensor data signals. Each side (left / right) can have its own sensor weighting block 190, or can use a common sensor weighting block 190. For example, the left error microphone signal 116L, the left IMU data signal 118L, and the left voice accelerometer signal 121L can each be provided into the sensor weighting block 190, which can be provided by the controller 124 of the electronic device 100 and / or by the controller 140 of the electronic device 130 / 132. The sensor weighting block 190 can also take as input the sensing inputs 180 regarding the usage mode or environmental factors that can affect the signal weighting. The sensor weighting block 190 can use the input signals and the sensing inputs 180, and provide encoded and / or weighted outputs, each corresponding to a respective input signal.

[0078] The sensing inputs 180 can include inputs indicative of the device’s sensing of the physical environment. For example, the sensing inputs 180 can be indicative of the operating mode, playback volume, ambient sound pressure level, and / or activity level. For example, the operating mode of the electronic device 100 can correspond to an ANC mode for noise cancellation, a transparent mode for audio pass-through, or an automatic ANC for dynamic mixing of the two. If the operating mode includes the ANC mode, the error microphone 116 can be weighted more heavily. However, when the operating mode includes the transparent mode, the error microphone 116 can be weighted less heavily. Similarly, if the playback volume of the electronic device 100 (e.g., as controlled by the electronic devices 130 and / or 132) is high, such that it is more difficult to use the voice accelerometer and error microphone due to the volume and residual echo, the motion sensors and external microphones (e.g., microphones 114 and / or environmental microphones, which can be used in addition to or in place of the error microphone 116) can be weighted more heavily. In contrast, as the ambient sound pressure level increases, any external microphones can be weighted less heavily in proportion to the ambient sound pressure level. In the presence of high user activity, the activity motion sensors can be very sensitive to the user’s activity and can make the signals noisy during the activity. Thus, when high user activity (e.g., heavy running or walking) occurs, then a reduction in the weight on the motion sensors can be performed, thereby reducing the weight on these signals during heavy activity.

[0079] The adjustment of the weights can be performed by a rule-based operation based on thresholds of the sensing inputs 180, which can turn certain signals on or off, or provide linear or non-linear amplification or attenuation based on a mapping of the sensing inputs 180 to various input ranges. For example, the ambient sound pressure level value can drive the weight on some of these sensors, such as the error microphone and external microphones.

[0080] In some implementations, a trained machine learning model encoder can also be used to determine the weights, which takes a vector of signals and the sensed inputs 180 as input to the machine learning model and produces an encoded representation of the input signals as the output signals from the sensor weighting block 190.

[0081] Figure 6 A flowchart of an example process 600 for utilizing a running dynamics estimation (RDE) is illustrated in accordance with some embodiments. Figure 6 One or more blocks (or operations) of the process flowchart can be performed by one or more other components and other suitable devices. Further for explanatory purposes, the blocks are described in a sequential or linear fashion. However, many of the blocks can occur in parallel, and / or the blocks of the process 600 need not be performed in the order shown, and / or one or more blocks of the process 600 need not be performed and / or can be replaced by other operations.

[0082] At block 602, the process 600 can include receiving a first signal from a first device, where the first device is configured to be disposed on a first side of a user. In some implementations, the first signal can include a signal from a first accelerometer or a first audio input. The audio input, for example, can correspond to an error microphone or an external microphone, as described above. The accelerometer can also include other motion data, such as motion data provided by an inertial measurement unit (IMU). Additional signals can be used as the first signal, including a voice accelerometer signal and other microphone signals. Further, the first signal can be a weighted signal, such as discussed above with respect to Figure 5 the weighted signals discussed.

[0083] At block 604, the process 600 can include receiving a second signal from a second device, where the second device is configured to be disposed on a second side of the user, where the first device is configured to be associated with the second device. In some implementations, the first signal can be from a first in-ear microphone, and the second signal can be from a second in-ear microphone. Additional signals can be used as the second signal, including a voice accelerometer signal and other microphone signals. Further, the second signal can be a weighted signal, such as discussed above with respect to Figure 5 the weighted signals discussed.

[0084] The second signal can be received in the same time window as the first signal, such that different successive footstep sounds within the time window can be obtained from the separate signals of each foot, as well as the sequential signals of each foot with the other foot. Although the first signal can be used primarily in conjunction with the foot on the side of the user as the location of the first signal, and the second signal can be used primarily in conjunction with the foot on the side of the user as the location of the second signal, it should be understood that each signal can include footstep sound information for both feet. The first signal and the second signal can be used in combination to determine motion / running characteristics with respect to each foot and both feet together.

[0085] Some aspects can also include receiving a third signal from the first device, the third signal having the other of the signal from the first accelerometer or the first audio input, and receiving a fourth signal from the second device, the fourth signal having the other of the signal from the second accelerometer or the second audio input. In such aspects, the determining one or more motion attributes of the user discussed at block 606 can be further based at least in part on the third signal and the fourth signal. Additional signals can be used as the third signal and / or the fourth signal, including a voice accelerometer signal and other microphone signals. Further, the third signal and the fourth signal can be weighted signals, such as discussed above with respect to Figure 5 the weighted signals discussed.

[0086] Similar to the above regarding the first and second signals, the third and fourth signals can be used in conjunction with the first and second signals, and with each other, within the same time window, to determine footfall information from which individual characteristics of each foot and characteristics common to both feet are derived, e.g., using the techniques described herein.

[0087] At block 606, the process 600 can include determining one or more motion attributes of the user based at least in part on the first and second signals (and optionally the third and fourth signals), where the one or more motion attributes include a left- or right-specific motion attribute. In some implementations, the process 600 can include estimating one or more of a step frequency, a ground contact time, a ground contact time balance, a stride length, or a vertical oscillation based at least in part on the one or more motion attributes. In some aspects, an estimate of the ground contact time balance can be used to determine heel strike information corresponding to the left foot of the user and heel strike information corresponding to the right foot of the user. The left-specific motion attribute can be used in association with the right-specific motion attribute to determine the one or more motion attributes. For example, the one or more motion attributes can include a determination of left- or right-compensation.

[0088] Determining the one or more motion attributes can include analyzing each of the input signals in conjunction with each other over a time period to derive information from the time-aligned signals, and the information can exhibit, e.g., a first footfall within the time period in a respective input signal, followed by a second footfall from the opposite foot, followed by another footfall from the first foot, and so on. In addition to the footfalls themselves, the first, second, and optional third and / or fourth signals can include sensed inputs indicative of surface type and other environmental factors, as well as running style characteristics (e.g., foot strike information, step frequency, ground contact time, ground contact time balance, stride length, and vertical oscillation).

[0089] Estimating can include computing an embedding based at least in part on the one or more motion attributes, and providing the embedding to a classification model to determine the estimate. For example, the classification model can correspond to one or more machine learning models trained on different motion types.

[0090] Estimating or determining can include providing the first signal and the second signal (and optionally, the third signal and the fourth signal) to a machine learning model trained to predict the motion attributes based on the input signals. Further, estimating or determining can include receiving the one or more motion attributes from the machine learning model. For example, the machine learning model can be used to predict the motion attributes. In some implementations, each attribute can have its own machine learning model, while in other implementations, at least some of the attributes can be predicted by a combined machine learning model. Specifically, it can not be feasible or possible to reliably determine such attributes based on signal analysis alone, but by utilizing a machine learning model, the attributes can be determined (e.g., estimated or predicted) to a threshold degree of reliability based on the training data of the model.

[0091] At block 608, the process 600 can include providing feedback to the user device based at least in part on the one or more motion attributes (block 608). In some implementations, for example, the feedback includes an indication of detected right or left side compensation. For example, ground contact time balance can indicate that the user spends more time on the right foot than the left foot, which indicates that the right foot is favored for some compensation to ease the burden on the left foot. While such left or right compensation can temporarily alleviate an injury or pain, long-term compensation can lead to adverse consequences, such as compensatory injury to the other foot.

[0092] In some implementations, for example, the feedback includes an estimate of the percentage of time spent running on a hard surface versus a soft surface. The type, frequency, and amplitude of the signals can indicate whether the running is on a hard surface or a soft surface. This information can be provided to the user to determine whether the user has reached their training goals. For example, running on softer surfaces can reduce the wear and tear on the user’s joints, and thus the user can be encouraged to mix the types of surfaces on which they run to reduce the risk of injury and / or wear. For example, whether the user is a cross country runner or not can be used to correlate shoe types with running types. For example, if the user indicates that they are a cross country runner, shoes with harder soles can be more desirable, but if the user is actually running on a hard surface, the user can be recommended shoes with softer soles. Or the situation can be the opposite, running on softer ground with soft soled shoes can cause certain types of joint wear.

[0093] In some implementations, the user can indicate physical characteristics to provide better recommendations to the user. For example, the user can provide one or more of their weight, shoe size, pronation information (if known), biological sex, age, injury history, fitness level, or activity level (this information can alternatively be supplied by a user profile that is automatically determined by the electronic devices 130 and / or 132). These characteristics can be used to provide more accurate modeling. For example, one predictive model can be used for biological females and another for biological males.

[0094] In some implementations, as discussed above, the signal can be provided to a companion device such as a smart watch or phone. In some implementations, the signal can be filtered on the first device and second device (e.g., a headset) before being transmitted to the companion device. The companion device can also be a tablet, wearable computerized glasses, or a head mounted display.

[0095] In some implementations, at least a portion of determining one or more motion attributes of the user is performed on the companion device. For example, some data processing such as noise reduction and signal filtering can be performed on the first device and second device (e.g., a headset), for example, by digital signal processing techniques, and the remaining processing can be performed on the companion device.

[0096] However, in some implementations, some processing on the first device and second device can result in real-time feedback being supplied to the user. For example, a sound notification emitted by the user's headset can alert the user to check their phone or watch for details about the running dynamics estimate, indicating potential problems and suggesting corrections.

[0097] In some aspects, process 600 may also include receiving information from a user related to an upcoming event, relating that information to route information for the upcoming event, and suggesting training routes to the user based on the route information. For example, the user may provide information about the event they are training for, such as a specific marathon. Electronic devices 130 and / or 132 may obtain information about the upcoming event. For example, electronic devices 130 and / or 132 may retrieve route information for the event from the event organizer or from a server device that tracks and provides such information. The route information may be analyzed by electronic devices 130 and / or 132, and routes near the device may be suggested based on similar characteristics. For example, if the route indicates uphill and downhill sections, hard surfaces and soft surfaces, the suggested route may attempt to match uphill and downhill sections, hard surfaces and soft surfaces as the event route. In some cases, matching does not need to match the order of these features, but may attempt to match, for example, intervals and distances of ascent or descent, and / or distances between grass or dirt and pavement. In some cases, route information may include historical route information for the event, for example, when it is not available for the upcoming event. For example, some events may have similar routes year after year. In such cases, historical routes can be used to approximate the route of an upcoming event.

[0098] Figure 7 A sample user interface 700, according to some specific implementation, is illustrated for displaying detailed information about running dynamics estimation, which is determined, for example, by running dynamics estimation using the process described herein. The interface is displayed as output to a smartphone; however, output to any suitable electronic device, such as electronic devices 130 and / or 132, is also conceivable.

[0099] Element 702 may correspond to a title that shows a specific running dynamic, as estimated. For example, running dynamic may correspond to ground contact time. The name of the dynamic may be shown in the title section. Element 704 may correspond to the selection of a time period for a graph showing data related to the running dynamic. For example, in this example, the time period may correspond to the past hour, day, week, month, six months, or year. Other embodiments may provide control elements to input specific custom date ranges.

[0100] Element 706 indicates what kind of data to display for element 708 within the selected time period. In this example, average data for a 6-month time period is displayed; however, any appropriate statistical data, such as maximum, minimum, range, etc., can be displayed. Element 708 can correspond to the numerical value and unit of the statistical type specified in element 706. For example, in use, the word "statistic" can be replaced with a number such as 254 ms, for example, indicating the average ground contact time over the entire six-month time period. Element 710 can be used to show the actual data range for the six-month time period. For example, the graph indicated by element 712 can be horizontally scrollable and shows a window with a length corresponding to the time period specified by element 704. Element 710 can provide the date range of the data displayed in element 712.

[0101] Element 712 can correspond to a graph showing the running dynamics within the time period specified by element 704. The x-axis 714 can have labels based on both the date range and the time period, and can automatically adjust according to the date range (element 710) and the time period (element 704). The y-axis 716 can display linear or logarithmic data. Figure 7 In the example shown, the y-axis 716 is linear. The y-axis 716 can be marked with values ​​including y, y+w, y+2w, and y+3w. The y-value can be selected based on the lowest value of the data in the graph, and the w-value can be selected based on the highest value of the data in the graph, so as to display the extreme values ​​between the upper and lower values ​​of the y-axis 716.

[0102] Element 718 can be a label for box 720. For example, element 718 could display "About Ground Contact Time" or another running dynamic. This "About" information could correspond to a heading (element 702). Box 720 can contain a description of what the running dynamic refers to, including, for example, the units of measurement for the specific running dynamic displayed.

[0103] It should be understood that Figure 7 The interface depicted is merely an example and can be changed according to the designer's preferences. The graph (element 712) will change based on the underlying data and the selected values ​​of other elements.

[0104] Figure 7 A sample user interface 800, according to some specific implementation, is shown for displaying fitness details associated with the process of collecting sensor data and performing running dynamic estimation. Interface 800 is shown as output to a smartphone; however, output to any suitable electronic device, such as electronic devices 130 and / or 132, is also conceivable.

[0105] Element 802 is the page title. In the example implementation, the title is "Fitness Details," however, any suitable title may be used. Element 804 is a map appropriately zoomed to the area of ​​the route being run. The map may include streets, place names, satellite overlays, etc. Element 806 is an overlay on the map showing the route being run. Although shown as dashed lines, the map may be colored, and the route may be overlaid with a specific color. The map (element 804) and route (element 806) may correspond to an event map of the user's upcoming event. For example, the user may practice an actual route, or a route that mimics at least some attributes of the upcoming event (such as elevation changes, surface type, distance, and slope) may be provided. In some cases, the route may correspond to a historical route, for example, a route from a past event specified by the user. For example, suppose the user wants to participate in the 1995 Boston Marathon; the user may specify the event, and electronic devices 130 and / or 132 may provide route information, for example, by obtaining route information from another source. A route for the upcoming event may be obtained similarly. Element 808 shows the time elapsed during the workout, and element 810 shows the distance traveled during the workout.

[0106] Element 812 illustrates a menu item for selecting a running dynamic estimate for fitness calculations. For example, element 812 could correspond to statistics for cadence. Graph 814 can illustrate an expanded view of the sub-menu item under the statistics selection element 812. For example, graph 814 could correspond to a visual representation of the cadence estimate derived from sensor data collected during fitness. Graph 814 may include an x-axis corresponding to the duration of fitness and a y-axis scaled to show data associated with the illustrated metric. Graph 814 may include type values ​​representing cadence in this example, and may also include averages or other statistics for fitness, including the values ​​and units of the values. For example, the illustrated “Type” label could be replaced with “Average Cadence: 154 SPM,” where SPM corresponds to steps per minute. The values ​​xx-yy could be replaced with dynamic range. In this example, for example, these values ​​could be replaced with 122-175 SPM.

[0107] Element 816 may correspond to a submenu item of the statistics menu (element 814). For example, if the user selects element 816, graph 814 can be collapsed, and element 816 can be expanded to show a graph similar to the one displayed at graph 814. In this example, the collapsed graph 814 can then be replaced to display the label "Step Frequency".

[0108] Elements 818, 820, and 822 each provide interface items that display corresponding data when selected. For example, element 818, when selected, may display predictions based on running dynamics. These predictions may include injury-related predictions, such as providing predictions about whether an injury to a user's foot or leg is developing or healing. Such predictions can be made at least in part by comparing fitness data with previous fitness data and relating that fitness data to running dynamics that indicate an imbalance in movement between the right foot and right leg and the left foot and left leg. In some cases, the user may provide information about the injury, and this information can be used to help track the healing process. Predictions may also include those related to the user's physical attributes. For example, predictions may include those related to whether the user's foot position during stride is overpronation, underpronation, or neutral pronation. Furthermore, predictions may include those related to running technique, such as whether the user runs heel-strike, midfoot strike, or forefoot or ball-strike. Such predictions can be made by feeding sensor data to models and classification systems trained to predict such data, as described above.

[0109] Element 820 provides information about the percentage of a user's fitness on pavement versus dirt. This element also provides information on whether the user's fitness has reached their target fitness goals on pavement versus dirt, and offers route suggestions to modify the fitness to more closely match the user's desired surface type distribution. Element 822 can provide warning information, such as information about where running dynamics estimates indicate the user needs to improve their running style and / or posture. Suggestions for improving running dynamics, such as workouts or focus points, can also be provided. Product recommendations, such as footwear, may also be suggested.

[0110] Element 824 indicates that the tiles corresponding to graphs 814, 816, 818, 820, and 822 can be scrolled up and down, for example, to show additional running dynamics that the user can display.

[0111] As described above, one aspect of the present invention is the collection and use of data available from specific and legitimate sources to provide running dynamics estimation. This disclosure contemplates that, in some instances, the collected data may include personal information data that uniquely identifies or can be used to identify a specific person. Such personal information data may include voice data, demographic data, location-based data, online identifiers, telephone numbers, email addresses, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other personal information.

[0112] Some implementations described herein may include the use of learning-based and / or non-learning-based processes. This use may include collecting, preprocessing, encoding, labeling, organizing, analyzing, recommending, and / or generating data. Entities that collect, share, and / or otherwise utilize user data should provide transparency and / or obtain user consent when collecting such data. This disclosure recognizes that the use of data in the RDE process can be used to benefit users. For example, the data can be used to train models that can be deployed to improve the performance, accuracy, and / or functionality of applications and / or services. Therefore, the use of data enables the RDE process to be adapted and / or optimized to provide a more personalized, efficient, and / or enhanced user experience. Such adaptation and / or optimization may include customizing content, recommendations, and / or interactions for individual users, as well as simplifying processes and / or implementing more intuitive interfaces. This disclosure also considers further beneficial uses of data in the RDE process.

[0113] This disclosure envisions that, in some embodiments, the data used by the RDE process includes publicly available data. To protect user privacy, data may be anonymized, aggregated, and / or otherwise processed to remove or, where possible, limit any individual identification. As discussed herein, entities that collect, share, and / or otherwise utilize such data should obtain user consent before collecting such data and / or provide transparency when collecting such data. Furthermore, this disclosure envisions that entities responsible for the use of data (including, but not limited to, data used in connection with the RDE process) should strive to comply with robust privacy policies and / or privacy measures.

[0114] For example, such entities may implement and consistently follow strategies and measures deemed to meet or exceed industry standards and regulatory requirements for developing and / or training RDE processes. In doing so, efforts should be made to ensure that all intellectual property and privacy considerations are maintained. Training should include measures to protect training data (such as personal information) by providing adequate safeguards against misuse or exploitation. Such strategies and measures should cover all phases of RDE process development, training, and use, including data collection, data preparation, model training, model evaluation, model deployment, and ongoing monitoring and maintenance. Transparency and measurability should always be maintained. Such policies should be readily accessible to users and should be updated as data collection and / or use change. User data should be collected for the entity's lawful and reasonable use and not shared or sold outside of these lawful uses. Furthermore, such collection and sharing should be conducted in a transparent manner to users and / or with their informed consent. Additionally, such entities should consider taking any necessary steps to defend and safeguard access to such data and ensure that others with access to the data comply with their privacy policies and processes. Furthermore, such entities should be subject to appropriate third-party assessments for transparency purposes to demonstrate their compliance with widely accepted privacy policies and practices. In addition, policies and / or practices should be appropriate for the specific types of data collected and / or accessed, and tailored to specific use cases and applicable laws and standards, including jurisdiction-specific considerations.

[0115] In some implementations, the RDE process may utilize a model that can be trained (e.g., in a supervised or unsupervised learning manner) using a variety of training data, including data collected using the user's device. This use of the user-collected data may be limited to operations on the user's device. For example, model training may be performed locally on the user's device, so that no part of the data is transmitted to another device. In other implementations, model training may be performed using one or more other devices besides the user's device (e.g., a server), but in a privacy-preserving manner, such as through multi-party computation, encrypted by secretly sharing data, or other means, ensuring that user data is not leaked to these other devices.

[0116] In some implementations, the trained model may be centrally stored on a user device (e.g., on the user's phone), on multiple devices (e.g., as in federated learning), or in an earpiece. This distributed storage can also be accomplished in a privacy-preserving manner, for example, through encryption, where each piece of data is fragmented so that the data cannot be reassembled or used by a device alone (i.e., only in conjunction with another device) or only by the user device. In this way, user or device behavior patterns are not leaked, while leveraging the increased computational resources of other devices to train and execute the ML model. Therefore, user-collected data is protected. In some specific implementations, data from multiple devices can be combined in a privacy-preserving manner to train the ML model.

[0117] In some implementations, this disclosure envisions that data used for the RDE process may be kept strictly separate from the platform on which the RDE process is deployed and / or used for user interaction and / or data processing. In such implementations, data used for offline training of the RDE process may be kept in a secure data repository with restricted access and / or not retained for longer than necessary for the training purpose. In some implementations, the RDE process may utilize a local memory cache to temporarily store data during a user session. A local memory cache can be used to improve the performance of the RDE process. However, to protect user privacy, data stored in the local memory cache may be erased after the user session ends. Any temporary cached data used for online learning or inference may be quickly erased after processing. All data collection, transmission, and / or storage should be conducted using industry-standard encryption and / or secure communication.

[0118] In some implementations, as described above, techniques such as federated learning, differential privacy, secure hardware components, homomorphic encryption and / or multi-party computation, and others, can be used to further protect personal information data during the training and / or use of the RDE process. The RDE process should be monitored for changes in the underlying data distribution, such as concept drift or data skew that may degrade the performance of the RDE process over time.

[0119] In some implementations, a combination of offline and online training is used to train the RDE process. Offline training can use a carefully selected dataset to establish baseline model performance, while online training allows the RDE process to continuously adapt and / or improve. This disclosure recognizes the importance of maintaining strict data management measures throughout this process to ensure user privacy is protected.

[0120] In some implementations, the RDE process may be designed with safeguards to maintain adherence to its original intended purpose, even when the RDE process is adapted based on new data. Any significant changes to data collection and / or application used in the RDE process may (and in some cases should) be transparently communicated to affected stakeholders, and / or user consent may be obtained when the manner of collecting and / or using user data changes.

[0121] Regardless of the foregoing, this disclosure also contemplates implementation schemes for users to selectively restrict and / or block the use and / or access to data. That is, this disclosure contemplates that hardware and / or software elements may be provided to prevent or block access to data. For example, with respect to some services, the inventive technology should be configured to allow users to opt-in or opt-out to participate in data collection at any time during or after service registration. In another example, the inventive technology should be configured to allow users to opt out of providing certain data used for training RDE processes and / or as input during the inference phase of such systems. In yet another example, the inventive technology should be configured to allow users to choose to limit the length of time data is retained or to completely prohibit the RDE process from using their data. In addition to providing "opt-in" and "opt-out" options, this disclosure also contemplates providing notifications related to access to or use of personal information. For example, users may be notified when their data is input into an RDE process for training or inference purposes, and / or alerted when the RDE process generates output or makes decisions based on their data.

[0122] This disclosure recognizes that RDE processes should incorporate explicit limitations and / or oversight to mitigate risks that may still exist even when such systems have been designed, developed, and / or operated in accordance with industry best practices and standards. For example, outputs may be generated that could be considered erroneous, harmful, offensive, and / or biased; such outputs may not necessarily reflect the opinions or positions of the entity that developed or deployed these systems. Furthermore, in some cases, references to or non-references to third-party products and / or services in these outputs should not be construed as an endorsement or association by the entity providing the RDE process with third-party products and / or services. The generated content may be filtered to remove potentially inappropriate or dangerous material before being presented to users, while maintaining the ability for human oversight and / or to cover or correct erroneous or undesirable outputs as a safeguard.

[0123] This disclosure also envisions that users of the RDE process should avoid using the service in any way that infringes upon, misappropriates, or violates the rights of any party. Furthermore, the RDE process should not be used for any illegal or unlawful activities, nor should it be used to develop any application or use case that will commit or assist in the committing of a crime or other tortious, illegal, or unlawful act, including misinformation, false information, misrepresentation (e.g., deepfakes), deception, impersonation, and propaganda. The RDE process should not violate, misappropriate, or infringe upon any party's copyright, trademark, privacy and portrait rights, trade secrets, patents, or other proprietary or statutory rights, and should be appropriately attributed to the content as needed. Furthermore, the RDE process should not interfere with any security, digital signature, digital rights management, content protection, verification, or authentication mechanisms. The RDE process should not misrepresent machine-generated output as human-generated output.

[0124] Furthermore, the intent of this disclosure is that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by restricting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing identifiers, controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods such as differentiated privacy.

[0125] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it is also contemplated that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not become inoperable due to the absence of all or part of such personal information data.

[0126] Figure 9Example computing device 900, according to one or more specific embodiments, is illustrated and can be used to implement aspects of the art described herein. Computing device 900 can be any computing device or server used to generate the features and processes described above and / or can be part of any computing device or server used to generate the features and processes described above, including but not limited to laptop computers, smartphones, tablet devices, wearable devices such as goggles or glasses. Computing device 900 can include various types of computer-readable media and interfaces for various other types of computer-readable media. Computing device 900 includes persistent storage device 902, system memory 904 (and / or buffers), input device interface 906, output device interface 908, bus 910, ROM 912, one or more processing units 914, one or more network interfaces 916, and / or subsets and variations thereof.

[0127] Bus 910 generally represents all system, peripheral, and chipset buses that communicatively connect numerous internal devices of computing device 900. In one or more embodiments, bus 910 communicatively connects one or more processing units 914 to ROM 912, system memory 904, and persistent storage device 902. One or more processing units 914 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes disclosed in this subject matter. In different embodiments, one or more processing units 914 may be a single processor or a multi-core processor.

[0128] ROM 912 stores static data and instructions required by one or more processing units 914 and other modules of computing device 900. On the other hand, persistent storage device 902 can be a read-write memory device. Persistent storage device 902 can be a non-volatile memory cell that stores instructions and data even when computing device 900 is turned off. In one or more embodiments, mass storage devices (such as disks or optical discs and their corresponding disk drives) can be used as persistent storage device 902.

[0129] In one or more embodiments, a removable storage device (such as a floppy disk, flash drive, and its corresponding disk drive) may be used as persistent storage device 902. Like persistent storage device 902, system memory 904 may be a read-write memory device. However, unlike persistent storage device 902, system memory 904 may be volatile read-write memory, such as random access memory. System memory 904 may store any instructions and data that one or more processing units 914 may need during operation. In one or more embodiments, the processes disclosed in this subject matter are stored in system memory 904, persistent storage device 902, and / or ROM 912. One or more processing units 914 retrieve instructions to be executed and data to be processed from these various memory units to execute the processes of one or more embodiments.

[0130] Bus 910 is also connected to input device interface 906 and output device interface 908. Input device interface 906 enables a user to communicate information and select commands to computing device 900. Input devices that can be used with input device interface 906 may include, for example, an alphanumeric keypad and pointing devices (also referred to as "cursor control devices"). Output device interface 908 may, for example, enable the display of images generated by computing device 900. Output devices that can be used with output device interface 908 may include, for example, printers and display devices such as liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, flexible displays, flat panel displays, solid-state displays, projectors, or any other device for outputting information.

[0131] One or more embodiments may include a device that functions as both an input and an output device, such as a touchscreen. In these embodiments, the feedback provided to the user may be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user may be received in any form, including acoustic, verbal, or tactile input.

[0132] Finally, as Figure 9 As shown, bus 910 also couples computing device 900 to one or more networks and / or one or more network nodes via one or more network interfaces 916. In this way, computing device 900 may be part of a computer network (such as a LAN, wide area network (“WAN”), or intranet), or may be part of a network of networks (such as the Internet). Any or all components of computing device 900 may be used in conjunction with the disclosure herein.

[0133] Specific embodiments within the scope of this disclosure may be implemented in part or in whole using a tangible computer-readable storage medium (or a plurality of tangible computer-readable storage media of one or more types) that encodes one or more instructions. The tangible computer-readable storage medium may also be substantially non-transitory.

[0134] Computer-readable storage media can be any storage medium that can be read, written, or otherwise accessed by general-purpose or special-purpose computing devices, including any processing electronics and / or processing circuitry capable of executing instructions. For example, without limitation, computer-readable media can include any volatile semiconductor memory, such as RAM, DRAM, SRAM, T-RAM, Z-RAM, and TTRAM. Computer-readable media can also include any non-volatile semiconductor memory, such as ROM, PROM, EPROM, EEPROM, NVRAM, flash memory, nvSRAM, FeRAM, FeTRAM, MRAM, PRAM, CBRAM, SONOS, RRAM, NRAM, track memory, FJG, and Millipede memory.

[0135] Furthermore, computer-readable storage media may include any non-semiconductor memory, such as optical disc storage devices, magnetic disk storage devices, magnetic tape, other magnetic storage devices, or any other medium capable of storing one or more instructions. In one or more embodiments, the tangible computer-readable storage medium may be directly coupled to a computing device, while in other embodiments, the tangible computer-readable storage medium may be indirectly coupled to a computing device, for example, via one or more wired connections, one or more wireless connections, or any combination thereof.

[0136] Instructions can be directly executable or can be used to develop executable instructions. For example, instructions can be implemented as executable or non-executable machine code, or as high-level language instructions that can be compiled to produce executable or non-executable machine code. Furthermore, instructions can also be implemented as data, or may include data. Computer executable instructions can also be organized in any format, including routines, subroutines, programs, data structures, objects, modules, applications, applets, functions, etc. As those skilled in the art will recognize, details including, but not limited to, the number, structure, sequence, and organization of instructions can vary significantly without altering the underlying logic, functionality, processing, and output.

[0137] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, one or more specific implementations are executed by one or more integrated circuits such as ASICs or FPGAs. In one or more specific implementations, such integrated circuits execute instructions stored on the circuit itself.

[0138] Those skilled in the art will recognize that the various exemplary blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability between hardware and software, the various exemplary blocks, modules, elements, components, methods, and algorithms have been generally described above in terms of functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functionality in different ways for each specific application. Various components and blocks can be arranged differently (e.g., in different orders or partitioned in different ways), all without departing from the scope of the subject matter.

[0139] It should be understood that any particular order or hierarchy of the boxes in the process disclosed in this invention is an example of the exemplary method. Based on design preferences, it should be understood that the particular order or hierarchy of the boxes in the process may be rearranged or all illustrated boxes may be executed. Any box within these boxes may be executed simultaneously. In one or more embodiments, multitasking and parallel processing may be advantageous. Furthermore, the division of various system components in the above embodiments should not be construed as requiring such division in all embodiments, and it should be understood that program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0140] As used in this specification and any claim of this patent application, the terms "base station," "receiver," "computer," "server," "processor," and "memory" refer to electronic devices or other technical devices. These terms exclude persons or groups of persons. For the purposes of this specification, the terms "display" or "being displayed" mean displaying on an electronic device.

[0141] As used herein, the phrase “at least one of” following a series of items separated by the terms “and” or “or” modifies the list as a whole, not each member of the list (i.e., each item). The phrase “at least one of” does not require selection of at least one of each of the listed items; rather, it allows for the inclusion of at least one of any one of the items and / or at least one of any combination of items and / or at least one of each item. For example, the phrases “at least one of A, B, and C” or “at least one of A, B, or C” respectively refer to only A, only B, or only C; any combination of A, B, and C; and / or at least one of each of A, B, and C.

[0142] The predicates “configured to,” “operable to,” and “programmed to” do not imply any specific tangible or intangible modification to a particular subject matter but are intended to be used interchangeably. In one or more embodiments, a processor configured to monitor and control operations or components may also mean that the processor is programmed to monitor and control operations or that the processor is operable to monitor and control operations. Similarly, a processor configured to execute code may be interpreted as a processor programmed to execute code or operable to execute code.

[0143] Phrases such as aspect, that aspect, on the other hand, some aspects, one or more aspects, implementation, that implementation, another implementation, some implementations, one or more implementations, implementation scheme, that implementation scheme, another implementation scheme, some implementation schemes, one or more implementation schemes, configuration, that configuration, other configuration, some configurations, one or more configurations, subject matter technology, disclosure, this disclosure, other variations thereof, etc., are for convenience only and do not imply that disclosures involving one or more such phrases are essential to the subject matter technology, nor do they imply that such disclosures apply to all configurations of the subject matter technology. Disclosures involving one or more such phrases may apply to all configurations or one or more configurations. Disclosures involving one or more such phrases may provide one or more examples. Phrases such as aspect or some aspects may refer to one or more aspects, and vice versa, and this applies similarly to the other foregoing phrases.

[0144] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” or “example” is not necessarily to be construed as preferred or superior to other specific embodiments. Furthermore, the terms “comprising,” “having,” etc., as used in the specification or claims, are intended to be inclusive, similar to how the term “comprising” is interpreted when used as a transitional word in the claims.

[0145] All structural and functional equivalents of elements throughout the various aspects described herein that are known or later become apparent to those skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be made public, regardless of whether such disclosure is expressly stated in the claims. No claim element should be interpreted in accordance with 35 USC §112(f) unless the element is expressly stated using the phrase “component for…” or, in the case of a method claim, using the phrase “step for…”.

[0146] The preceding description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, this claim is not intended to be limited to the aspects shown herein, but rather to be consistent with the language of the claim, wherein references to singular elements are not intended to mean “only one” but rather “one or more” unless specifically indicated. Unless otherwise specifically stated, the term “some” means one or more. Male pronouns (e.g., his) include female and neutral pronouns (e.g., her and its), and vice versa. Titles and subtitles (if any) are used for convenience only and do not limit the disclosure of this subject matter.

Claims

1. A method, the method comprising: Receive a first signal from a first device during a first time period, wherein the first device is configured to be located on a first side of the user; During the first time period, a second signal is received from a second device, wherein the second device is configured to be located on a second side of the user, and wherein the first device is configured to be associated with the second device; The user's one or more motion attributes are determined at least in part based on the first signal in combination with the second signal, wherein the one or more motion attributes include a left-side or right-side specific motion attribute; as well as Feedback is presented on the user device based at least in part on one or more of the motion attributes.

2. The method of claim 1, wherein the first signal comprises a signal from a first accelerometer or a first audio input, and wherein the second signal comprises a signal from a second accelerometer or a second audio input.

3. The method according to claim 2, further comprising: During the first time period, a third signal is received from the first device, the third signal including either the signal from the first accelerometer or the first audio input; as well as During the first time period, a fourth signal is received from the second device, the fourth signal including either the signal from the second accelerometer or the second audio input, wherein the determination of the user's one or more motion attributes is further based at least in part on the third signal in combination with the fourth signal.

4. The method according to claim 3, further comprising: The first signal or the third signal is weighted based on the physical environment sensed by the first device.

5. The method according to claim 1, further comprising: One or more of the following motion properties are estimated, at least in part, based on one or more of the aforementioned motion properties: cadence, ground contact time, ground contact time balance, stride length, or vertical oscillation.

6. The method according to claim 5, further comprising: The embedding is calculated at least in part based on one or more of the motion properties; as well as The embedding is provided to the classification model to determine one or more estimates.

7. The method of claim 5, wherein an estimate of ground contact time balance is used to determine heel strike information corresponding to the user's left foot and heel strike information corresponding to the user's right foot.

8. The method of claim 1, wherein the determination comprises: An aggregated input is generated from the first signal and the second signal during a first time period, wherein the first signal and the second signal are filtered based on context information associated with pre-configured user settings; as well as One or more motion attributes during the first time period are determined based on the aggregated input.

9. A system comprising: processor; and Memory device, wherein the processor is configured to: Receive a first signal from a first device during a first time period, wherein the first device is configured to be located on a first side of the user; During the first time period, a second signal is received from a second device, wherein the second device is configured to be located on a second side of the user, and wherein the first device is configured to be associated with the second device; The user's one or more motion attributes are determined at least in part based on the first signal in combination with the second signal, wherein the one or more motion attributes include a left-side or right-side specific motion attribute; and Feedback is provided to the user device based at least in part on one or more of the motion attributes.

10. The system of claim 9, wherein the feedback includes an indication of detected right-side or left-side compensation.

11. The system of claim 9, wherein the feedback includes an estimate of the percentage of time spent running on a hard surface compared to the time spent running on a soft surface.

12. The system of claim 9, wherein the processor is further configured to: Receive user information indicating the user's physical characteristics, wherein the physical characteristics include one or more of height, weight, injury history, fitness level, or activity level.

13. The system of claim 9, wherein the processor is further configured to: Receive information related to upcoming events from the user; The information is correlated with the route information of the upcoming event; and The training route is suggested to the user based on the route information.

14. The system of claim 13, wherein the route information includes historical route information of previous versions of the upcoming event.

15. The system of claim 9, wherein the first signal comprises a first accelerometer signal, and wherein the second signal comprises a second accelerometer signal, wherein the processor is further configured to: During the first time period, a third signal including the first audio signal is received; During the first time period, receive a fourth signal including a second audio input; and The first signal or the third signal is weighted based on the sensed physical environment of the first device, wherein the determination of the user's one or more motion attributes is further based at least in part on the third signal in combination with the fourth signal.

16. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to: Receive a first signal from a first device during a first time period, wherein the first device is configured to be located on a first side of the user; During the first time period, a second signal is received from a second device, wherein the second device is configured to be located on a second side of the user, and wherein the first device is configured to be associated with the second device; The user's one or more motion attributes are determined at least in part based on the first signal in combination with the second signal, wherein the one or more motion attributes include a left-side or right-side specific motion attribute; and Feedback is provided to the user device based at least in part on one or more of the motion attributes.

17. The non-transitory computer-readable medium of claim 16, wherein the instructions further cause the one or more processors to perform the following operations: The first signal is filtered on the first device; Filtering the second signal on the second device; and Samples of each signal are provided to the auxiliary equipment associated with the first device and the second device.

18. The non-transitory computer-readable medium of claim 17, wherein determining at least a portion of the one or more motion attributes of the user is performed on the accompanying device.

19. The non-transitory computer-readable medium of claim 17, wherein the instructions further cause the one or more processors to perform the following operations: Provide at least some of the one or more motion attributes of the user on the first device or on the second device; and Provide real-time feedback to the user.

20. The non-transitory computer-readable medium of claim 16, wherein the first signal originates from a first in-ear microphone, and wherein the second signal originates from a second in-ear microphone.

Images