Determining the direction of movement

Abstract

Methods, systems, and media are provided for determining a user's motion direction. In some embodiments, the method involves using a control system to obtain user acceleration data associated with a user. The method involves using the control system to determine a motion period associated with the user's athletic activity using the user acceleration data. The motion period indicates a duration between two successive movements by the user. The method involves using the control system to determine a motion direction corresponding to the athletic activity using the user acceleration data based on a direction of acceleration orthogonal to the motion direction such that at least a portion of the user acceleration data is anti-periodic over a period of time corresponding to the motion period.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 293,444, filed December 23, 2021, and U.S. Provisional Patent Application No. 63 / 376,347, filed September 20, 2022, each of which is incorporated by reference in its entirety.

[0002] Technical Field The present disclosure relates to systems, methods, and media for determining direction of motion. [Background technology]

[0003] Determining the direction of a user's movement can be useful, however, determining the direction of a user's movement can be difficult due to, for example, artifacts in the measured sensor data due to the user's movement, confounding sensor data due to the user looking around while moving, and the like.

[0004] Notation and Nomenclature Throughout this disclosure, including the claims, the terms "speaker," "loudspeaker," and "audio reproduction transducer" are used interchangeably to refer to any sound emitting transducer (or set of transducers). A typical headphone set includes two speakers. A speaker may be implemented to include multiple transducers (e.g., woofers and tweeters) that may be driven by a single common speaker feed or multiple speaker feeds. In some examples, the speaker feeds may receive different processing in different circuit branches coupled to different transducers.

[0005] Throughout this disclosure, including the claims, the phrase performing an operation "on" a signal or data (e.g., filtering, scaling, transforming, or applying a gain to a signal or data) is used broadly to refer to performing an operation on the signal or data directly, or on a processed version of the signal or data (e.g., on a version of that signal that has been subjected to preliminary filtering or preprocessing before performing the operation).

[0006] Throughout this disclosure, including the claims, the term "system" is used broadly to denote a device, system, or subsystem. For example, a subsystem that implements a decoder may be referred to as a decoder system, and a system that includes such a subsystem (e.g., a system that generates X output signals in response to multiple inputs, where the subsystem generates M of the inputs and the other XM inputs are received from external sources) may also be referred to as a decoder system.

[0007] Throughout this disclosure, including the claims, the term "processor" is used broadly to denote a system or device that is programmable or otherwise configurable (e.g., with software or firmware) to perform operations on data (e.g., audio or video or other image data). Examples of processors include field programmable gate arrays (or other configurable integrated circuits or chipsets), digital signal processors programmed and / or otherwise configured to perform pipeline processing on audio or other sound data, programmable general-purpose processors or computers, and programmable microprocessor chips or chipsets. Summary of the Invention [Means for solving the problem]

[0008] Methods, systems, and media are provided for determining a user's motion direction. In some embodiments, the method may include obtaining user acceleration data associated with a user using a control system. The method may involve using the control system to determine a motion period associated with the user's athletic activity using the user acceleration data, the motion period indicating a duration between two successive movements by the user. The method may involve using the control system to determine a motion direction corresponding to the athletic activity using the user acceleration data based on a direction of acceleration orthogonal to the motion direction such that at least a portion of the user acceleration data is anti-periodic over a period of time corresponding to the motion period.

[0009] In some examples, the at least the portion of the user acceleration data is anti-periodic over a period of time corresponding to a motion cycle, such that a cross-correlation between the at least the portion of the user acceleration data and a version of the at least the portion of the user acceleration data delayed by a motion cycle is negative.

[0010] In some examples, the at least the portion of user acceleration data that is counter-periodic includes local maxima and minima that occur after subsequent time intervals corresponding to athletic activity.

[0011] In some examples, the athletic activity includes walking or running, and the direction of motion includes a direction in which the user is walking or running.

[0012] In some examples, the method further includes transforming the user acceleration data from a user-centered coordinate system to a fixed coordinate system to generate transformed user acceleration data, and the period of motion is determined using the transformed user acceleration data. In some examples, transforming the user acceleration data includes using user head orientation data obtained from the one or more gyroscopes. In some examples, the head orientation data is used to identify a direction of motion.

[0013] In some examples, determining the period of motion includes using data associated with a vertical component of the user acceleration data. In some examples, determining the period of motion further includes providing the data associated with the vertical component of the user acceleration data to a plurality of narrowband filters, each narrowband filter of the plurality of narrowband filters being associated with a different frequency; and generating a prediction of the motion frequency based on an output of the plurality of narrowband filters, the prediction of the motion frequency being used to determine the period of motion. In some examples, using the prediction of the motion frequency to determine the period of motion includes identifying a period of an output of a selected narrowband filter of the plurality of narrowband filters, the selected narrowband filter of the plurality of narrowband filters producing a maximum output signal.

[0014] In some examples, the at least a portion of the user acceleration data that is counter-periodic over a time period corresponding to a motion cycle includes one or more horizontal components of user acceleration data. In some examples, identifying a direction of acceleration orthogonal to the direction of motion includes determining a cross-covariance vector corresponding to data associated with the one or more horizontal components of user acceleration data and a version of data associated with the one or more horizontal components of user acceleration data delayed by the motion cycle. In some examples, the cross-covariance vector is determined by determining a cross-product vector from two or more horizontal components of user acceleration data and two or more horizontal components of user acceleration data delayed by the motion cycle for each of a plurality of sample times, and combining two or more of the cross-product vectors from two or more of the plurality of sample times to determine the cross-covariance vector. In some examples, the two or more of the cross-product vectors are combined using a weighting, with the cross-product vector associated with a more recent sample time being weighted more heavily than the cross-product vector associated with a less recent sample time. In some examples, the weighting includes an exponential decay function. In some examples, the method further includes applying a band pass filter to the one or more horizontal components of user acceleration data before identifying a direction of acceleration orthogonal to the direction of motion.

[0015] In some examples, the movement direction is determined by: determining an intermediate angle corresponding to twice the angle associated with the movement direction; and determining an angle associated with the movement direction based on the intermediate angle. In some examples, determining an angle associated with the movement direction based on the intermediate angle includes: halving the intermediate angle to determine a candidate movement direction; and selecting the angle associated with the movement direction as the candidate movement direction or 180 degrees from the candidate movement direction based on the head orientation data. In some examples, determining an angle associated with the movement direction based on the intermediate angle includes: determining a difference between the intermediate angle and an angle associated with an orientation of the user's head, and determining an angle associated with the movement direction based on the difference.

[0016] In some examples, the user acceleration data is obtained from one or more accelerometers disposed in or on a user device configured to be carried or worn by the user. In some examples, the user device includes headphones. In some examples, the method further involves rendering audio content to be presented by the headphones based on the identified direction of motion. In some examples, the audio content is rendered to have a spatial perception centered on the identified direction of motion.

[0017] Some or all of the operations, functions, and / or methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including, but not limited to, random access memory (RAM) devices, read-only memory (ROM) devices, etc. Thus, some innovative aspects of the subject matter described in this disclosure may be implemented via one or more non-transitory media having software stored thereon.

[0018] At least some aspects of the present disclosure may be implemented via an apparatus. For example, one or more devices may be capable of at least partially performing the methods disclosed herein. In some implementations, the apparatus is or includes an audio processing system having an interface system and a control system. The control system may include one or more general purpose single or multi-chip processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or combinations thereof.

[0019] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. It should be noted that the relative dimensions of the following figures may not be drawn to scale. [Brief description of the drawings]

[0020] [Figure 1] FIG. 13 illustrates axes and vectors associated with movement according to some embodiments.

[0021] [Diagram 2] 1 is a plot showing vertical and lateral components of acceleration in accordance with some embodiments.

[0022] [Diagram 3] 1 is a flowchart of an exemplary process for determining direction of motion, according to some embodiments.

[0023] [Figure 4] 4 is a flowchart of an exemplary process for transforming acceleration data from a user-centered coordinate system to a fixed coordinate system in accordance with some embodiments.

[0024] [Diagram 5] 1 is a flowchart of an exemplary process for determining a motion period, according to some embodiments.

[0025] [Figure 6] 1 is an exemplary plot of vertical acceleration components over a period of time in accordance with some embodiments.

[0026] [Figure 7] 4 illustrates various bandpass filter gains that may be applied to vertical acceleration data according to some embodiments.

[0027] [Figure 8]4 illustrates the output of various bandpass filters according to some embodiments.

[0028] [Figure 9] 9 is a plot of the output of the bandpass filter of FIG. 8 with maximum amplitude.

[0029] [Figure 10] 1 is a flowchart of an example process for determining cross-covariance vectors indicative of a direction orthogonal to a direction of motion, according to some embodiments.

[0030] [Figure 11] 4 is an example of a bandpass filter that may be applied to an acceleration component according to some embodiments.

[0031] [Figure 12] FIG. 1 shows a block diagram illustrating example components of an apparatus capable of implementing various aspects of the disclosure.

[0032] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] It may be useful to determine the direction of motion of a user engaged in a motor activity such as walking or running. For example, the direction of motion may be useful to present content in a manner that is dependent on the direction of motion. This content may include visual content (e.g., when the user is viewing content presented by a virtual reality (VR) or augmented reality (AR) headset, etc.), audio content, etc. As a more specific example, some audio content (e.g., music, podcasts, radio shows, movie soundtracks, etc.) may involve spatial components such that the audio content is rendered (e.g., by headphones, speakers, etc.) such that components of the audio content are perceived to be in different spatial locations relative to a listener. When a user is moving, it may be difficult to render the audio content in a manner that is faithful to the creator's intent without knowing the user's direction of motion. As an example, in cases where certain components of the audio content (e.g., speakers, musical features, etc.) should be positioned so that they are centered relative to the user, it may be difficult to accurately center the audio components without knowledge of the direction of motion while the user is moving (e.g., walking, running, etc.). Thus, it can be useful to determine the direction of motion of a moving user.

[0034] However, accurately determining the motion direction can be difficult. For example, some conventional techniques may utilize motion sensors, which may include accelerometers, gyroscopes, and / or magnetometers, on a device worn by a user, e.g., on the user's head. By way of example, such motion sensors may be located in or on headphones, earphones, glasses, AR / VR headsets, etc., worn on the user's head. In such cases, if a user is moving in a particular motion direction (e.g., walking, running, etc.), but looking around in one or more directions other than the motion direction, an estimation of the motion direction may be inaccurate due to motion sensor information collected with respect to the user's head moving in a direction other than the motion direction. If audio content is to be rendered relative to the center of the user (e.g., the user's torso), this may result in artifacts in spatial rendering, e.g., when the user looks around while walking.

[0035] Disclosed herein are systems, methods, and media for determining motion direction. The techniques described herein allow the motion direction to be determined regardless of head orientation, such that the motion direction of a moving user can be accurately determined even if the user's head is pointed in a direction other than the motion direction. The techniques can be implemented regardless of where the motion sensors (e.g., accelerometers, gyroscopes, and / or magnetometers) are located, for example, in or on a head-worn device, in or on a device carried by the user in his or her pocket, etc.

[0036] In some implementations, the motion direction may be determined by obtaining user acceleration data of a user (e.g., a user engaged in an athletic activity). Based on the user acceleration data, a movement period may be determined, indicating the duration between two successive movements of the user. As an example, if the athletic activity being performed by the user is walking or running, the movement period may correspond to the duration between two successive steps taken by the user while walking or running. The motion direction may then be determined based on a direction of acceleration orthogonal to the motion direction, the orthogonal direction corresponding to a direction of acceleration in which at least a portion of the user acceleration data is anti-periodic across the motion direction. It should be noted that as used herein, "direction" generally refers to a vector associated with an angle corresponding to an axis of a given coordinate system.

[0037] As an example, if the athletic activity is walking or running, the vertical component of acceleration (e.g., may correspond to the user's head or other body part moving up and down) may be substantially periodic with respect to the motion period. However, there may be a horizontal component of acceleration, which may substantially correspond to the lateral movement of the user's head to the left or right during the athletic activity, and is substantially anti-periodic with respect to the motion period. In other words, the vertical component of acceleration may undergo a complete cycle over the motion period, while the horizontal component of acceleration may undergo only a half cycle over the motion period (e.g., corresponding to the head movement to the left or right). It should be noted that, as used herein, the term "anti-periodic" may be understood to mean that the cross-correlation between a signal and a signal delayed by the determined motion period is negative. That is, the motion direction may be considered as a direction perpendicular to the direction in which the periodic acceleration is maximum, or in other words, the direction in which the anti-periodic acceleration is maximum. Additionally or alternatively, "counter-periodic" may be understood to mean that both major and minor movements (e.g., corresponding to head movements to the left and right, respectively) occur within a time interval corresponding to a movement period.

[0038] It should be noted that, although athletic activities are generally described herein using examples relating to walking or running, any suitable athletic activity involving periodic body movements may be considered. Other examples of athletic activities include skipping, jogging, cycling, climbing, skating, cross-country skiing, etc. Furthermore, it should be noted that, although the techniques described herein generally describe the vertical component of acceleration as being periodic for a given motion cycle and the horizontal component of acceleration as being anti-periodic for the motion cycle, in some cases this may be reversed. For example, in some cases, the horizontal component of acceleration may be periodic for the motion cycle depending on the athletic activity being considered.

[0039] Further, it should be noted that in some implementations, after determining the direction of motion, content may be rendered based on the direction of motion. For example, audio content may be rendered such that at least a portion of the audio content is perceived as being substantially aligned and positioned with the estimated orientation of the user's torso while the user is moving in the direction of motion. In some embodiments, the rendered audio content may be played, for example, through headphones, speakers, etc.

[0040] FIG. 1 is a diagram illustrating various axes and vectors associated with an exemplary use motion. As shown, a subject 105 is walking in a direction 102. Note that in some implementations, the subject may be wearing a device having one or more sensors configured to detect movement, orientation, acceleration in various axes, etc. For example, the subject 105 is wearing a head-mounted device that includes a head motion tracker 103. The head motion tracker 103 is configured to be connected to a coordinate system (X F ,Y F ,Z F ) where X is the orientation of the subject's head 104 relative to a stable reference frame formed by F Axis 111 and Y F Axis 112 defines a horizontal stable reference system, ZF The axis (not shown) points up.

[0041] In the example shown in Figure 1, subject 105 has rotated head 104 to face left in looking direction 101, e.g., walking direction 102. Walking direction 102 is shown as projection axis 122 projected onto a horizontal stable reference frame. Looking direction 101 is shown as projection axis 121 projected onto a horizontal stable reference frame.

[0042] Angle α is the angle 130 between axis 121 and axis 122, and is proportional to the direction in which the subject is looking (α look ) and the direction in which the subject is walking (α move ) The direction 123 represents a direction perpendicular to the walking direction 102.

[0043] The angle α may change over time even if the subject's direction of motion remains constant, e.g., due to the subject looking around while walking in a generally constant direction. In some implementations, the techniques described herein may effectively determine α, and thus a direction of motion that is unaffected by the subject's looking direction. Because the acceleration data may be collected using one or more accelerometers located in or on a head-mounted device (e.g., a head-up display, headphones, etc.), the acceleration data may include acceleration artifacts that are typically associated with the subject's head motion. Thus, the techniques described herein may allow the direction of motion to be identified regardless of the looking direction.

[0044] In some implementations, the motion direction may be identified based on the user acceleration data. For example, the user acceleration data may include multiple components, such as a vertical component (e.g., corresponding to up-down motion), a forward component (e.g., corresponding to front-back motion), and / or a lateral component (e.g., corresponding to side-to-side motion). Note that in some cases, the acceleration data for the first component may be periodic with respect to a motion period that defines the duration between two successive motions (e.g., a step period that defines the time during two successive steps). For example, the acceleration data associated with the vertical component (e.g., up-down) may be periodic with respect to a motion period when the user's head moves up and down while walking or running. In some cases, the acceleration data for the second component may be anti-periodic with respect to the motion period. For example, the acceleration data associated with the lateral component (e.g., corresponding to side-to-side motion) may be anti-periodic with respect to the motion period.

[0045] 2 illustrates a plot of acceleration data associated with a vertical component and acceleration data associated with a lateral component obtained during a walking activity. As shown, a curve 202 is plotted against the vertical component, i.e., up and down (e.g., Z F 2 represents user acceleration data associated with a lateral (e.g., side-to-side) component. Note that curve 202 is periodic with a period corresponding to a motion period (e.g., the time between two successive steps). Curve 204 represents user acceleration data associated with a lateral (e.g., side-to-side) component. Note that curve 204 is anti-periodic with respect to the motion period. In particular, note that if curve 204 is delayed by a motion period (e.g., the period for which curve 202 is periodic), then a positive peak in curve 204 will have a corresponding negative peak in the delayed version of curve 204 (and vice versa), where a positive peak represents a shift of the user's head to the left during a normal walking pattern, and a negative peak represents a shift of the user's head to the right.

[0046] In some implementations, a motion direction associated with a user's athletic activity may be determined using user acceleration data. The acceleration data may be obtained using one or more accelerometers, for example, located in a device or on a surface carried or worn by the user. For example, the one or more accelerometers may be associated with a mobile phone in a user's pocket or located in or on a wearable device worn by the user (e.g., headphones, earphones, smart glasses, head-up display, smart watch, a mobile phone worn on an armband or waistband, etc.). In some implementations, the user acceleration data may be utilized in conjunction with user head orientation information, as described below in conjunction with FIG. 4. In some implementations, the user acceleration data may be used to determine a motion period associated with the user's athletic activity. A motion period may indicate a duration between two successive movements associated with the athletic activity, e.g., two successive steps associated with a walking or running activity. The motion period may then be utilized to determine a motion direction. For example, the motion direction may be determined based on a direction of acceleration orthogonal to the motion direction, where at least a portion of the user acceleration data is anti-periodic over a time period corresponding to the motion period. In some implementations, a signal or portion of data may be considered anti-periodic if the cross-correlation of the signal or portion of data with a version of the signal or portion of data delayed by a motion period is negative. Additionally or alternatively, in some implementations, a signal or portion of data may be considered anti-periodic if the signal or portion of data includes local maxima and minima that occur after a subsequent time interval corresponding to motor activity.

[0047] 3 is a flowchart of an example process 300 for determining a motion direction associated with a user's athletic activity. In some implementations, the blocks of process 300 may be performed by a processor or a control system, such as the control system shown in FIG. 12 and described below in connection with FIG. 12. In some embodiments, the blocks of process 300 may be performed in an order other than that shown in FIG. 3. In some implementations, two or more blocks of process 300 may be performed substantially in parallel. In some implementations, one or more blocks of process 300 may be omitted.

[0048] Process 300 can begin at 302 by acquiring user acceleration data, which is associated with a user athletic activity. The athletic activity can include any suitable type of activity that includes acceleration along at least one axis that is periodic as a function of time. Examples of athletic activities include walking, running, etc. The user acceleration data can be acquired from one or more accelerometers located in or on a device worn or carried by the user, such as a mobile phone, a wearable device (e.g., headphones, smart glasses, head-up display, AR / VR glasses, etc.). Note that in some implementations, the user acceleration data can be processed prior to use. For example, in some embodiments, the user acceleration data can be transformed from a coordinate system centered on the user's head or body to a fixed coordinate system, as shown in and described below in connection with FIG. 4.

[0049] At 304, process 300 may determine a motion period associated with the user's athletic activity. The motion period may indicate the duration between successive movements of the athletic activity. For example, if the athletic activity is a walking or running activity, the motion period may indicate the time period between successive steps taken by the user during the walking or running activity. An exemplary technique for determining the motion period is shown in and described below in connection with FIG. 5.

[0050] At 306, process 300 can determine a motion direction corresponding to the athletic activity based on a direction of acceleration orthogonal to the motion direction in which at least a portion of the user acceleration data is anti-periodic over a time period corresponding to a motion period. For example, the motion direction may be a direction orthogonal to the direction of acceleration orthogonal to the motion direction that is closest to the user's looking direction (e.g., the looking direction is identified based at least in part on the user head orientation data). An exemplary technique for determining the motion direction is shown in and described below in connection with FIG. 10.

[0051] In some implementations, the user acceleration data may be transformed from a user-centered coordinate system to a fixed coordinate system (e.g., fixed with respect to the external environment, the ground, etc.). For example, the user acceleration data may be transformed from a coordinate system centered on the user's head (e.g., as shown and described above in connection with FIG. 1 ) based on acceleration data and / or orientation data obtained from sensors located in a head-worn device or on a surface (e.g., as shown and described above in connection with FIG. 1 ) to a coordinate system centered on the user's head (e.g., as shown and described above in connection with FIG. 1 ). F , Y F , Z F The user acceleration data may be transformed to a fixed coordinate system (such as that defined by the axial and axial axes). This may effectively allow the user acceleration data to be used to determine the direction of motion regardless of the orientation of the user's head (e.g., when the user is looking around while moving). In some implementations, the transformation of the user acceleration data from the user-centered coordinate system to the fixed coordinate system may be performed using the user head orientation data. The head orientation data may be obtained, for example, from one or more gyroscopes and / or magnetometers located in or on a wearable device worn on the user's head. For example, the one or more gyroscopes and / or magnetometers may be located in or on headphones, an AR / VR headset, smart glasses, etc.

[0052] 4 is a flowchart of an example process 400 for transforming user acceleration data from a user-centered coordinate system to a fixed coordinate system, according to some embodiments. In some implementations, the blocks of process 400 may be performed by a processor or a control system, such as the control system shown in and described below in connection with FIG. 12. In some embodiments, the blocks of process 400 may be performed in an order other than that shown in FIG. 4. In some implementations, two or more blocks of process 400 may be performed substantially in parallel. In some implementations, one or more blocks of process 400 may be omitted.

[0053] The process 400 may begin at 402 by obtaining user acceleration data relative to a user-centered coordinate system. The user-centered coordinate system may be a coordinate system (X U ,Y U ,Z U In some implementations, the user-centered coordinate system may be represented by, for example, Z U The axis points upward from the user's head, U The axis is oriented forward and backward relative to the user's eyes, and the Y U The axes may be aligned with the user's head so that they are pointing left and right to the user's ears. If the user acceleration data is measured in G (e.g., m / sec scaled by 1 / 9.8), 2 (equivalent to a measurement at ), the user acceleration data is, for a given sample k,

number

[0054] At 404, process 400 may obtain user orientation data. The user orientation data may be obtained from an orientation tracking device, which may be located, for example, in a wearable device worn on the user's head or on a surface. The user orientation data may be obtained using one or more gyroscopes and / or magnetometers (e.g., such as those located in or on the orientation tracking device 103 of FIG. 1). The user orientation data may indicate an orientation of the user's head relative to a user-centric system. The user orientation data may be represented by a matrix M UF (k), where the elements of the matrix indicate the user orientation data for a given sample time k with respect to the axes of the user-centered coordinate system. UF (k) may be a 3 by 3 matrix. Note that the orientation data may be specified in quaternions, Euler angles, and / or any other suitable format.

[0055] At 406, process 400 converts the user acceleration data from a user-centered coordinate system to a fixed coordinate system (e.g., the X coordinate system shown in FIG. 1 and described above in connection with FIG. 1). F Axis, Y F axis, and Z F The user acceleration data relative to a fixed coordinate system is generally referred to herein as Acc F (k). The transformation from the user-centered coordinate system to the fixed coordinate system may be performed using the user orientation data obtained in block 404. For example, in some implementations, the product of the user orientation data and the user acceleration data with respect to the user-centered coordinate system may correspond to the user acceleration data with respect to the fixed coordinate system. As a more specific example, in some implementations, the user acceleration data with respect to the fixed coordinate system may be

number

[0056] As described above, in some implementations, the motion direction may be determined based at least in part on a motion period associated with the motion activity. For example, the motion direction may be determined based on a direction of acceleration orthogonal to the motion direction, the orthogonal direction being a direction of acceleration in which at least a portion of the acceleration data is anti-periodic over a motion period. In some implementations, the motion period may be determined by providing components of the user acceleration data (which may be user acceleration data relative to a fixed coordinate system, as described above in connection with FIG. 4) to one or more band pass filters, each having a different center frequency. The outputs of the band pass filters may then be used to determine an initial estimate of the motion frequency. For example, the initial estimate of the motion frequency may correspond to a center frequency of the band pass filter that produces a maximum amplitude output. A motion period may then be determined based on the initial estimate of the motion frequency. For example, the motion period may be determined as the duration between two successive cycles of the output of the band pass filter that produces a maximum amplitude output. As a more specific example, the motion period may be determined as the duration between successive zero crossings of the output of the band pass filter that produces a maximum amplitude output.

[0057] 5 is a flowchart of an example process 500 for determining a motion period associated with a user activity, according to some embodiments. In some implementations, the blocks of process 500 may be performed by a control system or processor, such as the control system or processor shown in and described below in connection with FIG. 12. In some embodiments, the blocks of process 500 may be performed in an order other than that shown in FIG. 5. In some implementations, two or more blocks of process 500 may be performed substantially in parallel. In some implementations, one or more blocks of process 500 may be omitted.

[0058] The process 500 may begin at 502 by acquiring user acceleration data, the user acceleration data including vertical and horizontal components of acceleration. For example, the vertical component of acceleration may correspond to acceleration data with respect to the Z axis, and the horizontal component of acceleration may correspond to acceleration data with respect to the X and Y axes. FIG. 6 illustrates an example 600 of vertical components of user acceleration data acquired during a walking activity, according to some embodiments. Similar to that described above in connection with FIGS. 3 and 4, the user acceleration data may be acquired using one or more accelerometers disposed within or on a device worn or carried by the user. The user acceleration data may be with respect to a user-centered coordinate system or may be transformed to a fixed coordinate system (e.g., as shown and described above in connection with FIG. 4).

[0059] At 504, process 500 may apply one or more band pass filters to the acceleration data associated with the vertical component of the user acceleration data. Any suitable number of band pass filters may be used, such as 1, 2, 5, 10, 20, etc. Each band pass filter of the one or more band pass filters may have a different center frequency. In some implementations, the center frequency may be in a range of approximately 1 Hz to 3 Hz. Example center frequencies include 1 Hz, 1.05 Hz, 1.2 Hz, 1.23 Hz, 1.5 Hz, 1.8 Hz, 1.89 Hz, 2 Hz, 2.34 Hz, 2.7 Hz, 2.89 Hz, 3 Hz, etc. In examples where two or more band pass filters are used, the center frequencies of the two or more band pass filters may be linearly distributed (e.g., separated by a fixed frequency difference) or may not be linearly distributed. In some implementations, the gains may be different for band pass filters having different center frequencies. For example, in some embodiments, the gain may be relatively higher for a bandpass filter having a relatively low center frequency than for a bandpass filter having a relatively high center frequency.

[0060] Referring to Figure 7, an example of five bandpass filters is shown according to some embodiments. As shown, the five bandpass filters (702, 704, 706, 708, and 710) have center frequencies in the range of about 1.2 Hz to 2.2 Hz. In particular, the five bandpass filters shown in Figure 7 have center frequencies of 1.2 Hz, 1.4 Hz, 1.62 Hz, 1.89 Hz, and 2.2 Hz, respectively.

[0061] 5, at 506, the process 500 can identify an initial prediction of the motion frequency based on the output of one or more bandpass filters. Note that the motion frequency may be considered as the inverse of the motion period. Thus, the initial prediction of the motion frequency may be considered as the inverse of the initial prediction of the motion period. In some implementations, the process 500 can identify the initial prediction of the motion frequency as the center frequency associated with the bandpass filter having the highest amplitude output.

[0062] Referring to Figure 8, the outputs of the five bandpass filters shown in Figure 7 are shown in accordance with some embodiments. The outputs (802, 804, 806, 808, and 810) are generated in response to providing the vertical component of the user acceleration data shown in Figure 6 provided to each bandpass filter. Note that the output 804 of the bandpass filter having a center frequency of 1.89 Hz produces the highest amplitude output. Thus, for the particular example of the user acceleration data shown in Figure 6 and the bandpass filters shown in Figure 7, 1.89 Hz may be considered an initial estimate of the movement frequency.

[0063] Referring back to FIG. 5, it should be noted that in some cases, for a given vertical component of acceleration data, the output of two bandpass frequencies, one with a center frequency that is an integer multiple of the other, may produce similar magnitude outputs. As an example, if the motion frequency is 1.2 Hz, bandpass filters with center frequencies of 1.2 Hz and 2.4 Hz may produce similar outputs. In some implementations, process 500 may identify the motion frequency based on the output of the one or more bandpass filters, with a bias toward the lower center frequency. For example, referring to the above example of a motion frequency of 1.2 Hz, process 500 may select an initial prediction of the motion frequency to be 1.2 Hz instead of 2.4 Hz, due to 1.2 Hz being lower than 2.4 Hz.

[0064] It should be noted that in some implementations, the process 500 may be configured to determine whether motor activity including periodic motion is occurring based on the output of one or more band pass filters. For example, the process 500 may determine that motor activity including periodic motion is occurring in response to the amplitude of at least one band pass filter exceeding a predetermined threshold. It should be noted that in some embodiments, the process 500 may terminate in response to determining that motor activity with periodic motion is not occurring. Additionally, in some embodiments, the direction of motion may not be determined.

[0065] At 508, the process 500 can identify a motion period based on the initial prediction of the motion frequency. In some implementations, the process 500 can identify the motion period as the duration between cycles of the output of a bandpass filter associated with the motion frequency (e.g., the bandpass filter that produces the highest amplitude output). In other words, the determination of the motion period can be considered a refinement of the initial prediction of the motion frequency.

[0066] Referring to FIG. 9, a close-up view of the output of the bandpass filter having a center frequency of 1.89 Hz from FIG. 8 is shown. Recall that the bandpass filter having a center frequency of 1.89 Hz produced the maximum amplitude output, as shown in connection with FIG. 8 and described above. As shown, a motion period 902 can be considered as the time between successive cycles of the bandpass filter output. More specifically, as shown in FIG. 9, a motion period may be the time between successive zero crossings of the filter output. In the example shown in FIG. 9, the motion period 902 (e.g., the time between successive zero crossings) is 0.549 seconds. Note that a motion period of 0.549 seconds corresponds to a motion frequency of 1.82 Hz, which is relatively close to the initial prediction of a motion frequency of 1.89 Hz.

[0067] As mentioned above, in some implementations, the direction of motion may be determined based on a direction of acceleration orthogonal to the direction of motion, the direction of acceleration orthogonal to the direction of motion having a component of acceleration data that is anti-periodic with respect to the motion period. In some cases, the component of acceleration data that is anti-periodic may be a horizontal component of acceleration (which may include acceleration data with respect to the X-axis and Y-axis). In some implementations, the direction of motion may be determined by determining the cross covariance of the component of acceleration that is anti-periodic with respect to the motion period and the anti-periodic acceleration component delayed by the motion period. The direction of motion may then be determined based on the cross covariance. In some embodiments, the cross covariance is determined based on the direction of motion (generally referred to herein as "α move In some implementations, the value of β may then be halved, and α may then be used to determine an angle (generally referred to herein as "β") that is twice the angle associated with the head orientation direction (referred to as "β"), in which the angle is half the value of β and is closest to the head orientation direction (to disambiguate between β / 2 and angles that are π radians away from β / 2). move By selecting

[0068] 10 is a flowchart of an example process 1000 for determining a motion direction based on user acceleration data and a motion cycle. In some implementations, the blocks of process 1000 may be performed by a control system or processor, such as the control system shown in FIG. 12 and described below in connection with FIG. 12. In some embodiments, the blocks of process 1000 may be performed in an order other than that shown in FIG. 10. In some implementations, two or more blocks of process 1000 may be performed substantially in parallel. In some implementations, one or more blocks of process 1000 may be omitted.

[0069] Process 1000 may begin at 1002 by obtaining a horizontal component of user acceleration data and a motion cycle associated with the user's athletic activity. The motion cycle may be obtained based on acceleration data associated with a vertical component, as shown in connection with FIG. 5 and described above. The horizontal component of the user acceleration data may be represented according to a two-dimensional basis formed by an X-axis and a Y-axis. In other words, in some embodiments, the horizontal component of the user acceleration data may correspond generally to the user acceleration data relative to the X-axis and the Y-axis. When the user acceleration data is transformed into a fixed frame coordinate system (e.g., as shown in connection with FIG. 4 and described above), the horizontal component of the user acceleration data (generally referred to herein as Acc H ) is expressed as, for a given sample k,

number

[0070] At 1004, the process 1000 may optionally apply a filter to the horizontal component of the user acceleration data. For example, the filter may be a bandpass filter that effectively removes low frequency acceleration components and attenuates high frequency components. As a more specific example, the bandpass filter may be configured to attenuate low frequency components below about 0.5 Hz, below about 1 Hz, below about 5 Hz, etc. Additionally or alternatively, the bandpass filter may be configured to attenuate high frequency components above about 8 Hz, above about 10 Hz, above about 20 Hz, etc. An example 1100 of such a filter is shown in FIG. 11. The filter shown in FIG. 11 is constructed as a combination of a second order Butterworth high pass filter with a cutoff frequency of 2 Hz and a first order Butterworth low pass filter with a cutoff frequency of 1 Hz and an additional gain of 3.0. It should be noted, however, that a suitable bandpass filter may be constructed in any suitable manner.

[0071] Referring back to FIG. 10, at 1006, process 1000 may determine a cross-covariance between a horizontal component of the user acceleration data and a version of the horizontal component delayed by a motion period. The cross-covariance may indicate a direction orthogonal to a motion direction associated with the athletic activity. In some implementations, the delay of the horizontal component of the user acceleration data may be performed by shifting a sample k of the user acceleration data by a value p, where p represents the number of sample intervals associated with the determined motion period. For example, for each time sample k, a cross product vector may be formed from the X and Y acceleration components at that sample time and the X and Y acceleration components at a previous time delayed by a motion period from that sample time. By way of example, the product vector may be

number

[0072] In the above formula, the subscript indicates which element of the AccH vector is to be extracted. For example, the subscript "1" indicates that the element of the AccH vector related to the X-axis is to be extracted, and the subscript "2" indicates that the element of the AccH vector related to the Y-axis is to be extracted.

[0073] After determining the product vector C(k), the process 1000 may use the current product vector and the most recent past product vectors to determine a delay-offset cross-covariance vector, generally represented herein as X(k). In some implementations, more recent cross-product vectors (e.g., more recent values ​​of vector C(k)) may be weighted more heavily using weight w(j). For example, the cross-covariance vector may be

number

[0074] In the above equation, the weight w(j) may be selected to be higher for more recent cross product vectors of vector C. For example, the weight w(j) may be relatively higher for lower values ​​of j. In one example, the weight w(j) may be determined using an exponential function such that the value of the weight decays as a function of time or number of samples. As an example, w(j) may be

number

[0075] If an exponential function is used to determine the weights w(j), the cross-covariance vector X(k) may be determined iteratively. For example, using the example exponential function above, X(k) can be expressed as:

number

[0076] At 1008, process 1000 can utilize the cross-covariance to determine a motion direction associated with the athletic activity. For example, in some implementations, process 1000 can determine an angle that is twice the angle of the motion direction based on the cross-covariance vector X(k). For example, in some implementations, the angle that is twice the angle of the motion direction, generally represented herein as β, can be:

number

[0077] As before, in the above equation, the subscripts represent elements extracted from the cross-covariance vector X(k).

[0078] In some implementations, given an angle β, the process 1000 may calculate an angle β, generally referred to herein as α, by dividing the angle β by two. move This can be expressed as either pointing in front of the user or pointing behind the user, move Considering that this may lead to ambiguity in the direction of move may be selected as the angle that is half the value of β that is closest to the vector associated with the user's head orientation direction. Note that, as discussed above in connection with FIG. 4, the head orientation direction may be determined by an orientation tracking device (e.g., using one or more gyroscopes and / or magnetometers).

[0079] Alternatively, in some implementations, the value of β is determined based on the head orientation direction (typically, for a given sample k, α look For example, generally referred to herein as β diff The corrected value of β, called

number

[0080] The corrected value of β is thenwalk For example, α move teeth,

number

[0081] In the above formula, "round" represents a rounding operation to the nearest integer.

[0082] FIG. 12 is a block diagram illustrating examples of components of a device capable of implementing various aspects of the disclosure. As with other figures provided herein, the types and numbers of elements shown in FIG. 12 are provided merely as examples. Other implementations may include more, fewer, and / or different types and numbers of elements. According to some examples, device 1200 may be configured to perform at least some of the methods disclosed herein. In some implementations, device 1200 may be or include a television, one or more components of an audio system, a mobile device (such as a cellular phone), a laptop computer, a tablet device, a smart speaker, or another type of device.

[0083] According to some alternative implementations, apparatus 1200 may be or include a server. In some such examples, apparatus 1200 may be or include an encoder. Thus, in some cases, apparatus 1200 may be a device configured for use in an audio environment, such as a home audio environment, and in other cases, apparatus 1200 may be a device configured for use in the "cloud," e.g., a server.

[0084] In this example, the device 1200 includes an interface system 1205 and a control system 1210. The interface system 1205, in some implementations, may be configured to communicate with one or more other devices of an audio environment. The audio environment, in some examples, may be a home audio environment. In other examples, the audio environment may be another type of environment, such as an office environment, an automobile environment, a train environment, a street or sidewalk environment, a park environment, etc. The interface system 1205, in some implementations, may be configured to exchange control information and associated data with audio devices of the audio environment. The control information and associated data, in some examples, may relate to one or more software applications that the device 1200 is executing.

[0085] The interface system 1205, in some implementations, may be configured to receive or provide a content stream. The content stream may include audio data. The audio data may include, but is not limited to, an audio signal. In some instances, the audio data may include spatial data, such as channel data and / or spatial metadata. In some examples, the content stream may include video data and audio data corresponding to the video data.

[0086] The interface system 1205 may include one or more network interfaces and / or one or more external device interfaces (such as one or more universal serial bus (USB) interfaces). According to some implementations, the interface system 1205 may include one or more wireless interfaces. The interface system 1205 may include one or more devices for implementing a user interface, such as one or more microphones, one or more speakers, a display system, a touch sensor system, and / or a gesture sensor system. In some examples, the interface system 1205 may include one or more interfaces between the control system 1210 and a memory system, such as the optional memory system 1215 shown in FIG. 12. However, the control system 1210 may include a memory system in some cases. The interface system 1205 may be configured to receive input from one or more microphones in the environment in some implementations.

[0087] The control system 1210 may include, for example, a general-purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, and / or discrete hardware components.

[0088] In some implementations, the control system 1210 may reside in more than one device. For example, in some implementations, a portion of the control system 1210 may reside in a device in one of the environments illustrated herein, and another portion of the control system 1210 may reside in a device outside the environment, such as a server, a mobile device (e.g., a smartphone or tablet computer), etc. In other examples, a portion of the control system 1210 may reside in a device in one environment, and another portion of the control system 1210 may reside in one or more other devices of the environment. For example, a portion of the control system 1210 may reside in a device implementing a cloud-based service, such as a server, and another portion of the control system 1210 may reside in another device implementing a cloud-based service, such as another server, a memory device, etc. The interface system 1205 may also reside in more than one device, in some examples.

[0089] In some implementations, the control system 1210 may be configured to at least partially perform the methods disclosed herein. According to some examples, the control system 1210 may be configured to implement a method for determining a direction of motion, a method for determining a direction of motion based on a direction orthogonal to the direction of motion, etc.

[0090] Some or all of the methods described herein may be performed by one or more devices according to instructions (e.g., software) stored in one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, and the like. The one or more non-transitory media may be present, for example, in any memory system 1215 and / or control system 1210 shown in FIG. 12. Thus, various innovative aspects of the subject matter described in this disclosure may be implemented in one or more non-transitory media having software stored thereon. The software may include, for example, instructions for determining a direction of motion, instructions for determining a direction of motion based on a direction orthogonal to the direction of motion, and the like. The software may be executable by one or more components of a control system, such as, for example, the control system 1210 of FIG. 12.

[0091] In some examples, the device 1200 may include an optional microphone system 1220 shown in FIG. 12. The optional microphone system 1220 may include one or more microphones. In some implementations, one or more of the microphones may be part of or associated with another device, such as a speaker of a speaker system, a smart audio device, etc. In some examples, the device 1200 may not include the microphone system 1220. However, in some such implementations, the device 1200 may still be configured to receive microphone data for one or more microphones in the audio environment via the interface system 710. In some such implementations, a cloud-based implementation of the device 1200 may be configured to receive microphone data, or noise metrics at least partially corresponding to the microphone data, from one or more microphones in the audio environment via the interface system 1210.

[0092] According to some implementations, the device 1200 may include an optional loudspeaker system 1225 shown in FIG. 12. The optional loudspeaker system 1225 may include one or more loudspeakers, which may also be referred to herein as a "speaker" or more generally as an "audio reproduction transducer." In some examples (e.g., cloud-based implementations), the device 1200 may not include the loudspeaker system 1225. In some implementations, the device 1200 may include headphones. The headphones may be connected or coupled to the device 1200 via a headphone jack or via a wireless connection (e.g., BLUETOOTH®).

[0093] Some aspects of the disclosure include systems or devices configured (e.g., programmed) to perform one or more examples of the disclosed methods, and tangible computer-readable media (e.g., disks) storing code for implementing one or more examples of the disclosed methods or steps thereof. For example, some disclosed systems may be or include a programmable general-purpose processor, digital signal processor, or microprocessor programmed with software or firmware and / or otherwise configured to perform any of a variety of operations on data, including embodiments of the disclosed methods or steps thereof. Such a general-purpose processor may be or include a computer system that includes an input device, a memory, and a processing subsystem programmed (and / or otherwise configured) to perform one or more examples of the disclosed methods (or steps thereof) in response to asserted data.

[0094] Some embodiments may be implemented as a configurable (e.g., programmable) digital signal processor (DSP) configured (e.g., programmed and otherwise configured) to perform the necessary processing on the audio signal(s), including performing one or more examples of the disclosed methods. Alternatively, embodiments of the disclosed system (or elements thereof) may be implemented as a general-purpose processor (e.g., a personal computer (PC) or other computer system or microprocessor, which may include input devices and memory) that is programmed with software or firmware and / or otherwise configured to perform any of the various operations, including one or more examples of the disclosed methods. Alternatively, elements of some embodiments of the inventive system are implemented as a general-purpose processor or DSP configured (e.g., programmed) to perform one or more examples of the disclosed methods, the system also including other elements (e.g., one or more loudspeakers and / or one or more microphones). A general-purpose processor configured to perform one or more examples of the disclosed methods may be coupled to an input device (e.g., a mouse and / or a keyboard), a memory, and a display device.

[0095] Another aspect of the disclosure is a computer-readable medium (e.g., a disk or other tangible storage medium) storing code (e.g., executable code) for performing one or more examples of the disclosed methods or steps thereof.

[0096] While particular embodiments of and applications of the present disclosure have been described herein, it will be apparent to those skilled in the art that many variations on the embodiments and applications described herein are possible without departing from the scope of the present disclosure as described and claimed herein. While certain forms of the present disclosure have been illustrated and described, it will be understood that the present disclosure should not be limited to the specific embodiments described and illustrated, or to the specific manner described.

Claims

1. 1. A method for determining a direction of motion of a user, the method comprising: obtaining user acceleration data associated with a user using a control system; using the control system to determine a motion period associated with a user's athletic activity using the user acceleration data, the motion period being indicative of a duration between two successive movements by a user; and using the control system to determine a direction of motion corresponding to the athletic activity using the user acceleration data based on a direction of acceleration orthogonal to the direction of motion such that at least a portion of the user acceleration data is anti-periodic over a period of time corresponding to the motion period. method.

2. 2. The method of claim 1 , wherein the at least the portion of the user acceleration data is anti-periodic over the time period corresponding to the motion period, such that a cross-correlation between the at least the portion of the user acceleration data and a version of the at least the portion of the user acceleration data delayed by the motion period is negative.

3. The method of claim 1 , wherein the at least the portion of the user acceleration data that is counter-periodic includes local maxima and minima that occur after subsequent time intervals corresponding to the athletic activity.

4. The method of claim 1 , wherein the athletic activity includes walking or running, and the direction of motion includes a direction in which a user is walking or running.

5. 2. The method of claim 1, further comprising transforming the user acceleration data from a user-centered coordinate system to a fixed coordinate system to generate transformed user acceleration data, and wherein the period of motion is determined using the transformed user acceleration data.

6. The method of claim 5 , wherein converting the user acceleration data includes using user head orientation data obtained from one or more gyroscopes.

7. The method of claim 6 , wherein the head orientation data is used to identify the direction of movement.

8. The method of claim 1 , wherein determining the period of motion includes using data associated with a vertical component of the user acceleration data.

9. Determining the period of motion includes: providing data associated with a vertical component of the user acceleration data to a plurality of narrow band filters, each narrow band filter of the plurality of narrow band filters being associated with a different frequency; generating a prediction of a motion frequency based on outputs of the plurality of narrow band filters, the prediction of the motion frequency being used to determine the motion period. The method according to claim 8.

10. 10. The method of claim 9, wherein using the prediction of the motion frequency to determine the motion period comprises identifying a period of an output of a selected narrow band filter of the plurality of narrow band filters, the selected narrow band filter of the plurality of narrow band filters being one that produces a maximum output signal.

11. The method of claim 1 , wherein the at least a portion of the user acceleration data that is counter-periodic over the period of time corresponding to the motion period includes one or more horizontal components of the user acceleration data.

12. 12. The method of claim 11 , wherein identifying a direction of acceleration orthogonal to the direction of motion comprises determining a cross-covariance vector corresponding to data associated with the one or more horizontal components of the user acceleration data and a version of data associated with the one or more horizontal components of the user acceleration data delayed by the motion period.

13. The cross-covariance vectors are: determining a cross product vector from two or more horizontal components of the user acceleration data and two or more horizontal components of the user acceleration data delayed by the motion period for each of a plurality of sample times; Combining two or more of the cross product vectors from two or more of the plurality of sample times to determine the cross-covariance vector. The method of claim 12, wherein the determined

14. 14. The method of claim 13, wherein the two or more of the cross product vectors are combined using a weighting, such that cross product vectors associated with more recent sample times are weighted more heavily than cross product vectors associated with less recent sample times.

15. The method of claim 14 , wherein the weighting comprises an exponential decay function.

16. The method of claim 11 , further comprising applying a band pass filter to the one or more horizontal components of the user acceleration data prior to identifying a direction of acceleration orthogonal to the direction of motion.

17. The direction of movement is: determining an intermediate angle corresponding to twice the angle associated with the direction of motion; Determining an angle associated with the direction of motion based on the intermediate angle The method of claim 1 , wherein the determined

18. Determining an angle associated with the direction of motion based on the intermediate angle comprises: halving the intermediate angle to determine candidate directions of motion; and selecting an angle associated with the motion direction as the candidate motion direction or as 180 degrees from the candidate motion direction based on head orientation data.

20. The method of claim 17.

19. Determining an angle associated with the direction of motion based on the intermediate angle comprises: determining a difference between the intermediate angle and an angle associated with an orientation of the user's head; determining an angle associated with the direction of motion based on the difference.

20. The method of claim 18.

20. The method of claim 1 , wherein the user acceleration data is obtained from one or more accelerometers disposed within or on a user device configured to be carried or worn by a user.

21. The method of claim 20 , wherein the user device comprises headphones.

22. 22. The method of claim 21, further comprising rendering audio content to be presented by the headphones based on the identified direction of movement.

23. 23. The method of claim 22, wherein the audio content is rendered to have a spatial perception centered around an identified direction of movement.

24. 24. Apparatus configured to implement the method of any one of claims 1 to 23.

25. 24. One or more non-transitory media having stored thereon software including instructions for controlling one or more devices to perform the method of any one of claims 1 to 23.

Images