Orienting sensor axial self-calibration
By calculating the mapping between headphone orientation sensors and calibrating the sensor axes, the spatial audio inaccuracy caused by headphone misalignment was solved, achieving more accurate recording of user head movements and audio rendering.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BOSE CORP
- Filing Date
- 2024-09-10
- Publication Date
- 2026-06-12
Smart Images
Figure CN122207269A_ABST
Abstract
Description
Cross-references to related applications
[0001] This application claims priority to U.S. nonprovisional patent application No. 18 / 465,691, filed on September 12, 2023, entitled "Orientation Sensor Axial Self-Calibration," the entire contents of which are incorporated herein by reference. Background Technology
[0002] This disclosure relates to an earphone configured to self-calibrate the axial alignment of its orientation sensor and a method for calibrating the axial alignment of an orientation sensor disposed within the earphone. Summary of the Invention
[0003] All examples and features mentioned below can be combined in any technically possible way.
[0004] According to one aspect, a pair of headphones with orientation sensor axial alignment self-calibration includes: a first earpiece housing a first orientation sensor, the first orientation sensor outputting a first orientation signal, wherein the first orientation signal represents the orientation of the first earpiece and is relative to a first orientation axis of the first orientation sensor; a second earpiece housing a second orientation sensor, the second orientation sensor outputting a second orientation signal, wherein the second orientation signal represents the orientation of the second earpiece and is relative to a second orientation axis of the second orientation sensor; and a controller configured to calculate a mapping between the first orientation sensor axis and the second orientation sensor axis based on the difference between the first orientation signal and the second orientation signal, wherein the controller is further configured to calibrate the first orientation axis based on the midpoint of the mapping and to calibrate the second orientation axis based on the inverse of the midpoint of the mapping, such that when the first earpiece and the second earpiece are worn antisymmetrically with respect to at least one mirror-symmetric plane of the user's head, at least one of the roll and yaw of the first orientation sensor axis and the second orientation sensor axis is more closely aligned with the axis of the user's head.
[0005] In one example, the mapping is calculated based on an adaptive algorithm.
[0006] In one example, the mapping is computed non-adaptively.
[0007] In one example, the first orientation sensor and the second orientation sensor are each inertial measurement units.
[0008] In one example, the first orientation sensor and the second orientation sensor each include at least one gyroscope sensor.
[0009] In one example, the first orientation sensor and the second orientation sensor are different sensor types, where the first orientation sensor is an accelerometer and gyroscope sensor, and the second orientation sensor is an accelerometer.
[0010] In one example, the controller is housed in at least one of the first or second earpieces.
[0011] In one example, the controller is also configured to render a spatialized audio signal based on a calibrated first orientation signal and a calibrated second orientation signal.
[0012] In one example, the spatialized audio signal is determined based on a spatialized audio algorithm that includes an interaural time difference parameter, wherein the controller is also configured to adjust the interaural time difference parameter based on a vector representing the distance between a first orientation sensor and a second orientation.
[0013] According to another aspect, a method for calibrating the axial alignment of an orientation sensor includes: receiving a first orientation signal representing the orientation of a first earpiece of an earphone pair, the first orientation signal being relative to a first orientation axis of a first orientation sensor; receiving a second orientation signal representing the orientation of a second earpiece of the earphone pair, the second orientation signal being relative to a second orientation axis of a second orientation sensor; calculating a mapping between the first orientation sensor axis and the second orientation sensor axis based on a difference between the first orientation signal and the second orientation signal; calibrating the first orientation axis based on the midpoint of the mapping; and calibrating the second orientation axis based on the inverse of the midpoint of the mapping, such that when a user wears the first earpiece and the second earpiece antisymmetrically about at least one mirror-symmetric plane of the user's head, at least one of the roll and yaw of the first orientation sensor axis and the second orientation sensor axis is more closely aligned with the axis of the user's head.
[0014] In one example, the mapping is calculated based on an adaptive algorithm.
[0015] In one example, the mapping is computed non-adaptively.
[0016] In one example, the first orientation sensor and the second orientation sensor are each inertial measurement units.
[0017] In one example, the first orientation sensor and the second orientation sensor each include at least one angular rate sensor.
[0018] In one example, at least one angular rate sensor is a gyroscope sensor.
[0019] In one example, the first orientation sensor and the second orientation sensor are different sensor types, where the first orientation sensor is an accelerometer and gyroscope sensor, and the second orientation sensor is an accelerometer.
[0020] In one example, the method further includes: rendering a spatialized audio signal based on a calibrated first orientation signal and a calibrated second orientation signal, wherein the spatialized audio signal is determined according to a spatialized audio algorithm including an interaural time difference parameter; and adjusting the interaural time difference parameter based on a vector representing the distance between the first orientation sensor and the second orientation.
[0021] According to another aspect, at least one non-transitory storage medium stores program code for execution on at least one processor, the program code, when executed, calibrating the axial alignment of an orientation sensor pair, comprising: receiving a first orientation signal representing the orientation of a first earpiece of the earphone pair, the first orientation signal being relative to a first orientation axis of a first orientation sensor; receiving a second orientation signal representing the orientation of a second earpiece of the earphone pair, the second orientation signal being relative to a second orientation axis of a second orientation sensor; calculating a mapping between the first orientation sensor axis and the second orientation sensor axis based on the difference between the first orientation signal and the second orientation signal; calibrating the first orientation axis based on the midpoint of the mapping; and calibrating the second orientation axis based on the inverse of the midpoint of the mapping, such that when the first earpiece and the second earpiece are worn antisymmetrically about at least one mirror-symmetric plane of the user's head, at least one of the roll and yaw of the first orientation sensor axis and the second orientation sensor axis is more closely aligned with the axis of the user's head.
[0022] In one example, the mapping is calculated based on an adaptive algorithm.
[0023] In one example, the mapping is computed non-adaptively.
[0024] In one example, the first orientation sensor and the second orientation sensor are each inertial measurement units.
[0025] Details of one or more specific embodiments are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the specification, drawings, and claims. Attached Figure Description
[0026] In the accompanying drawings, the same reference numerals generally refer to the same parts in all different views. Furthermore, the drawings are not necessarily drawn to scale, and the focus is usually on illustrating the principles of various aspects.
[0027] Figure 1A A frontal view of the human head and its associated axes are depicted.
[0028] Figure 1B A top view of a human head and its associated axes are depicted.
[0029] Figure 2A A front view and associated axis of a pair of earbuds worn in an aligned orientation are depicted.
[0030] Figure 2B A front view and associated axis of a pair of in-ear headphones worn with the headphone body rolled inward are depicted.
[0031] Figure 2C A front view and associated axes of a pair of earbuds worn with the earbuds rolled outwards are depicted.
[0032] Figure 3A A top view and associated axes are depicted of a pair of earbuds worn in an aligned orientation.
[0033] Figure 3B A top view and associated axis of an in-ear headphone pair with inward yaw adjustment are depicted.
[0034] Figure 3C A top view and associated axis of a pair of earbuds with yaw-out adjustment are depicted.
[0035] Figure 4 A block diagram depicts a pair of headphones configured using axial self-calibration with an orientation sensor, based on an example.
[0036] Figure 5 A method for calibrating the axial alignment of an orientation sensor associated with an earphone pair, based on an example, is described. Detailed Implementation
[0037] Headphones are typically equipped with orientation sensors (e.g., inertial measurement units) that detect and report data to determine the orientation of the headphones. This information is used to provide, for example, spatialized audio—audio perceived by the user as originating from at least one location different from the actual location of the electroacoustic transducers within the headphones or earbuds. More specifically, the detected orientation changes of the headphones and earbuds correspond to changes in the orientation of the user's head. This information can be used to adjust the acoustic signals delivered to the user to simulate the acoustic signal changes that would occur if the acoustic signal originated from a virtual source. As a result, when the user's head rotates, the user perceives the audio as originating from a virtual source, rather than from the transducers located within the earbuds or headphones.
[0038] To accurately render spatialized audio, orientation sensors are used to precisely track changes in the user's head orientation. This means that the axis of the orientation sensor must be predictably mapped to the axis of the user's head. Figure 1A and Figure 1B The front and top views of the user's head are shown respectively. Figure 1A and Figure 1B The associated head axes are also shown; in this example, these head axes include the XH axis, YH axis, and ZH axis (in... Figure 1A In the view, XH extends out of the page, and... Figure 1B In the view, ZH extends out of the page. The XH-ZH plane, mirror-symmetric to the head, is also shown. Figure 2A The diagram depicts a pair of earbuds 202 and 204 (in this example, Bose QuietComfort Earbuds II) in an alignment orientation when worn by a user. In this orientation, the axes of the orientation sensors disposed in the earbuds 202 and 204 are aligned with the axis of the user's head. Specifically, the axes of the right earbud 204 (including axes XR, YR, and ZR) and the axes of the left earbud 202 (including axes XL, YL, and ZL) are aligned with the axes of the user's head (XH, YH, and ZH), which are also shown in this view for reference. Figure 3A A top view of the earbuds 202 and 204 in an aligned position is also shown.
[0039] However, users do not always wear in-ear headphones in the correct alignment. For example... Figure 2B As shown, users sometimes roll the earbuds 202 and 204 outwards, or as... Figure 2C As shown, roll the earbuds 202 and 204 inwards. Alternatively, users sometimes... Figure 3B As shown, yaw inward or as Figure 3C The image shows the headphones being yawed outwards. (Users may also adjust the orientation of the earbuds in more than one way, such as by rolling the earbuds inwards and yawing them outwards.) Although not shown, users may also adjust the pitch of the earbuds up or down. In each of these cases, at least two axes of the sensor are no longer aligned with the user's head axis. For example, if the user rolls the earbuds inwards or outwards away from their head... Figure 2A If the position is such that the Y-axis and Z-axis of the earbuds are no longer aligned with the YH-axis and ZH-axis of the user's head, then the position will also be affected. Similarly, if the user adjusts the yaw to move the earbuds away from the head, the position will also be affected. Figure 3A If the position is such that the X and Y axes of the earphone are no longer aligned with the XH and YH axes of the user's head.
[0040] This adjustment, away from the user's head axis, can cause orientation sensors to misinterpret the user's head movements, thus reducing the accuracy of the rendered spatialized audio. Movements of the user's head in one direction (such as pitch) will record changes across different axes (such as yaw or roll), resulting in an undesirable perception of the virtual source shifting in space.
[0041] However, it is worth noting that users typically use the same method in each ear (i.e., regarding...). Figure 1A and Figure 1B The earbuds are adjusted (anti-symmetrically) in the XH-ZH mirror plane shown. In other words, the adjustment of one earbud (202) is a mirror image of the adjustment of the other earbud (204). (For the purposes of this disclosure, anti-symmetry is synonymous with mirror symmetry.) Therefore, in Figure 3A and Figure 3B In the middle, the pitch of the earbuds 202 and 204 is antisymmetric about the XH-ZH mirror plane, whether tilted inwards or outwards. Similarly, in Figure 3B and Figure 3C In this context, regardless of whether the yaw of the earbuds 202 and 204 is adjusted inward or outward, the change is antisymmetric about the XH-ZH mirror plane.
[0042] Go to Figure 4 The diagram shows a block diagram of an earphone pair 400, which has an orientation sensor self-calibration system for adjusting the axis of the orientation sensor in a manner correcting for anti-symmetric misalignment. Earphone 400 includes a left earpiece 402 and a right earpiece 404. The left earpiece 402 includes a controller 406 in communication with an orientation sensor 408, which detects the orientation of the left earpiece 402 and outputs an orientation signal representing the detected orientation to the controller 406. The orientation signal output from the orientation sensor 408 relative to the axis of the orientation sensor 408 is provided to the controller 406. In other words, the orientation sensor 408 does not necessarily provide an absolute orientation, but rather a relative orientation, which is given by the change in the orthogonal axis of the orientation sensor as the orientation sensor 408 rotates in space. Similarly, the right earpiece 404 includes a controller 410 in communication with an orientation sensor 412, which detects an orientation signal representing the orientation of the right earpiece 404 and reports the orientation signal to the controller 410. The orientation signal output from the orientation sensor 412 relative to the axis of the orientation sensor 412 is provided to the controller 410.
[0043] exist Figure 4In this example, the left earpiece 402 and the right earpiece 404 receive audio signals from a source such as a mobile device 414 (although other suitable sources are conceivable). The audio signal is received at the left earpiece 402 via a wireless connection b1 (e.g., a Bluetooth connection) at transceiver 416, which provides the audio signal to controller 406. Transceiver 416 also relays the audio signal via wireless connection b2 to transceiver 418, which provides the audio signal to controller 410. (In an alternative example, transceivers 416 and 418 may each receive wireless signals directly from a source, rather than relaying the signal to another earpiece.) Controller 406 drives electroacoustic transducer 420 based on the audio signal received at transceiver 416, and controller 410 drives electroacoustic transducer 422 based on the audio signal received at transceiver 418. Controllers 406 and 410 can drive electroacoustic transducers 420 and 422 to provide spatialized acoustic signals to the user based on the orientation of the user's head detected by orientation sensors 408 and 412. Generally, the generation of spatialized audio from orientation signals is known, and therefore a detailed explanation is omitted here. Any suitable spatialized audio algorithm can be used.
[0044] For the purposes of this disclosure, the term "headphone" refers to any wearable device worn on a user's head that is easily adjustable in a manner that misaligns the axis of the orientation sensor with the axis of the user's head, including both strap-type and non-strap-type examples. Therefore, "headphone" includes form factors such as on-ear headphones, over-ear headphones, in-ear headphones, earbud headphones, and open-back headphones. Furthermore, for simplicity and to emphasize more relevant aspects of the headphone 400, [details omitted]. Figure 4 Certain features of the block diagram, such as, for example, battery, indicator LEDs, external buttons / inputs, feedback microphones, feedforward microphones, etc.
[0045] Furthermore, although a wireless connection to source 414 and a wireless connection between handsets 402 and 404 are described, wired connections may also be used in other examples. A wired connection may, for example, connect to mobile device 414, or a wired connection may simply connect handsets 402 and 404 together (such as within a neckband). Regarding the use of a wireless connection, any suitable wireless protocol may be employed. While Bluetooth or DECT is commonly the standard for wireless headphone connections, it is conceivable that other standards or proprietary standards may also be used. Transceivers 416 and 418 may be implemented as wireless modules of an appropriate proprietary standard. For example, transceivers 416 and 418 may each be implemented as a Bluetooth system-on-a-chip.
[0046] As will be described in detail below, in order to calibrate orientation sensors 408 and 412, a mapping between orientation sensors 408 and 412 can be determined based on the orientation signal. The axis of one orientation sensor (e.g., orientation sensor 408) can then be calibrated by rotating the axis of the orientation sensor to the midpoint of the mapping between orientation sensors 408 and 412. The axis of the other orientation sensor (e.g., orientation sensor 412) can be calibrated by rotating the axis inversely to the midpoint between orientation sensors 408 and 412. Therefore, when the user wears the earpieces 402 and 404 antisymmetrically about a mirror symmetry plane (i.e., the XH-ZH mirror symmetry plane of the user's head), at least one of the roll or yaw of the axis of orientation sensor 408 and the axis of orientation sensor 412 will be more closely aligned with the axis of the user's head. Simply put, by finding a rotation that balances the difference between the two orientation sensors 408 and 412 and rotating the axis of one orientation sensor frame in the opposite direction to the rotation of the other sensor frame, the axes "meet in the middle" and eliminate any mirror symmetry offset. (This rotation can be encoded in any suitable way, including matrices, quaternions, vectors, and angles.)
[0047] For the purposes of this disclosure, alignment of the sensor axis with the head axis means that changes in orientation (pitch, roll, or yaw) caused by user head movements are accurately recorded by the sensor and reported as changes in pitch, roll, or yaw. If the axes are not aligned, such changes in user head orientation will not correspond to the same changes in orientation reported by the orientation sensor.
[0048] For the purposes of this disclosure, a controller includes one or more processors and one or more non-transitory storage media, as well as any associated hardware for performing the various functions described in this disclosure. In one example, the controller may include a microcontroller that includes a processor and non-transitory storage media. The controller may also include multiple microcontrollers that cooperate to perform the various functions described. Therefore, in Figure 4 In the example, each of the controllers 406 and 410 includes at least one processor and a non-transitory storage medium storing program code that, when executed by the processor, provides spatialized audio to the acoustic transducers 420 and 422 based on the output of the orientation sensors 408 and 412.
[0049] The calibration described in this disclosure (including the steps of method 500) can be performed by any suitable controller of the earphone 400. Thus, in one example, calibration can be performed by controller 406 or controller 410. In practice, calibration can be performed by a combination of controllers 406 and 410 acting in concert. In this example, the controllers 406 and 410 working together can be considered as a single controller distributed between the earpieces 402 and 404. Furthermore, in some examples, the controller can be located outside the earpieces 402 and 404. For example, in a wired example, a downlink cable controller can perform the calibration. In practice, it is conceivable that calibration can be performed, at least in part, by a controller located outside the earphone 400 (such as by a mobile device 414, or even by a remote server accessed via an internet connection).
[0050] Orientation sensors 408 and 412 can be any orientation sensor comprised of one or more sensors that output data from which the sensor's orientation (i.e., the sensor's three-dimensional axes represent its orientation in space) can be determined. In one example, each orientation sensor 408 or 412 can be an inertial measurement unit (IMU). An IMU is a sensor that typically includes an accelerometer, a gyroscope, and sometimes a magnetometer, and outputs orientation, acceleration, and angular velocity. However, an IMU is only one example of a suitable orientation sensor. In an alternative example, the orientation sensors may each include multiple gyroscope sensors, each outputting angular rate data on at least one axis, such that the angular rate of the earpiece in three dimensions can be determined. Furthermore, the orientation sensors in each earpiece do not need to be of the same type. For example, the orientation sensor can be an accelerometer disposed in one earpiece, and the orientation sensor can include an accelerometer and a gyroscope sensor in another earpiece.
[0051] Orientation signals may include any suitable orientation data (including data that directly represents the orientation of the orientation sensor (and the earpiece to which it is attached, e.g., changes in pitch, roll, and yaw)), or may include other data from which orientation can be derived (such as the specific force and angular rate of the orientation sensor). Therefore, as mentioned above, orientation data can be encoded in various ways, including matrices, quaternions, vectors, and angles. Given that the orientation sensor consists of multiple sensors, it should be understood that the orientation signal may include multiple signals. In one example, as mentioned above, the orientation signal may include data encoded as an angular rate vector. A vector of angular rates represents the change in orientation at a given moment, i.e., the speed at which the orientation sensor rotates about each of its three orthogonal axes (X, Y, and Z). In various alternative examples, the orientation signal may include data encoded as a rotation vector, a game rotation vector, a geomagnetic rotation vector, or a quaternion. These forms will be understood, and therefore a more detailed description of each form is omitted here.
[0052] This mapping is a mathematical relationship (i.e., difference) between the orientations of orientation sensors 408 and 412, which, when applied to the axis of orientation sensor 408, yields the axis of orientation sensor 412. More specifically, since orientation sensors 408 and 412 are effectively attached to the same rigid body when placed in a user's ear, the orientation signal can be aligned with a rotation matrix. Typically, the mapping will operate in only a single direction. For example, the rotation matrix of the mapping will rotate the axis of orientation sensor 408 to the orientation of the axis of orientation sensor 412, but the inverse rotation matrix will rotate the axis of orientation sensor 412 to match the orientation of orientation sensor 408. For the purposes of this disclosure, the mapping is generally described as rotating the axis of orientation sensor 408 to the orientation of the axis of orientation sensor 412. However, this is arbitrary. The mapping can also be described based on the orientation that rotates the axis of orientation sensor 412 to the axis of orientation sensor 408.
[0053] Furthermore, applying a mapping to the orientation data of one sensor (e.g., orientation sensor 408) yields the orientation data of another sensor (e.g., orientation sensor 412) for the same sample. Thus, for example, if the orientation data of each orientation sensor is represented as an angular rate vector, the mapping (e.g., a rotation matrix) rotates the angular rate vector of orientation sensor 408 to the orientation of orientation sensor 412. For example, for a single sample, the orientation data of orientation sensor 408 could be represented as... (1)
[0054] in (The orientation data from the orientation sensor 408) is a three-dimensional vector of angular velocity, including the angular velocity along the x-axis. Angular rate in the direction of the y-axis Angular rate in the direction of the z-axis Similarly, for a single sample, the orientation data of the orientation sensor 408 can be represented as: (2)
[0055] in (The orientation data from orientation sensor 412) is a three-dimensional vector of angular velocity, including the angular velocity along the x-axis. Angular rate in the direction of the y-axis Angular rate in the direction of the z-axis Therefore, the mapping can be represented as a rotation vector R, as follows: , (3)
[0056] (symbol This means that the relationship of equation (3) is for n All integer values are true, and assumptions should be made for the following equations (3) through (6), although not explicitly stated. Conversely, the mapping can be represented by the inverse rotation matrix applied to the right-oriented data, as follows: (4)
[0057] Alternatively, the mapping can be represented as a unit quaternion Q, or as any other suitable rotation operator that encodes the rotation.
[0058] Since both orientation sensors 408 and 412 are mounted to the same rigid body (i.e., the user's head), they observe the same angular velocity, and therefore a rotation matrix exists that rotates the data from orientation sensor 408 to obtain the data for right rotation matrix 412. (Although the equations in this disclosure are proposed relative to the gyroscope sensor output of angular velocity, it will be apparent to those skilled in the art that quaternions or other similar outputs representing orientation can also be used and placed in a manner following the methods set forth herein.)
[0059] Once the mapping between orientation sensor 408 and orientation sensor 412 is determined, the axes of orientation sensors 408 and 412 can be calibrated according to the determined mapping. As described above, assuming that earpieces 402 and 404 are arranged anti-symmetrically about the XH-ZH mirror-symmetric plane on the user's head in at least one of the roll or yaw phases, the axes of orientation sensors 408 and 412 can be calibrated by adjusting the axis of one sensor (e.g., orientation sensor 408) to the midpoint of the mapping and the reverse—adjusting the axis of the other sensor (e.g., orientation sensor 412) to the midpoint of the mapping. (Although there are technically two potential mappings between orientation sensors 408 and 412, the mapping that brings the sensor axes closest to the user's head ("in front of the user" rather than "behind the user") is chosen. Due to the particularity of the way the axes are chosen in the product, this corresponds to the "shortest rotation".)
[0060] In other words, to remove any anti-symmetric components of the orientation sensor axes, the axes of orientation sensor 408 and orientation sensor 412 can be calibrated using a rotation matrix that "balances the differences between the sensors" and rotates the axis of orientation sensor 408 in the opposite direction to that of the axis of orientation sensor 412. This can be expressed by the following equation: (5)
[0061] Where R mid Let R be the midpoint of the rotation matrix R defined in equation (3). More specifically, the rotation matrix R is the midpoint rotation matrix R. mid The square of . Conversely, R mid Let R be the root of the rotation matrix that rotates the axis of orientation sensor 408 to the orientation of orientation sensor 412. This can be observed by rewriting equation (3) such that the square of the midpoint rotation is R. 2 mid The data from orientation sensor 408 is rotated to the data from orientation sensor 412 in the same manner as the rotation matrix R: (6)
[0062] As described below, the midpoint rotation matrix R can be determined using a closed-form solution or iteratively (i.e., adaptively). midOnce the midpoint of the mapping is determined, the axes of orientation sensors 408 and 412 can be calibrated based on that midpoint. Therefore, determining the mapping is an intermediate step. Once the mapping is determined, it can be assumed that the headphones are positioned on the human head in a substantially mirror-symmetrical manner, and thus the head axis is aligned with the midpoint of the mapping. For the purposes of this disclosure, calibrating the axes of orientation sensors 408 and 412 means adjusting the data output from orientation sensors 408 and 412 according to rotation based on the midpoint mapping or the inverse midpoint mapping. As long as the headphones are positioned in a substantially mirror-symmetrical manner, as a result of calibration, the axes of orientation sensors 408 and 412 will be more closely aligned in at least one of roll or yaw, and potentially both (although pitch misalignment may still persist). Calibration itself can be done, for example, by adjusting the data output from the orientation sensors (e.g., at the controller), or, if the orientation sensors have an associated processor, the processor of the orientation sensors can perform the adjustment before outputting the data to the controller.
[0063] Examples of closed-form solutions and iterative solutions will be briefly discussed. These examples are provided only as examples of methods for finding the midpoint of the mapping used to calibrate the orientation sensor; other examples may also be used.
[0064] In the example of the closed-form solution, multiple samples of the angular velocity vector can be used to construct the least-squares solution. Equation 5 is true for all samples, so multiple samples can be stacked to form a matrix equation. The assumptions are as follows: (7) (8)
[0065] Then, equation 5 becomes: (9)
[0066] Furthermore, as long as the angular rate matrix has at least three independent samples, R mid The solution can then be found as follows: (10)
[0067] Now that we have found the square of the midpoint rotation matrix, we can first find the value related to R. 2 mid The corresponding quaternion, then find the quaternion with half rotation to find the one with R. mid The corresponding quaternion. In other words, it is possible to find a quaternion corresponding to R. 2 mid The corresponding quaternion can be represented as qR 2 = [a, b, c, d]. Next, the rotation is qR. 2 midHalf of the quaternions can be determined by the following formula: (11)
[0068] Where θ = cos⁻¹(a) and v = [bcd] / ∥[bcd]∥.
[0069] However, a drawback of closed-form solutions is that when the rate gyroscope has noise, gain error, etc., it is difficult to guarantee R. 2 mid It is an effective rotation matrix. Therefore, although the closed-form method is feasible, it is not very flexible. To compensate for invalid rotation matrices, the solution can be used to find the closest effective rotation matrix, but this is computationally complex because it requires the matrix to be an orthogonal unitary matrix, which is a constraint that makes it impossible to compute an efficient solution.
[0070] Therefore, using the iterative solution from Equation 5, the error equation can be derived, and its gradient can be calculated based on the quaternion parameters to obtain the update equation for the quaternion. The advantage of working directly with quaternions is that the constraint corresponding to their rotation is that it is of unit length, which is a relatively easy constraint to implement compared to rotation matrix constraints. However, it should be understood that in alternative examples, an update equation based on the rotation matrix can be used.
[0071] In this example, the error equation used to derive the iterative method is the misaligned 2-norm. (12)
[0072] Where, for the rotation matrix, R -1 mid = R T mid .
[0073] The rotation matrix can be written using the quaternion parameters q = [a, b, c, d]. A quaternion corresponding to the square of the rotation matrix can be used, as follows: (13)
[0074] This matrix can be substituted into the error equation, and the gradient of the error equation can be calculated relative to the quaternion parameters. (14)
[0075] According to Equation 13, the partial derivative is: (15) (16) (17) (18)
[0076] The matrix above contains only four linearly independent rows, thus significantly reducing the number of calculations. The following intermediate variables can be used:
[0077] (19)
[0078] (20)
[0079] (twenty one)
[0080] (twenty two)
[0081] Based on the intermediate variables, the error gradient can be written as:
[0082] (twenty three)
[0083] In one example, the update equation used for adaptation is:
[0084] (twenty four)
[0085] Here, μ is a design parameter that balances fast convergence with smoother estimation. If μ is set too high, the algorithm may produce noisy results or even become unstable.
[0086] The final step of the iteration is to normalize q to have a unit length, which constrains it to correspond to a pure rotation. Although R mid It is the target, but the q obtained from this method corresponds to R. 2 mid After the algorithm converges, the same technique described in equation (11) can be used to determine R. mid Specifically, if the quaternion obtained by the iterative method is represented as q = [a,b,c,d], then the quaternion after half rotation is:
[0087] (26)
[0088] In the simulation, the algorithm converged on the order of seconds, which is satisfactory. However, if the convergence rate is still too slow, the step direction can be made second-order by incorporating the error into the Jacobian. This is likely to be very simple, since the gradient in the quaternion parameters is linear, and the Jacobian will be a constant value. The iterative method can be set up to directly solve for R. mid The corresponding quaternion is used, but the equation for the error gradient involves more matrix multiplications, so it is far less efficient than solving it directly.
[0089] Furthermore, it should be understood that any suitable adaptive equation can be used. Therefore, although the example above is described based on the least mean square adaptive algorithm, other adaptive algorithms, such as recursive least squares, can also be used.
[0090] Additionally, the mapping can be determined based on the different types of orientation sensors. For example, instead of orientation sensors 408 and 412 each comprising an inertial measurement unit or each comprising a gyroscope, sensor 408 may comprise both an accelerometer and a gyroscope, and sensor 412 may comprise an accelerometer. (In another example, sensor 408 may comprise an accelerometer, and sensor 412 may comprise both an accelerometer and a gyroscope.) It will be understood that these types of sensors can be used as standalone sensors or as components of another type of sensor, such as an inertial measurement unit, as described above, which typically comprises both an accelerometer and a gyroscope.
[0091] However, using accelerometers requires eliminating the gravity term inherent in their output signals. This can be achieved in any number of ways, including correlating the measurements of the left and right accelerometer signals using rigid body equations, as described below. Accelerometers also inherently possess bias, which must be eliminated, for example, by differentiating the accelerometer signals or by applying a high-pass filter, since the bias is approximately constant.
[0092] The left accelerometer signal (i.e., the signal from the accelerometer of sensor 408) can be represented as follows:
[0093] (27)
[0094] in For the measured linear acceleration, For the accelerometer's response to gravity (subscript C) L (Representing the coordinate system of the left accelerometer) For the accelerometer deviation, and This is noise (omitted in the following equations). Similarly, the right accelerometer signal (i.e., the signal from the accelerometer of sensor 412) can be expressed as:
[0095] (28)
[0096] in For the measured linear acceleration, For the accelerometer's response to gravity (subscript) (Representing the coordinate system of the right accelerometer) For the accelerometer deviation, and This is noise (which is omitted in the following equations).
[0097] To determine the mapping (which can also be encoded as a rotation matrix R or a unit quaternion Q, as described above), the outputs of sensors 408 and 412 can first be correlated according to the rigid body equations. For example, the linear acceleration of the right accelerometer can be represented in the left coordinate system C_L at the spatial point where the left accelerometer is located as follows:
[0098] (29)
[0099] in For point The acceleration at the point (i.e., the point in space of the orientation sensor 412), according to the point The acceleration at a point in space (of the orientation sensor 408) is used to represent the acceleration. Let be the angular velocity measured by the left gyroscope, and r be the angular velocity from point . Translate to point The vector. More specifically, r can be written as:
[0100] (30)
[0101] Equation (29) can be rewritten as
[0102] (31)
[0103] Subscript To highlight the coordinate system that uses the left accelerometer, and It is a skew-symmetric matrix given by the following formula:
[0104] (32)
[0105] Next, the following measurements were obtained from the left accelerometer and left gyroscope (of sensor 408) and the right accelerometer (of sensor 412). , (The left accelerometer's response to gravity at time t0, i.e., the accelerometer's initialization) Therefore, the rotation matrix (or quaternion q) L ) from arrive (That is, the rotation from sensor frame 408 at initialization to the current frame (or another subsequent frame, assuming the same subsequent frame is always used) is encoded.
[0106] Using these measurements, the rigid body equations (equation (31)) can be rewritten as follows:
[0107] (33)
[0108] In equation (33), on the left, the right linear acceleration... Used expression Instead, in which. On the right, Used expression replace.
[0109] It is worth noting that in equation (33), the gravity term Since they appear on both the left and right sides, they cancel each other out. Therefore, equation (33) can be simplified and rewritten as:
[0110] (34)
[0111] The following replacements have been made: , , and According to this equation, the rotation matrix R, the vector r, and the combined deviation term... It remains to be seen.
[0112] In any sensor frame ( or The deviation is constant (it changes very slowly): Therefore, it is approximately constant in measurement. Thus, the bias term can be eliminated by differentiating both sides of equation (34) component by component. ,get:
[0113] (35)
[0114] in: Equation (35) cannot be purely interpreted as a relationship between vectors with physical meaning because the vector derivatives are not obtained in the moving frame. Instead, the derivatives of the signal are derived component by component to eliminate constant bias.
[0115] Alternatively, a high-pass filter can be applied to each side of equation (34). This removes bias, which produces less noise than taking the derivative:
[0116] (36)
[0117] Equation (36) can be further simplified. With the gravity and deviation terms canceling out and eliminating, - and By calling the expression , Equation (36) can be rewritten as:
[0118] (37)
[0119] In either case (by taking the derivative or using a high-pass filter), with the gravity term and bias removed, the remaining two unknowns are ( and Equation (37) can be solved using any suitable method. For example (using a high-pass filter as an example), equation (37) can be written as an optimization problem, where and The following items will be optimized together:
[0120] (38)
[0121] Add to R The constraint must be a rotation matrix, resulting in:
[0122] (39)
[0123] The global optimization problem can be rewritten using quaternions. Therefore, the optimization problem is not about optimizing R, but can be rewritten as optimizing a quaternion Q that encodes the same rotations as R. For this purpose, modifications can be made... , , , To make it compatible with quaternion operations:
[0124] - , ,
[0125] - ,in
[0126] Therefore, as those skilled in the art will understand, at least two different methods (i.e., least squares and gradient descent) can be used to solve for R and r. A first example of these examples (least squares example, Q and r) can be found according to the following equations, where Q is a unit quaternion: (40)
[0127] The second example in these examples (gradient descent) can be found in the cost function. :
[0128] (41)
[0129] To calculate gradient descent, a starting point is chosen. And calculate at each step gradient: This utilizes Update: At the same time, the following constraints are enforced: Rotation is encoded by normalization: .
[0130] Once the mapping (rotation matrix R or quaternion Q) is found, the method can be performed as described above, i.e., by: calculating the midpoint of the rotation matrix; applying the midpoint to one of the sensors (e.g., sensor 408); and applying the inverse of the midpoint to the other sensor (e.g., sensor 412).
[0131] Additionally, the vector r (representing the vector between orientation sensors 408 and 412) can be used to adjust the interaural time difference (IAD) of the spatial algorithm. The IAD is a known parameter of the spatialized audio algorithm, representing the time difference of sound arrival between the user's ears. This arrival time difference is related to the size of the user's head (i.e., the distance between the user's ears), where vector r represents the size of the user's head. Therefore, vector r can be used as input to the spatialized audio algorithm as an estimate of the distance between the user's ears to adjust the IAD parameter. By more accurately measuring the distance between the user's ears, the performance of the spatialized audio can be improved.
[0132] Although the vector r is described above as specifically related to the distance between the accelerometers, it should be understood that any suitable orientation sensor output can be used. Specifically, based on the example described above, any suitable orientation sensor output can be correlated with rigid body equations, which can be solved to obtain the vector r.
[0133] Figure 5 A flowchart depicts a method for calibrating the axial alignment of an orientation sensor. The steps of method 500 can be performed by a controller as described above (such as controllers 406, 410, or a controller consisting of two controllers 406, 410 acting in concert). Therefore, in this example, the steps of method 500 can be performed by one or more processors executing program code stored in one or more non-transitory storage media. For the purposes of this method, the earpieces will be described as a “first” earpiece and a “second” earpiece. This is to emphasize that the method does not depend on the left or right earpiece performing specific steps. Thus, the left earpiece can be the first earpiece, and the right earpiece can be the second earpiece; alternatively, the right earpiece can be the first earpiece, and the left earpiece can be the second earpiece.
[0134] At step 502, a first orientation signal indicating the orientation of the first earpiece is received. The first orientation signal is received from an orientation sensor disposed in the first earpiece. At step 504, a second orientation signal indicating the orientation of the second earpiece is received. The second orientation signal is received from an orientation sensor disposed in the second earpiece.
[0135] As described above, the first and second orientation sensors can be any orientation sensor composed of one or more sensors that output data, from which the orientation of the sensor can be determined (i.e., the sensor's three-dimensional axes represent its orientation in space). Examples of sensors include inertial measurement units or multiple gyroscope sensors. Furthermore, sensors include different sensor types, such as an accelerometer and gyroscope in one earpiece and an accelerometer in another earpiece.
[0136] The orientation signal may include data that directly represents the orientation of the orientation sensor (and the earpiece to which it is attached) (e.g., changes in pitch, roll, and yaw), or may include other data from which orientation can be derived (such as the specific force and angular rate of the orientation sensor). The orientation signal output from the first orientation sensor is relative to the axis of the first orientation sensor. In other words, the first orientation sensor does not necessarily provide absolute orientation, but rather a relative orientation, which is typically provided based on the change of its orthogonal axis as the first orientation sensor rotates in space. Similarly, an orientation signal output from the second orientation sensor relative to the axis of the orientation sensor is provided to the controller.
[0137] Since the orientation sensor consists of multiple sensors, it should be understood that the orientation signal may include multiple signals. In various alternative examples, the orientation signal may include data encoded as a rotation vector, a game rotation vector, a geomagnetic rotation vector, or a quaternion.
[0138] At step 506, the mapping between the axes of the first and second orientation sensors is determined based on the difference between the first and second orientation signals. The mapping (e.g., represented by the rotation matrix in equation (3)) is a mathematical relationship (i.e., the difference) between the orientations of the first and second orientation sensors. When applied to the axis of the first orientation sensor, the mapping yields the axis of the second orientation sensor. Since the first and second orientation sensors are effectively attached to the same rigid body when placed in the user's ear, the orientation signals can be aligned with the rotation matrix. Typically, the mapping will work only in a single direction. For example, the rotation matrix of the mapping will rotate the axis of the first orientation sensor to the orientation of the axis of the second orientation sensor, but the inverse rotation matrix will rotate the axis of the second orientation sensor to match the orientation of the first orientation sensor. Applying the mapping to the orientation data of the first orientation sensor yields the orientation data of the second orientation sensor for the same sample.
[0139] The mapping (or the midpoint of the mapping) can be determined using a closed-form solution (examples of which are described above in conjunction with equations (1)–(10)) or iteratively (i.e., adaptively) (examples of which are described above in conjunction with equations (12)–(24)). The adaptive algorithm used can be any suitable adaptive algorithm, including the least mean square algorithm or the recursive least squares algorithm. Furthermore, in various examples, the adaptive algorithm can find the midpoint mapping based on quaternions or rotation matrices. If the different sensor types described above are used (using an accelerometer and a gyroscope as one sensor, and an accelerometer as another sensor), this can be performed according to the examples described above in conjunction with equations (27)–(41).
[0140] At step 508, the first orientation axis is calibrated according to the midpoint of the mapping. Calibration of the axis of the first orientation sensor can be achieved by adjusting the data output from the orientation sensor such that the axis is aligned with the axis of the first orientation sensor rotated via the midpoint mapping (as described above in conjunction with equation (5) based on the rotation matrix). This can be achieved, for example, by adjusting the data output from the first orientation sensor, or, if the first orientation sensor has an associated processor, by the processor of the first orientation sensor performing the adjustment before output. The midpoint mapping can be determined based on the mapping determined in step 506, for example, as described above in conjunction with equations (11) and (26).
[0141] At step 510, the second orientation axis is calibrated according to the inverse of the midpoint of the mapping. Similar to the first orientation sensor, the axis of the second orientation sensor can be calibrated by adjusting the data output from the second orientation sensor such that the axis is aligned with the axis of the second orientation sensor rotated via the inverse midpoint mapping (as described above in conjunction with equation (5) based on the rotation matrix). This can be achieved, for example, by adjusting the data output from the second orientation sensor, or, if the second orientation sensor has an associated processor, by the processor performing the adjustment before output. The midpoint mapping can be determined according to the mapping determined in step 506, for example, as described above in conjunction with equations (11) and (26).
[0142] At step 512, a spatialized audio signal is rendered based on a calibrated first orientation signal and a calibrated second orientation signal. That is, the spatialized audio signal is rendered based on the first orientation signal and rotated according to the midpoint of the mapping between the orientation sensors, and rendered based on the second orientation signal and rotated inversely according to the midpoint of the mapping between the orientation sensors. As will be understood, the spatialized audio signal includes a left audio signal delivered to the acoustic transducer in the left earpiece and a right audio signal delivered to the acoustic transducer in the right earpiece. The spatialized audio signal delivered to the acoustic transducer in the earpiece causes transduction of a spatialized acoustic signal that is perceived by the user as originating from at least one location different from the transducer.
[0143] The generation of spatialized audio signals can be achieved through any suitable spatialized audio algorithm, as known in the art. Additionally, as described above, the vector *r* can be determined based on the rigid body equations and the outputs of the orientation sensors, representing the distance between the orientation sensors, and, by extension, the distance between the user's ears. This value can be used to adjust (i.e., tune) the inter-ear time difference of the spatial algorithm. As described above, the inter-ear time difference is a known parameter of the spatialized audio algorithm, representing the time difference of sound arrival between the user's ears. The time difference of arrival will be related to the size of the user's head (i.e., the distance between the user's ears), where the vector *r* represents the size of the user's head. Therefore, the vector *r* can be used as input to the spatialized audio algorithm as an estimate of the distance between the user's ears to adjust the inter-ear time difference parameter. Spatialized audio performance can be improved by more accurately measuring the distance between the user's ears.
[0144] The functionality or portions thereof described herein, and its various modifications (hereinafter referred to as “functionality”), may be implemented at least in part in a computer program product (e.g., a computer program tangibly implemented in an information carrier, such as one or more non-transitory machine-readable media or storage devices) for execution by or control of the operation of one or more data processing devices (e.g., a programmable processor, a computer, multiple computers and / or programmable logic components).
[0145] Computer programs can be written in any programming language (including compiled or interpreted languages) and can be deployed in any form (including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment). Computer programs can be deployed on a single computer, distributed across a site or multiple sites, or executed on multiple computers interconnected by a network.
[0146] The actions associated with all or part of the functions in the implementation can be performed by one or more programmable processors executing one or more computer programs to perform the functions of the calibration process. All or part of the functions can be implemented as special-purpose logic circuits, such as FPGAs and / or ASICs (Application-Specific Integrated Circuits).
[0147] Processors suitable for executing computer programs include, for example, both general-purpose microprocessors and special-purpose microprocessors, as well as any one or more processors in any type of digital computer. Generally, a processor receives instructions and data from read-only memory or random access memory, or both. The components of a computer include a processor for executing instructions and one or more memory devices for storing instructions and data.
[0148] While several inventive embodiments have been described and illustrated herein, those skilled in the art will readily conceive of a variety of other components and / or structures for performing the functions described herein and / or obtaining one or more of the results and / or advantages described herein, and each such variation and / or modification is considered to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily understand that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications using the teachings of this invention. Those skilled in the art will recognize, or can determine, many equivalents of the specific inventive embodiments described herein using only conventional experimentation. Therefore, it should be understood that the above embodiments are presented by way of example only, and that inventive embodiments may be practiced in ways other than those specifically described and claimed within the scope of the appended claims and their equivalents. The inventive embodiments disclosed herein relate to each individual feature, system, article, material, and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials and / or methods is included within the scope of this disclosure if such features, systems, articles, materials and / or methods do not contradict each other.
Claims
1. A pair of earphones with orientation sensor axial alignment self-calibration, the earphone pair comprising: A first earpiece, the first earpiece housing a first orientation sensor, the first orientation sensor outputting a first orientation signal, wherein the first orientation signal represents the orientation of the first earpiece and is relative to a first orientation axis of the first orientation sensor; A second earpiece, the second earpiece housing a second orientation sensor, the second orientation sensor outputting a second orientation signal, wherein the second orientation signal indicates the orientation of the second earpiece and is relative to a second orientation axis of the second orientation sensor; And a controller configured to calculate a mapping between a first orientation sensor axis and a second orientation sensor axis based on the difference between the first orientation signal and the second orientation signal, wherein the controller is further configured to calibrate the first orientation axis based on the midpoint of the mapping and to calibrate the second orientation axis based on the inverse of the midpoint of the mapping, such that when the user wears the first and second earpieces antisymmetrically about at least one mirror-symmetric plane of the user's head, at least one of the roll and yaw of the first and second orientation sensor axes is more closely aligned with the user's head axis.
2. The earphone pair according to claim 1, wherein the mapping is calculated according to an adaptive algorithm.
3. The earphone pair according to claim 1, wherein the mapping is calculated non-adaptively.
4. The earphone pair according to claim 1, wherein the first orientation sensor and the second orientation sensor are each inertial measurement units.
5. The earphone pair according to claim 1, wherein the first orientation sensor and the second orientation sensor each include at least a gyroscope sensor.
6. The earphone pair according to claim 1, wherein the first orientation sensor is an accelerometer and a gyroscope sensor, and wherein the second orientation sensor is a gyroscope sensor.
7. The earphone pair according to claim 1, wherein the controller is housed in at least one of the first earpiece or the second earpiece.
8. The earphone pair of claim 1, wherein the controller is further configured to render a spatialized audio signal based on a calibrated first orientation signal and a calibrated second orientation signal.
9. The earphone pair of claim 8, wherein the spatialized audio signal is determined according to a spatialized audio algorithm including an interaural time difference parameter, wherein the controller is further configured to adjust the interaural time difference parameter according to a vector representing the distance between the first orientation sensor and the second orientation.
10. A method for calibrating the axial alignment of an orientation sensor, the method comprising: Receive a first orientation signal indicating the orientation of the first earpiece of the earphone pair, the first orientation signal being relative to a first orientation axis of the first orientation sensor; Receive a second orientation signal indicating the orientation of the second earpiece of the earphone pair, the second orientation signal being relative to the second orientation axis of the second orientation sensor; The mapping between the first orientation sensor axis and the second orientation sensor axis is calculated based on the difference between the first orientation signal and the second orientation signal; The first orientation axis is calibrated based on the midpoint of the mapping; as well as The second orientation axis is calibrated inversely to the midpoint of the mapping, such that when the user wears the first and second earpieces antisymmetrically about at least one mirror-symmetric plane of the user's head, at least one of the roll and yaw of the first and second orientation sensor axes is more closely aligned with the user's head axis.
11. The method of claim 10, wherein the mapping is calculated according to an adaptive algorithm.
12. The method of claim 10, wherein the mapping is computed non-adaptively.
13. The method of claim 10, wherein the first orientation sensor and the second orientation sensor are each an inertial measurement unit.
14. The method of claim 10, wherein the first orientation sensor and the second orientation sensor each comprise at least one gyroscope sensor.
15. The method of claim 10, wherein the first orientation sensor is an accelerometer and a gyroscope sensor, and wherein the second orientation sensor is an accelerometer.
16. The method of claim 10, further comprising: A spatialized audio signal is rendered based on a calibrated first orientation signal and a calibrated second orientation signal, wherein the spatialized audio signal is determined according to a spatialized audio algorithm, the spatialized audio algorithm including an interaural time difference parameter, and The interaural time difference parameter is adjusted based on a vector representing the distance between the first orientation sensor and the second orientation.
17. At least one non-transitory storage medium storing program code for execution on at least one processor, the program code calibrating the axial alignment of an orientation sensor pair when executed, comprising: Receive a first orientation signal indicating the orientation of the first earpiece of the earphone pair, the first orientation signal being relative to a first orientation axis of the first orientation sensor; Receive a second orientation signal indicating the orientation of the second earpiece of the earphone pair, the second orientation signal being relative to the second orientation axis of the second orientation sensor; The mapping between the first orientation sensor axis and the second orientation sensor axis is calculated based on the difference between the first orientation signal and the second orientation signal; The first orientation axis is calibrated based on the midpoint of the mapping; as well as The second orientation axis is calibrated inversely to the midpoint of the mapping, such that when the user wears the first and second earpieces antisymmetrically about at least one mirror-symmetric plane of the user's head, at least one of the roll and yaw of the first and second orientation sensor axes is more closely aligned with the user's head axis.
18. The non-transitory storage medium of claim 17, wherein the mapping is calculated according to an adaptive algorithm.
19. The non-transitory storage medium of claim 17, wherein the mapping is computed non-adaptively.
20. The non-transient storage medium according to claim 17, wherein the first orientation sensor and the second orientation sensor are each an inertial measurement unit.