Pose correction method and device, chip, equipment and storage medium

Abstract

The application discloses a pose correction method and device, a chip, equipment and a storage medium, and relates to the technical field of positioning. The method comprises the following steps: determining a pose drift amount based on first sensor data of a first device and second sensor data of a second device, wherein the pose drift amount is used to represent a drift condition of a pose of the first device relative to the second device; and performing pose correction based on the pose drift amount to obtain corrected first sensor data and / or second sensor data. The embodiment scheme of the application improves the accuracy of the device pose tracking result without increasing the cost of additional hardware.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and in particular to an attitude correction method, device, chip, equipment, and storage medium. Background Technology

[0002] With the development of audio playback devices and audio applications, headphone devices can provide users with an immersive audio experience through spatial audio. Spatial audio uses head tracking technology to track the user's head movements and remaps the sound field based on the motion data, ensuring that the sound field is fixed at any specific location in space, such as the location of the playback device.

[0003] In related technologies, headphone devices achieve head tracking through a built-in inertial measurement unit. However, there is a device-based drift error in the measurement process, and the head tracking posture error increases with the length of use, which in turn increases the sound field position shift and affects the spatial audio effect. Summary of the Invention

[0004] This application provides an attitude correction method, apparatus, chip, device, and storage medium, which can improve the accuracy of device attitude tracking. The technical solution is as follows:

[0005] On one hand, embodiments of this application provide an attitude correction method, the method being used in a first device, the method comprising:

[0006] The attitude drift is determined based on the first sensor data of the first device and the second sensor data of the second device. The attitude drift is used to characterize the drift of the first device relative to the second device.

[0007] Attitude correction is performed based on the attitude drift amount to obtain the corrected first sensor data and / or second sensor data.

[0008] On the other hand, embodiments of this application provide an attitude correction method for a terminal device, the method comprising:

[0009] The attitude drift is determined based on the data from the first sensor and the data from the second sensor. The first sensor data refers to the sensor data from the first device, and the second sensor data refers to the sensor data from the second device. The attitude drift is used to characterize the attitude drift of the first device relative to the second device.

[0010] Attitude correction is performed based on the attitude drift amount to obtain the corrected first sensor data and / or second sensor data.

[0011] On the other hand, embodiments of this application provide an attitude correction device, which is used in a first device, and the device includes:

[0012] The first determining module is used to determine the attitude drift amount based on the first sensor data of the first device and the second sensor data of the second device, wherein the attitude drift amount is used to characterize the attitude drift of the first device relative to the second device;

[0013] The first attitude correction module performs attitude correction based on the attitude drift amount to obtain the corrected first sensor data and / or second sensor data.

[0014] On the other hand, embodiments of this application provide an attitude correction device for a terminal device, the device comprising:

[0015] The second determining module is used to determine the attitude drift amount based on the first sensor data and the second sensor data. The first sensor data is the sensor data of the first device, and the second sensor data is the sensor data of the second device. The attitude drift amount is used to characterize the attitude drift of the first device relative to the second device.

[0016] The second attitude correction module is used to perform attitude correction based on the attitude drift amount to obtain the corrected first sensor data and / or second sensor data.

[0017] On the other hand, embodiments of this application provide a chip, the chip including a processor, the processor being configured to:

[0018] The attitude drift is determined based on the first sensor data of the first device and the second sensor data of the second device. The attitude drift is used to characterize the attitude drift of the first device relative to the second device.

[0019] Attitude correction is performed based on the attitude drift amount to obtain the corrected first sensor data and / or second sensor data.

[0020] On the other hand, embodiments of this application provide an electronic device, the device including a processor and a memory, the memory storing at least one program, the at least one program being loaded and executed by the processor to implement the attitude correction method as described above.

[0021] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one program, wherein the at least one instruction is loaded and executed by a processor to implement the attitude correction method as described above.

[0022] On the other hand, embodiments of this application provide a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the attitude correction method described above.

[0023] In this embodiment, the first device determines the attitude drift of the first device relative to the second device over time based on the first sensor data and the second sensor data, that is, it determines the attitude drift amount that characterizes the sensor error. Without increasing hardware costs, the first device obtains the observations required to correct the sensor error in the attitude by calculating the attitude drift amount, and then performs attitude correction on the first device and / or the second device, which improves the accuracy of the sensor data and the accuracy of the attitude of the first device and the second device determined based on the sensor data, thereby improving the accuracy of the attitude tracking results. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram of an implementation environment provided by an exemplary embodiment of this application is shown;

[0026] Figure 2 A schematic diagram of an implementation environment provided by another exemplary embodiment of this application is shown;

[0027] Figure 3 A flowchart of an attitude correction method provided in an exemplary embodiment of this application is shown;

[0028] Figure 4 A flowchart illustrating the attitude drift determination process provided in an exemplary embodiment of this application is shown.

[0029] Figure 5 A flowchart of an automatic correction method provided in an exemplary embodiment of this application is shown;

[0030] Figure 6 A flowchart of an attitude correction method provided in another exemplary embodiment of this application is shown;

[0031] Figure 7 A flowchart of an attitude correction method provided in another exemplary embodiment of this application is shown;

[0032] Figure 8 A structural block diagram of an attitude correction device provided in an exemplary embodiment of this application is shown;

[0033] Figure 9 A structural block diagram of an attitude correction device provided in another exemplary embodiment of this application is shown;

[0034] Figure 10 This invention illustrates a structural block diagram of a terminal device provided in an exemplary embodiment of this application;

[0035] Figure 11 A structural block diagram of a headphone device provided in another exemplary embodiment of this application is shown. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0037] For ease of understanding, the terms used in the embodiments of this application will be explained below.

[0038] An IMU (Inertial Measurement Unit) is a sensor that determines the orientation and position of a carrier using inertial devices. Internally, it consists of a flexible accelerometer (accelerometer) and a gyroscope. The accelerometer measures the carrier's acceleration signal, and the gyroscope measures the carrier's angular rate signal. An inertial measurement unit can measure the angular velocity and acceleration of an object in three-dimensional space without relying on external information, and use this to calculate the object's attitude.

[0039] Kalman Filter (KF) is an algorithm for optimally estimating the state of a system. It utilizes the state equations of a linear system and the observed data from the system output to make an optimal estimate of the system state. Since both the system and observed data contain noise and interference, the optimal estimation process can also be viewed as a filtering process. Kalman Filter is applicable to linear, discrete, and finite-dimensional systems. In applications, an autoregressive moving average system with external variables or a system that can be represented by a rational transfer function can be transformed into a system represented in state space, thus enabling computation using Kalman Filter.

[0040] Spatial sound, also known as 3D sound, is sound simulated by speakers or headphones that can create a stereo sound field, based on psychoacoustic effects and acoustic algorithms. Spatial sound simulates sound sources emanating from specific locations in three-dimensional space, including the horizontal directions of the listener (front, back, left, right) and the vertical directions (above, below). In real life, sound has direction and distance, and the sound source itself has a certain width. Based on the direction, distance, and width of the sound, the sound source can be located. In spatial sound, the Head Related Transfer Function (HRTF) and spatial convolution of sound waves are used to mimic the propagation of natural sound waves, making the sound seem to come from a point in three-dimensional space. Because the brain judges the direction and distance of sound through subtle differences in the sound received by the left and right ears, such as ITD (Interaural Time Difference) and ILD (Interaural Level Difference), headphones and other devices need to detect the user's head posture and the spatial position of the user's head relative to the sound-emitting device when implementing spatial sound. In the context of using headphone devices to achieve spatial sound effects, related technologies involve headphone devices tracking the user's head posture using sensors such as the IMU built into them.

[0041] Spatial audio functionality determines the relationship between the sound field position and the headphone device based on the user's head posture. Therefore, during the sound field position update process, the headphone device needs to accurately determine its own posture. Due to the low performance of MEMS (Micro-Electro-Mechanical Systems) devices, the IMU itself has various inherent errors. In this embodiment, without increasing hardware costs, the first device or terminal device can determine the posture drift as an error observation based on first sensor data and second sensor data, correcting the posture of the first device (i.e., the headphone device) to obtain more accurate posture tracking results, improving the accuracy of posture tracking, and thus enhancing the realism of spatial sound effects in scenarios using spatial audio functionality.

[0042] Figure 1 A schematic diagram of an implementation environment provided by an exemplary embodiment of this application is shown. The implementation environment includes a first device 101, a second device 102, and a terminal device 103.

[0043] The first device 101 is an earphone device with posture tracking function, and the first device 101 has sensors such as IMU; the second device 102 is a smart wearable device with posture tracking function, and the wearing position is usually relatively fixed. For example, the second device can be a smart earphone, smartwatch, bracelet, AR glasses, etc. The second device 102 can have sensors such as IMU and has posture tracking function.

[0044] In one possible implementation, when both the first device 101 and the second device 102 are in a wearing state, the relative posture between the first device 101 and the second device 102 remains unchanged. Therefore, the amount of posture drift between the first sensor data of the first device 101 and the second sensor data of the second device 102 can characterize the error introduced by the sensor drift of the first device 101 and the second device 102.

[0045] Figure 1 In this example, both the first device 101 and the second device 102 are headphone devices, but this does not constitute a limitation.

[0046] Terminal device 103 can be a smart TV, smartphone, tablet computer, or other device with audio playback function.

[0047] In one possible implementation, the first device 101 and the second device 102 can transmit sensor data via a Bluetooth link or through Wi-Fi. Optionally, the first device 101 and the second device 102 can establish a data communication connection with the same terminal device.

[0048] Furthermore, the first device 101 can simultaneously perform attitude correction with multiple second devices 102 that have attitude tracking functions. This application embodiment only illustrates the example of the first device 101 combined with one second device 102, but it does not constitute a limitation.

[0049] Figure 2 A schematic diagram of an implementation environment provided by another exemplary embodiment of this application is shown. This implementation environment includes a first device 201, a second device 202, and a terminal device 203.

[0050] The first device 201 is an earphone device with posture tracking function, and includes sensors such as an IMU. The second device 202 is a smart wearable device with posture tracking function, and its wearing position is usually relatively fixed. For example, the second device can be a smart earphone, smartwatch, bracelet, AR glasses, etc. The second device 202 may include sensors such as an IMU and has posture tracking function. In one possible implementation, when both the first device 101 and the second device 102 are in a wearing state, the relative posture between the first device 101 and the second device 102 remains unchanged. Figure 2 In this example, the first device 201 is an earphone device and the second device 202 is a smart glasses device, but this does not constitute a limitation.

[0051] Terminal device 203 is a terminal device with a sensor data processing module. It receives sensor data transmitted from first device 201 and second device 202, and performs calculations and Kalman filtering on the sensor data to correct it. In this embodiment, terminal device 203 has an audio playback function, which can be a native application of the terminal device or a function of a third-party application.

[0052] Optionally, communication connections can be established between the first device and the terminal device, and between the second device and the terminal device, via Bluetooth links, Wi-Fi, or other means.

[0053] Furthermore, the terminal device 203 can simultaneously establish data communication connections with multiple first devices 201 and second devices 202 that have attitude tracking functions to perform attitude correction. This application embodiment only illustrates the terminal device 203 combined with one first device 201 and one second device 202 as an example, but it does not constitute a limitation.

[0054] Please refer to Figure 3 This document illustrates a flowchart of an attitude correction method provided in an exemplary embodiment of this application. The embodiments of this application use this method for... Figure 1 Taking the first device shown as an example, the method may include the following steps.

[0055] Step 301: Determine the attitude drift amount based on the first sensor data of the first device and the second sensor data of the second device. The attitude drift amount is used to characterize the attitude drift of the first device relative to the second device.

[0056] In this embodiment, both the first and second devices are smart wearable devices. When both devices are worn, there is a certain correlation between the posture of the first and second devices and the user's posture. Therefore, the first sensor data of the first device, while representing the posture of the first device, can also represent the user's current posture. When the first device is an earphone, the first sensor data can represent the user's head posture. In this embodiment, when the first and second devices are worn, their relative postures remain unchanged, meaning their motion postures are consistent. Therefore, the posture drift of the first device relative to the second device represents the data error caused by the sensor drift in both devices. Consequently, the posture drift can serve as an objective standard for correcting the first and second sensor data.

[0057] Optionally, the first device can be an earphone device, and the second device can be a smart glasses, a smart necklace, a smart earphone, or other devices. Based on the wearing method of the first and second devices, the movement posture of the first and second devices remains consistent when they are worn, that is, the posture of the first device relative to the second device is not affected.

[0058] In one possible implementation, the first device acquires first sensor data transmitted by its own sensors. This first sensor data includes the attitude data of the first device. When the first device uses an IMU as a sensor, the first sensor data can be attitude data based on accelerometers and gyroscopes. Optionally, the mathematical expression of the attitude can be three Euler angles, or it can be an attitude rotation matrix, quaternions, rotation vectors, etc. This embodiment uses Euler angles as an example to represent the attitude of the first and second devices. Since the relative attitude of the first and second devices remains unchanged, the relative difference between the first and second sensor data is a definite value, characterizing the relative attitude between the two devices. In practical applications, because sensor devices are composed of MEMS components, sensor data has hardware errors, such as gyroscope bias instability, thermal noise, and random walk errors. During use, these errors accumulate over time, affecting the accuracy of the data and causing the relative difference between the first and second sensor data to change. This manifests as a drift in the attitude of the first device relative to the second device based on the sensor data.

[0059] Furthermore, based on the objective law that the relative difference between the first sensor data and the second sensor data remains constant, the first device determines that the change in the relative difference is the attitude drift caused by the sensor error. That is, the attitude drift can reflect the error accumulated by the sensor itself during the application process. While characterizing the attitude drift of the first device relative to the second device, it also characterizes the error of the sensor data.

[0060] Step 302: Perform attitude correction based on attitude drift to obtain corrected first sensor data and / or second sensor data.

[0061] The attitude drift amount characterizes the sensor errors introduced by the first and second sensor data. The first device can perform attitude correction on the first device and / or the second device based on this error to eliminate the error and improve the accuracy of the attitude tracking results. In the embodiments of this application, based on the attitude drift amount, the first device can correct the attitude of the first device determined based on the first sensor data and the attitude of the second device determined based on the second sensor data, thereby obtaining more accurate attitude information of the first and second devices. Correspondingly, the first device can also correct the gyroscope data in the first and second sensor data, correcting errors such as zero-bias instability of the gyroscope in the IMU sensor, thereby improving the accuracy of subsequent gyroscope measurement data.

[0062] In one possible implementation, after obtaining the attitude drift, the first device incorporates the attitude drift into an automatic correction algorithm to correct the sensor data. Optionally, the first device may employ filtering algorithms such as the Least Square Method, Gradient Descent, Kalman Filter, Extended Kalman Filter (EKF), or Unscented Kalman Filter (UKF) to correct the sensor data.

[0063] In summary, in this embodiment, the first device acquires first sensor data and second sensor data, and calculates the attitude drift using the offset of the relative difference between the first and second sensor data. Based on the attitude drift representing the error introduced by the sensor itself, the first device corrects its attitude data using the attitude drift. The first device determines the deviation of its attitude relative to the second device over time based on the first and second sensor data, i.e., it determines the attitude drift representing the sensor error. Without increasing hardware costs, the first device calculates the observation required to correct the deviation, i.e., the attitude drift, and then corrects the attitude of the first device and / or the second device, improving the accuracy of the sensor data and the accuracy of the device attitude determined based on the sensor data, thereby improving the accuracy of the attitude tracking results.

[0064] The spatial audio function determines the current user's head posture by positioning, and then determines the sound field effect based on the positional relationship between the user's head posture and the sound field.

[0065] Please refer to Figure 4 The diagram illustrates a flowchart of the attitude drift determination process provided in an exemplary embodiment of this application.

[0066] Step 401: When the first device is in the wearing state and there is a need for posture tracking, acquire the first initial sensor data of the first device and receive the second initial sensor data sent by the second device, which is sent by the second device when it is in the wearing state.

[0067] The presence of posture tracking requirement could refer to a terminal device establishing a communication connection with the first device activating its spatial audio function and sending a command to the first earphone device to acquire first sensor data. Since both the first and second devices are smart wearable devices, when they are worn, they are located at the user's fixed position, and the first sensor data of the first device can characterize the user's posture. Because the first sensor data of the first device, in its initial state, does not yet include errors accumulated over time due to zero-bias instability, the second sensor data of the second device can also accurately characterize the posture of the second device. Therefore, the first initial sensor data and the second initial sensor data can serve as standard values ​​for calibrating the sensor data; that is, the difference between the first initial sensor data and the second initial sensor data can accurately characterize the relative posture between the first and second devices.

[0068] In one possible implementation, the first device acquires current sensor data as first initial sensor data in an initial state. Based on the first initial sensor data, the first device can determine its own initial attitude data, which can be mathematically represented as Euler angles (X). 10 ,Y 10 Z 10 The aforementioned initial state refers to the situation where the first device detects that it is being worn and has received an attitude tracking command. Correspondingly, when the first device detects that the second device is being worn, it can send a command to acquire sensor data from the second device and receive second initial sensor data transmitted by the second device. Based on the obtained second initial sensor data, the first device can determine the initial attitude data (X) of the second device. 20 ,Y 20 Z 20 ).

[0069] In an illustrative example, the first device is an earphone device and the second device is a smart glasses device. When the first device detects that it is in an in-ear state and receives a posture tracking command, the first device acquires the first initial sensor data output by its own sensors. Correspondingly, the first device detects the state of the second device. When the second device is in a wearing state, the first device receives the second initial sensor data sent by the second device.

[0070] Step 402: Determine the initial relative attitude based on the first initial sensor data and the second initial sensor data.

[0071] Based on the first initial sensor data (X) 10 ,Y 10 Z 10 The first device's attitude in its initial state is represented by the second initial sensor data (X). 20 ,Y 20 Z 20 The attitude data represents the attitude of the second device in the initial state. Furthermore, since the sensor data in the initial state has not yet been affected by errors such as zero bias instability, the attitude data accurately represents the relative attitude between the first and second devices in the initial state.

[0072] In one possible implementation, the first device will transmit the first initial sensor data (X). 10 ,Y 10 Z 10 ) and the second initial sensor data (X) 20 ,Y 20 Z 20 By subtracting the first and second devices, the initial relative attitude (X) between them is obtained. 30 ,Y 30 Z 30 ), that is:

[0073] (X 30 ,Y 30 Z 30 )=(X 10 ,Y 10 Z 10 )-(X 20 ,Y 20 Z 20 )

[0074] Step 403: During the attitude tracking process, when the attitude correction cycle is reached, the first real-time sensor data of the first device is acquired, and the second real-time sensor data sent by the second device is received.

[0075] In one possible implementation, the first device can set an attitude correction period, which is also the sensor data acquisition period. For example, the first device sets the attitude correction period to 1 second. During attitude tracking, the first device acquires first real-time sensor data based on the attitude correction period. This first sensor data may contain attitude data characterizing the real-time attitude of the first device, and the mathematical expression of this attitude data may be Euler angles, that is, the first sensor data contains the attitude (X) of the first device. 11 ,Y 11 Z 11The corresponding first device receives second real-time sensor data based on the attitude correction cycle, and the second real-time sensor data includes the real-time attitude data (X) of the second device. 21 ,Y 21 Z 21 ).

[0076] Step 404: Determine the real-time relative attitude based on the first real-time sensor data and the second real-time sensor data.

[0077] Based on the initial relative attitude (X) 30 ,Y 30 Z 30 This accurately reflects the relative attitude between the first and second devices, and the relative attitude between the first and second devices remains unchanged during attitude tracking. Therefore, under error-free conditions, the relative difference between the first real-time sensor data and the second real-time sensor data should be close to the initial relative attitude (X). 30 ,Y 30 Z 30 The same applies to attitude tracking. During attitude tracking, as usage time increases, both the first sensor data of the first device and the second sensor data of the second device exhibit a certain degree of error, failing to accurately represent the current attitude of the first and second devices. Correspondingly, the relative difference between the first sensor data and the second sensor data is similar to the initial relative attitude (X). 30 ,Y 30 Z 30 The values ​​are not equal, and the offset represents the error of the first sensor data.

[0078] In one possible implementation, the first device will transmit the first real-time sensor data (X). 11 ,Y 11 Z 11 ) and second real-time sensor data (X 21 ,Y 21 Z 21 The difference is calculated to determine the real-time relative attitude (X). 31 ,Y 31 Z 31 ), that is:

[0079] (X 31 ,Y 31 Z 31 )=(X 11 ,Y 11 Z 11 )-(X 21 ,Y 21 Z 21 )

[0080] Step 405: The attitude difference between the real-time relative attitude and the initial relative attitude is determined as the attitude drift.

[0081] Based on the unchanged relative attitude between the first and second devices, and assuming no error, the initial relative attitude (X) 30 ,Y 30 Z 30 ) and real-time relative attitude (X 31 ,Y 31 Z 31 The initial relative attitude (X) should be equal. During attitude tracking, due to the accumulated error over time in the first sensor data of the first device, in practical applications, the initial relative attitude (X) should be equal. 30 ,Y 30 Z 30 ) and real-time relative attitude (X) 31 ,Y 31 Z 31 The values ​​are not equal, meaning that the real-time relative attitude deviates from the initial relative attitude due to the error. The deviation between the two can characterize the error of the first sensor data.

[0082] In one possible implementation, the first device will display the relative attitude (X) in real time. 31 ,Y 31 Z 31 ) and initial relative attitude (X) 30 ,Y 30 Z 30 The difference between the two is calculated, and the attitude difference is determined as the attitude drift (X) that characterizes the error of the first sensor data. 40 ,Y 40 Z 40 ), that is:

[0083] (X 40 ,Y 40 Z 40 )=(X 31 ,Y 31 Z 31 )-(X 30 ,Y 30 Z 30 )

[0084] In summary, based on the unchanged relative attitude between the first device and the second device, the embodiments of this application obtain the observation for correcting sensor data errors, i.e., the attitude drift, without increasing the hardware cost.

[0085] Based on the determined attitude drift, the first device automatically calibrates the IMU data. Optionally, the automatic calibration algorithm can be a filtering algorithm such as the least squares method, gradient descent algorithm, Kalman filter algorithm, extended Kalman filter (EKF) algorithm, or unscented Kalman filter (UKF) algorithm. In this embodiment, the Kalman filter algorithm is used as an example to illustrate the automatic calibration process.

[0086] Step 510: Perform Kalman filtering based on the first real-time sensor data and / or the second sensor data and the attitude drift to obtain the attitude error.

[0087] The first real-time sensor data is actual measurement data with a certain degree of error. Correspondingly, the attitude drift is the measurement error calculated based on the first real-time sensor data. By performing Kalman filtering on the first real-time sensor data based on the attitude drift, a more accurate attitude error can be obtained through optimal estimation. This step may include the following sub-steps. Please refer to... Figure 5 The diagram illustrates a flowchart of an automatic correction calculation provided in an exemplary embodiment of this application. For the sake of brevity, this embodiment is described using attitude correction of first sensor data only; the method for attitude correction of second sensor data is the same as described below.

[0088] Step 511: Construct a state transition equation based on the first real-time sensor data. The state transition equation is used to characterize the state transition of the attitude error at adjacent time points.

[0089] In one possible implementation, to perform Kalman filtering on the first sensor data to complete sensor data correction, an error vector is first constructed as the system state vector. This error vector includes attitude error and sensor error. Based on this embodiment, the first device needs to calculate the first sensor data error using the Kalman filter algorithm. The first device constructs the error vector and uses it as the system state vector for optimal estimation. Then, the Kalman filter algorithm is used to obtain a more accurate error vector, i.e., a more accurate attitude error and sensor error. The aforementioned error vector can be expressed as:

[0090] X k =[φε] T

[0091] The error vector is a 6×1 vector, where k represents the current time, X kLet X represent the true value of the error vector at time k, φ represent the three-dimensional attitude angle error (i.e., attitude error), and ε represent the three-dimensional gyroscope error (i.e., sensor error). In the initial state, the initial error state is set to X0 = 0.

[0092] Furthermore, to construct the state transition equation representing the current system state, the first device constructs a state transition matrix based on the first real-time sensor data. In the KF process, the state transition matrix characterizes the probability that the system state, i.e., the error vector, will transition from the previous state to the next state. The aforementioned state transition matrix can be expressed as:

[0093]

[0094] Where k represents the current time, k-1 represents the previous time, and I 3×3 It is the identity matrix, 0 3×3 It is a zero matrix. Let β be the first real-time attitude data represented by the attitude matrix. g The corresponding time parameter in the first-order Markov process is taken as a constant value in this calculation. s The data sampling period is defined as follows. Since this calculation process involves matrix computation, the first device needs to calculate the attitude data (represented by Euler angles) from the first sensor data to obtain three rotation matrices corresponding to the three Euler angles, thus obtaining the aforementioned attitude matrix.

[0095] Based on the error vector and state transition matrix obtained in the above process, the first device can construct a state transition equation. In this embodiment, the first device acquires the first real-time sensor data based on the attitude correction cycle, that is, this application uses the discrete Kalman filter algorithm. Therefore, the state transition equation can be expressed as:

[0096] X k =Φ k / k-1 X k-1 +W k-1

[0097] Among them, X k-1 Let W be the error vector from the previous time step. k-1 The system noise is a Gaussian distribution with Qk-1 as the covariance, where Qk-1 is the system noise matrix set based on the device accuracy and experience of the IMU in the first device.

[0098] Step 512: Construct the observation equation using attitude drift as the observation.

[0099] The observation equation can be expressed as:

[0100] Z k =Hk X k +V k

[0101] Among them, Z k For the observation, attitude offset is used as the observation in this application, V k For observation noise, it refers to the noise with covariance R. k is a Gaussian distribution, where R is a _ ... k Based on the accuracy of the IMU device in the first device and empirical settings, H k The observation matrix is ​​represented in this application as follows:

[0102]

[0103] In the above embodiment, the first device calculates the attitude drift based on first sensor data and second sensor data expressed in Euler angles; that is, the attitude drift is expressed in Euler angles. Since this calculation process is a matrix calculation, the first device needs to convert the attitude drift into attitude data expressed as a rotation matrix, and then participate in the determination of the observation equation.

[0104] Step 513: Perform Kalman filtering based on the state transition equation and the observation equation to obtain the attitude error.

[0105] Based on the state transition equation and observation equation, the first device can perform Kalman filtering on the system state, i.e., the error vector, to obtain the optimally estimated error vector, which in turn yields a more accurate attitude error and sensor error.

[0106] In one possible implementation, the first device performs Kalman filtering based on the state transition equation and the observation equation. The Kalman filtering algorithm consists of two parts: a prediction phase and an update phase. In the prediction phase, the algorithm first calculates the prior state estimate (Priori) of the error at the current time based on the error vector and the state transition matrix. This prior state estimate, also known as the predicted value, can be expressed as:

[0107]

[0108] in, This is the state estimate from the previous time step (Previous). The algorithm then predicts the error covariance matrix for the current time step based on the error covariance matrix from the previous time step. This prediction method for the error covariance matrix can be expressed as:

[0109]

[0110] Among them, P k / k-1 The prior estimate of the error covariance matrix represents the prediction of the above values. Trust level, P k-1 Let be the covariance matrix of the previous time step.

[0111] During the update phase, the Kalman filter algorithm is based on the calculated attitude offset Z. k In other words, the error obtained from actual measurement determines the difference between the predicted value and the true value, i.e., the residual. And the confidence level of the residual is represented by the Kalman gain:

[0112]

[0113] Among them, K k For Kalman gain, This is the transpose of the observation matrix. Furthermore, the Kalman filter algorithm applies this to the predicted values. The update is performed, and the posterior state value, i.e., the estimated error vector, is obtained. This calculation process can be expressed as:

[0114]

[0115] in, This is the updated error vector, which is the target error vector that can more accurately represent the current attitude error. Then, the Kalman filter algorithm updates the error covariance matrix; the calculation process can be expressed as:

[0116] P k =(IK k H k )P k / k-1

[0117] Furthermore, the first device obtains the attitude error from the target error vector. Target error vector Includes three-dimensional attitude error and 3D sensor error Among them, the three-dimensional attitude error characterizes the error of the attitude data. Based on this attitude error, the attitude data of the first device can be corrected. Correspondingly, the three-dimensional sensor error characterizes the data error of the sensor in the first device, that is, the measurement error of the gyroscope. Based on this three-dimensional sensor error, the measurement data of the gyroscope can be corrected, thereby improving the accuracy of subsequent gyroscope data.

[0118] Step 520: Perform attitude correction based on attitude error to obtain the corrected first sensor data.

[0119] Based on the above attitude error The first device can correct the attitude of the first device represented by the data from the first sensor. The calculation process can be expressed as follows:

[0120]

[0121] in, The attitude matrix corrected at the current moment, This is the attitude matrix before the current time step. The first sensor data is represented by a rotation matrix. Optionally, the first device can perform mathematical operations on the rotation matrix to obtain the first sensor data represented by Euler angles.

[0122] Based on the Kalman filtering process described above, in constructing the error vector, both attitude error and sensor error are introduced, i.e., error vector X. k The first three dimensions constitute the attitude error, represented by Euler angles, and the error vector X k The latter three dimensions represent the three-axis data error in the first sensor data.

[0123] In one possible implementation, Kalman filtering is performed based on the first real-time sensor data and / or the second sensor data, as well as the attitude drift, to obtain the sensor error.

[0124] The Kalman filtering process is the same as step 510 and will not be repeated here. After Kalman filtering, a relatively accurate target error vector can be estimated. The target error vector includes sensor error.

[0125] Furthermore, sensor calibration is performed based on sensor error. The calibration process can be expressed as:

[0126]

[0127] Where gyro' represents the corrected sensor data, and gyro represents the original sensor data. By correcting the first sensor, the accuracy of its output data is improved. Even between two correction cycles, the first device can still output relatively accurate first sensor data, thus still obtaining a highly accurate attitude tracking result.

[0128] In summary, the first device calculates the observations required for the filtering process, i.e., the attitude drift, based on the data from the first and second sensors without adding any hardware. Based on the first sensor data and the attitude drift, the first device filters the first sensor data and corrects the attitude error and the sensor based on the target error vector obtained from the filtering, thereby improving the accuracy of attitude tracking.

[0129] Based on the attitude drift, the first device calculates the corrected first sensor data. This first sensor data can accurately represent the current attitude of the first device, that is, accurately represent the current head attitude of the user. Therefore, based on this first sensor data, the spatial sound effects can be adjusted according to the current head attitude of the user.

[0130] In one possible implementation, the first device can send calibrated first sensor data to the terminal device so that the terminal device can perform spatial audio processing based on the first sensor data. When the terminal device is an audio playback device, since it has a spatial audio processing module, the first device needs to send the calibrated first sensor data to the terminal device. The calibrated first sensor data can accurately characterize the current posture of the first device. Based on this posture, the terminal device can determine the current head posture of the user and perform spatial audio processing on the audio data accordingly. After spatial audio processing, the terminal device can obtain audio whose sound effects conform to the characteristics of sound propagation in space. For example, when the current head posture is turned to the right and the sound source is directly in front of the user, based on the characteristics of sound propagation, compared to facing the sound source, the user's left ear receives a higher sound intensity while the user's right ear receives a lower sound intensity. Therefore, during spatial audio processing, the sound intensity of the audio to be played in the left earphone should be appropriately amplified.

[0131] When experiencing spatial sound effects, users often use two paired headphones or over-ear headphones to achieve an immersive sound experience. Therefore, in addition to calibrating the first device, the second device also needs to calibrate its sensor data.

[0132] In one possible implementation, the first device sends first sensor data to the second device so that the second device can perform attitude correction based on the second sensor data and the first sensor data.

[0133] In spatial audio, based on the head posture and the positional relationship between the audio playback device and the device, the sound played in the left earphone corresponds to the sound effect that the left ear should hear, and the sound played in the right earphone corresponds to the sound effect that the right ear should hear. That is, the left and right earphones play different sounds, and both sounds should be audio obtained after spatial audio processing based on the earphone device's posture. Therefore, when the second device is an earphone paired with the first device, the second sensor data output by the second device needs to be calibrated to obtain a more accurate representation of the second device's posture.

[0134] Similar to the method described above for correcting the first sensor data output by the first device, when both the first and second devices are earphone devices and are paired with each other, the second device acquires the first sensor data transmitted by the first device and the second sensor data output by the second device itself, and calculates the attitude drift amount of the second device relative to the first device based on the two data, and then corrects the second sensor data according to the attitude drift amount to obtain the corrected second sensor data.

[0135] To enable data transmission between the first device and the second device, a data communication connection needs to be established between them. Regarding the method of establishing this data communication connection, depending on the different types of the first and second devices, any of the following methods can be used:

[0136] 1. When the first device and the second device are paired devices, the first device and the second device establish a data communication connection.

[0137] The pairing method between the first device and the second device can be based on a Wi-Fi address, string, etc., which is not limited in this application. For example, the first device is the left earbud of a pair of TWS (True Wireless Stereo) earbuds, and the second device is the right earbud of the same pair of TWS earbuds. The first device and the second device are paired devices. When the first device and the second device are in pairing mode, they can establish a data communication connection through Bluetooth, Wi-Fi, etc.

[0138] 2. When the first device and the second device are not paired devices, and the first device has established a data communication connection with the terminal device, and the second device has established a data communication connection with the terminal device, a data communication connection is established between the terminal device and the second device.

[0139] In this configuration, the first device and the second device establish a data communication connection with the same terminal device. In one possible implementation, the first device can detect the status of the second device through the terminal device. When the second device is being worn, the first device sends a data acquisition command to the terminal device, which then forwards the command to the second device. The second device then transmits second sensor data to the terminal device, which in turn transmits the second sensor data back to the first device, allowing the first device to acquire the second sensor data output by the second device. It should be noted that the data communication connection method between the first device and the terminal device, and between the second device and the terminal device, can be a Bluetooth link, WiFi, etc., and this application does not limit this to any particular method.

[0140] In an illustrative example, the first device is an earphone, and the second device is a smart necklace. The earphone and the smart necklace are not paired; no data communication connection is established between them. Correspondingly, a data communication connection is established between the earphone and a terminal device, and a data communication connection is also established between the smart necklace and the same terminal device. Therefore, the earphone can acquire data from the second sensor through the terminal device.

[0141] Please refer to Figure 6 This document illustrates a flowchart of an attitude correction method provided in an exemplary embodiment of this application. The embodiments of this application use this method for... Figure 2 Taking the terminal device shown as an example, the method may include the following steps.

[0142] Step 601: The terminal determines the attitude drift amount based on the first sensor data and the second sensor data. The first sensor data is the sensor data of the first device, and the second sensor data is the sensor data of the second device. The attitude drift amount is used to characterize the attitude drift of the first device relative to the second device.

[0143] In one possible implementation, to determine the amount of attitude drift, in the initial state—that is, when both the first and second devices are detected to be in a wearing state and attitude tracking is required—the terminal device receives first initial sensor data sent by the first device and second initial sensor data sent by the second device. During attitude tracking, when the attitude correction cycle is reached, the terminal device receives first real-time sensor data sent by the first device and second real-time sensor data sent by the second device to correct the first real-time sensor data.

[0144] In one possible implementation, upon receiving first initial sensor data and second initial sensor data, the terminal device determines the initial attitude of the first device as (X) based on the first initial sensor data. 10 ,Y 10 Z 10 ), and based on the second initial sensor data, determine the initial attitude of the second device as (X). 20 ,Y 20 Z 20 The above postures are all represented in Euler angles. Optionally, the first device can also represent its posture using rotation matrices, quaternions, rotation vectors, etc. The terminal device will then represent the initial posture (X) of the first device. 10 ,Y 10 Z 10 ) and the initial attitude (X) of the second device 20 ,Y 20 Z 20 The difference is calculated to determine the initial relative attitude (X).30 ,Y 30 Z 30 ), that is:

[0145] (X 30 ,Y 30 Z 30 )=(X 10 ,Y 10 Z 10 )-(X 20 ,Y 20 Z 20 )

[0146] Furthermore, upon receiving the first real-time sensor data and the second real-time sensor data, the terminal device determines the real-time attitude of the first device as (X) based on the first real-time sensor data. 11 ,Y 11 Z 11 ), and based on the second real-time sensor data, determine the real-time attitude of the second device as (X). 21 ,Y 21 Z 21 The terminal device will display the real-time attitude (X) of the first device. 11 ,Y 11 Z 11 ) and the real-time attitude (X) of the second device 21 ,Y 21 Z 21 The difference is calculated to determine the real-time relative attitude (X). 31 ,Y 31 Z 31 ), that is:

[0147] (X 31 ,Y 31 Z 31 )=(X 11 ,Y 11 Z 11 )-(X 21 ,Y 21 Z 21 )

[0148] Based on the unchanged relative attitude of the first and second devices, the terminal device can determine the real-time relative attitude (X). 31 ,Y 31 Z 31 ) and initial relative attitude (X) 30 ,Y 30 Z 30 The deviation between the two devices is caused by the sensor error of the first device.

[0149] Then, the terminal device subtracts the initial relative attitude from the real-time relative attitude to obtain the attitude drift, which is:

[0150] (X 40 ,Y 40 Z 40 )=(X 31 ,Y 31 Z 31 )-(X 30 ,Y 30 Z 30 )

[0151] Step 602: The terminal performs attitude correction based on the attitude drift amount to obtain the corrected first sensor data and / or second sensor data.

[0152] This step is the same as step 302, and will not be repeated here.

[0153] In summary, the terminal device can receive first sensor data transmitted by the first device and second sensor data transmitted by the second device, and then perform calculations on the two data to obtain the attitude drift amount used to correct the first sensor data, and correct the first sensor data. In this embodiment of the application, when both the first device and the second device establish a communication connection with the same terminal device, the terminal device can be used to perform calculations on the sensor data. The first device and the second device do not need to have corresponding data processing modules to achieve sensor data correction. Without increasing hardware costs, the requirement for the processing module of the first device is reduced, while the accuracy of head attitude tracking is improved.

[0154] In the scenario where the terminal device performs attitude correction on the first sensor data, after obtaining the corrected first sensor data, the terminal device performs spatial audio processing based on the corrected first sensor data in order to obtain audio with sound effects that conform to the characteristics of spatial sound propagation.

[0155] In spatial audio processing, the spatial audio effect of the audio played on the first device corresponds to the current posture of the first device, and correspondingly, the spatial audio effect of the audio played on the second device corresponds to the current posture of the second device. Furthermore, when the first and second devices are paired, the terminal device needs to perform posture correction on the second sensor data based on the posture drift amount to obtain the corrected second sensor data.

[0156] In one possible implementation, in the initial state, the terminal device determines the initial orientation of the first device as (X). 10 ,Y 10 Z 10 ), and determine the initial attitude of the second device as (X 20 ,Y20 Z 20 Under the condition of satisfying the attitude correction period, the terminal device determines the real-time attitude of the first device as (X). 11 ,Y 11 Z 11 ), and determine the real-time attitude of the second device as (X 21 ,Y 21 Z 21 To perform attitude correction on the second sensor data, the terminal calculates the initial relative attitude as follows:

[0157] (X 32 ,Y 32 Z 32 )=(X 20 ,Y 20 Z 20 )-(X 10 ,Y 10 Z 10 )

[0158] Accordingly, the terminal device calculates the real-time relative attitude as follows:

[0159] (X 33 ,Y 33 Z 33 )=(X 21 ,Y 21 Z 21 )-(X 11 ,Y 11 Z 11 )

[0160] Based on the initial relative attitude and the real-time relative attitude, the terminal device calculates and determines the attitude drift as the difference between the two, that is, the attitude drift is:

[0161] (X 41 ,Y 41 Z 41 )=(X 33 ,Y 33 Z 33 )-(X 32 ,Y 32 Z 32 )

[0162] The second device can correct the second sensor data based on the attitude drift amount to obtain the corrected second sensor data. That is, the attitude drift amount can be used as an observation, and the real-time attitude data of the second device can be used as an attitude matrix to participate in the Kalman filter calculation.

[0163] When both the first device and the second device are earphone devices, and the first device and the second device are paired devices, the first device performs sensor data calibration based on the first sensor data and the second sensor data. Please refer to... Figure 7 It illustrates a flowchart of attitude correction provided in an exemplary embodiment of this application, in which this embodiment uses the method for Figure 1 The method, described in the first device shown, may include the following steps:

[0164] Step 701: If the first device and the second device are paired devices, the first device and the second device establish a data communication connection.

[0165] Step 702: When both the first device and the second device are in the wearing state and there is a need for posture tracking, the first device acquires its own first initial sensor data and receives the second initial sensor data sent by the second device.

[0166] Step 703: When both the first device and the second device are in the wearing state and there is a need for posture tracking, the second device acquires its own second initial sensor data and receives the first initial sensor data sent by the first device.

[0167] Step 704: When the attitude correction cycle is reached, the first device acquires first real-time sensor data and second real-time sensor data transmitted by the second device, so as to correct the first real-time sensor data.

[0168] Step 705: When the attitude correction cycle is reached, the second device acquires the second real-time sensor data and the first real-time sensor data transmitted by the first device, so as to correct the second real-time sensor data.

[0169] Step 706: Based on the first sensor data and the second sensor data, the first device determines a first attitude drift amount, which is used to characterize the attitude drift of the first device relative to the second device determined based on the sensor data.

[0170] In one possible implementation, based on first initial sensor data and second initial sensor data, the first device determines a first initial relative attitude. The first device determines its own initial attitude in the initial state as (X... 10 ,Y 10 Z 10 ), and determine the initial attitude of the second device in the initial state as (X). 20 ,Y 20 Z 20 Then, the first device subtracts its own initial attitude from the initial attitude of the second device to determine the first initial relative attitude as (X).30 ,Y 30 Z 30 ), that is:

[0171] (X 30 ,Y 30 Z 30 )=(X 10 ,Y 10 Z 10 )-(X 20 ,Y 20 Z 20 )

[0172] Furthermore, based on the first real-time sensor data and the second real-time sensor data, the first device determines the first real-time relative attitude.

[0173] In one possible implementation, based on the first real-time sensor data, the first device can determine the real-time attitude of the first device as (X). 11 ,Y 11 Z 11 ), and based on the second real-time sensor data, determine the real-time attitude of the second device as (X). 21 ,Y 21 Z 21 Based on the aforementioned attitude data, the first device obtains the first real-time relative attitude (X) by subtracting the values. 31 ,Y 31 Z 31 ), that is:

[0174] (X 31 ,Y 31 Z 31 )=(X 11 ,Y 11 Z 11 )-(X 21 ,Y 21 Z 21 )

[0175] Wherein, the first attitude drift is the attitude difference between the first real-time relative attitude and the first initial relative attitude, and the first attitude drift is (X 40 ,Y 40 Z 40 The calculation method can be expressed as:

[0176] (X 40 ,Y 40 Z 40 )=(X 31 ,Y 31 Z 31 )-(X 30 ,Y 30 Z 30)

[0177] Step 707: Based on the first sensor data and the second sensor data, the second device determines a second attitude drift amount, which is used to characterize the drift of the second device relative to the first device based on the sensor data.

[0178] In one possible implementation, the second device determines a second initial relative attitude based on first initial sensor data and second initial sensor data.

[0179] Based on the initial attitude of the first device (X) 10 ,Y 10 Z 10 ), and the initial attitude of the second device is (X 20 ,Y 20 Z 20 Then, the second device calculates the difference between its own initial attitude and the first device's initial attitude to determine the second initial relative attitude as (X). 32 ,Y 32 Z 32 ), that is:

[0180] (X 32 ,Y 32 Z 32 )=(X 20 ,Y 20 Z 20 )-(X 10 ,Y 10 Z 10 )

[0181] Furthermore, based on the first real-time sensor data and the second real-time sensor data, the second device determines the second real-time relative attitude.

[0182] Based on the real-time attitude of the first device (X) 11 ,Y 11 Z 11 ), and the real-time attitude of the second device is (X 21 ,Y 21 Z 21 The second device subtracts the two attitude data to obtain the second real-time relative attitude as (X). 33 ,Y 33 Z 33 ), that is:

[0183] (X 33 ,Y 33 Z 33 )=(X 21 ,Y 21 Z 21 )-(X11 ,Y 11 Z 11 )

[0184] Wherein, the second attitude drift is the attitude difference between the second real-time relative attitude and the second initial relative attitude, and the second attitude drift is (X 41 ,Y 41 Z 41 The calculation method can be expressed as:

[0185] (X 41 ,Y 41 Z 41 )=(X 33 ,Y 33 Z 33 )-(X 32 ,Y 32 Z 32 )

[0186] Step 708: Perform attitude correction on the first sensor data based on the first sensor data and the first attitude drift amount to obtain the corrected first sensor data.

[0187] Taking attitude correction of the first sensor data using Kalman filtering as an example, the first device determines an attitude matrix based on the real-time attitude data represented by the first real-time sensor data, and then constructs a state transition equation based on this attitude matrix. Correspondingly, the first device constructs an observation equation using the first attitude drift as an observation, and then performs Kalman filtering based on the aforementioned state transition equation and observation equation to obtain a target error vector. Finally, the first sensor data is corrected based on this target error vector.

[0188] Furthermore, the first sensor error is obtained based on the first sensor data and the first attitude drift, and the first sensor is corrected based on the first sensor error so that the first sensor can still output relatively accurate sensor data between two correction cycles, thereby improving the accuracy of the attitude tracking results.

[0189] Step 709: Perform attitude correction on the second sensor data based on the second sensor data and the second attitude drift amount to obtain the corrected second sensor data.

[0190] When using Kalman filtering to correct the attitude of the second sensor data, the second device determines its attitude matrix based on its real-time attitude data, and then constructs a state transition equation based on this attitude matrix. Correspondingly, the second device uses the second attitude drift as an observation to construct an observation equation, and performs Kalman filtering based on the aforementioned state transition equation and observation equation to obtain the target error vector. The second sensor data is then corrected based on this target error vector.

[0191] Furthermore, the second sensor error is obtained based on the second sensor data and the second attitude drift, and the second sensor is corrected based on the second sensor error.

[0192] Step 710: The terminal device performs spatial sound effect processing on the audio based on the corrected first sensor data transmitted by the first device and the corrected second sensor data transmitted by the second device.

[0193] Please refer to Figure 8 The diagram illustrates a structural block diagram of an attitude correction device provided in an exemplary embodiment of this application, the device comprising:

[0194] The first determining module 801 is used to determine the attitude drift amount based on the first sensor data of the first device and the second sensor data of the second device, wherein the attitude drift amount is used to characterize the attitude drift of the first device relative to the second device.

[0195] The first attitude correction module 802 is used to perform attitude correction based on the attitude drift amount to obtain the corrected first sensor data and / or second sensor data.

[0196] Optionally, the first determining module 801 is used for:

[0197] The initial relative attitude is determined based on the first initial sensor data and the second initial sensor data;

[0198] Based on the first real-time sensor data and the second real-time sensor data, the real-time relative attitude is determined.

[0199] The attitude difference between the real-time relative attitude and the initial relative attitude is determined as the attitude drift.

[0200] The optional device further includes an acquisition module for:

[0201] When the first device is in a wearing state and there is a need for posture tracking, the first initial sensor data of the first device is acquired, and the second initial sensor data sent by the second device is received. The second initial sensor data is sent by the second device when it is in a wearing state.

[0202] During attitude tracking, when the attitude correction cycle is reached, the first real-time sensor data of the first device is acquired, and the second real-time sensor data sent by the second device is received.

[0203] Optionally, the first attitude correction module 802 is used for:

[0204] Kalman filtering is performed based on the first real-time sensor data and / or the second sensor data, as well as the attitude drift, to obtain the attitude error.

[0205] The attitude is corrected based on the attitude error to obtain the corrected first sensor data and / or second sensor data.

[0206] Optionally, the first attitude correction module 802 is used for:

[0207] Kalman filtering is performed based on the first real-time sensor data and / or the second sensor data, as well as the attitude drift, to obtain the sensor error.

[0208] Sensor calibration is performed based on the sensor error. Optionally, the device further includes a transmitting module for:

[0209] The corrected first sensor data and / or second sensor data are sent to the terminal device so that the terminal device can perform spatial sound processing based on the first sensor data and / or second sensor data.

[0210] Optionally, the sending module is further configured to:

[0211] The first sensor data is sent to the second device so that the second device can perform attitude correction based on the second sensor data and the first sensor data.

[0212] Optionally, the acquisition module is further configured to:

[0213] When the first device and the second device are paired devices, the first device and the second device establish a data communication connection;

[0214] When the first device and the second device are not paired devices, and the first device has established a data communication connection with the terminal device, and the second device has established a data communication connection with the terminal device, a data communication connection is established between the terminal device and the second device.

[0215] The data communication connection is used to transmit data between the first device and the second device.

[0216] In summary, the first device acquires its own first sensor data through the acquisition module and acquires the second sensor data output by the second device. Then, it uses the first determination module to calculate the attitude drift amount based on the first sensor data and the second sensor data. Without increasing hardware costs, the error observation is determined based on the attitude drift amount of the first device relative to the second device.

[0217] Furthermore, based on the attitude drift, the first device corrects the first sensor data and / or the second sensor data through the first attitude correction module, and obtains the corrected first sensor data and / or the second sensor data that can accurately represent the current attitude of the first device, thereby determining the current user attitude, improving the accuracy of attitude tracking, and enhancing the realism of spatial sound effects when the first device is an earphone.

[0218] Please refer to Figure 9 The diagram illustrates a structural block diagram of an attitude correction device provided in an exemplary embodiment of this application, the device comprising:

[0219] The second determining module 901 is used to determine the attitude drift amount based on the first sensor data and the second sensor data. The first sensor data is the sensor data of the first device, and the second sensor data is the sensor data of the second device. The attitude drift amount is used to characterize the attitude drift of the first device relative to the second device.

[0220] The second attitude correction module 902 is used to perform attitude correction based on the attitude drift amount to obtain the corrected first sensor data and / or second sensor data.

[0221] Optionally, the device further includes a processing module, which, in the case of a headphone device, is used to perform spatial sound effect processing based on the corrected first sensor data and / or second sensor data.

[0222] Please refer to Figure 10 This diagram illustrates a structural block diagram of a terminal device 1000 provided in an exemplary embodiment of this application. The terminal device 1000 may be a portable mobile terminal, such as a smartphone, tablet computer, Moving Picture Experts Group Audio Layer III (MP3) player, or Moving Picture Experts Group Audio Layer IV (MP4) player. The terminal 1000 may also be referred to as a user device, portable terminal, or other names.

[0223] Typically, terminal device 1000 includes a processor 1001 and a memory 1002.

[0224] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1001 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1001 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0225] The memory 1002 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1002 are used to store at least one instruction, which is executed by the processor 1001 to implement the method provided in the embodiments of this application.

[0226] Optionally, the terminal device 1000 may also include a communication component 1003.

[0227] The communication component 1003 is used to establish a communication connection with the first device and the second device.

[0228] Please refer to Figure 11 The diagram illustrates a structural block diagram of an earphone device 1100 provided in an exemplary embodiment of this application. The earphone device 1100 may be a TWS (True Wireless Stereo) headset, a smart over-ear headphone, or the like.

[0229] The headphone device 1100 includes: a processor 1101 and a memory 1102.

[0230] Processor 1101 may include one or more processing cores, such as a 4-core processor or an 8-core processor. Processor 1001 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 1101 may include a main processor and a coprocessor. The main processor is used to process data in the wake-up state and may be a microcontroller unit (MCU). The coprocessor is a low-power processor used to process data in the standby state.

[0231] The memory 1102 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 are used to store at least one instruction, which is executed by the processor 1101 to implement the method provided in the embodiments of this application.

[0232] Optionally, the headphone device 1100 may also include: a speaker 1103, an IMU sensor 1104, and a communication component 1105.

[0233] The speaker 1103 is used to play sound processed by spatial sound effects; the IMU sensor 1104 is an attitude sensor composed of a three-axis accelerometer and a three-axis gyroscope, used to detect and determine the attitude of the headphone device 1100; the communication component 1105 can be a Bluetooth component, a serial communication component, etc., and the headphone device 1100 can establish a data communication connection with the terminal device or other wearable smart devices through the communication component 1105.

[0234] This application embodiment also provides a chip, which includes a processor, wherein the processor is configured to determine an attitude drift amount based on first sensor data of a first device and second sensor data of a second device, the attitude drift amount being used to characterize the attitude drift of the first device relative to the second device; and to perform attitude correction based on the attitude drift amount to obtain corrected first sensor data and / or second sensor data.

[0235] This application also provides a computer-readable storage medium storing at least one program that is executed by a processor to implement the attitude correction method as described in the above embodiments.

[0236] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the attitude correction method provided in the above embodiments.

[0237] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0238] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An attitude correction method, characterized in that, The method is used in a first device, and the method includes: When the first device is in a wearing state and there is a need for posture tracking, the first initial sensor data of the first device is acquired, and the second initial sensor data sent by the second device is received. The second initial sensor data is sent by the second device when it is in a wearing state. Based on the first initial sensor data and the second initial sensor data, the initial relative attitude is determined; During attitude tracking, when the attitude correction cycle is reached, the first real-time sensor data of the first device is acquired, and the second real-time sensor data sent by the second device is received. Based on the first real-time sensor data and the second real-time sensor data, the real-time relative attitude is determined; The attitude difference between the real-time relative attitude and the initial relative attitude is determined as the attitude drift, which is used to characterize the attitude drift of the first device relative to the second device. Attitude correction is performed based on the attitude drift amount to obtain the corrected first sensor data of the first device and / or the second sensor data of the second device.

2. The method according to claim 1, characterized in that, The attitude correction based on the attitude drift amount to obtain the corrected first sensor data of the first device and / or the second sensor data of the second device includes: Kalman filtering is performed based on the first real-time sensor data and / or the second sensor data, as well as the attitude drift, to obtain the attitude error. The attitude is corrected based on the attitude error to obtain the corrected first sensor data and / or second sensor data.

3. The method according to claim 2, characterized in that, The method further includes: Kalman filtering is performed based on the first real-time sensor data and / or the second sensor data, as well as the attitude drift, to obtain the sensor error. Sensor calibration is performed based on the sensor error.

4. The method according to any one of claims 1 to 3, characterized in that, The first device is an earphone device, and the method further includes: The corrected first sensor data and / or second sensor data are sent to the terminal device so that the terminal device can perform spatial sound processing based on the first sensor data and / or second sensor data.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The first sensor data is sent to the second device so that the second device can perform attitude correction based on the second sensor data and the first sensor data.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: When the first device and the second device are paired devices, the first device and the second device establish a data communication connection; When the first device and the second device are not paired devices, and the first device has established a data communication connection with the terminal device, and the second device has established a data communication connection with the terminal device, a data communication connection is established between the terminal device and the second device. The data communication connection is used to transmit data between the first device and the second device.

7. An attitude correction method, characterized in that, The method is used in a terminal device, and the method includes: When the first device is in a wearing state and there is a need for posture tracking, the device receives first initial sensor data sent by the first device and second initial sensor data sent by the second device, wherein the second initial sensor data is sent by the second device while in a wearing state. Based on the first initial sensor data and the second initial sensor data, the initial relative attitude is determined; During attitude tracking, when the attitude correction cycle is reached, the system receives first real-time sensor data sent by the first device and second real-time sensor data sent by the second device. Based on the first real-time sensor data and the second real-time sensor data, the real-time relative attitude is determined; The attitude difference between the real-time relative attitude and the initial relative attitude is determined as the attitude drift, which is used to characterize the attitude drift of the first device relative to the second device. Attitude correction is performed based on the attitude drift amount to obtain the corrected first sensor data of the first device and / or the second sensor data of the second device.

8. The method according to claim 7, characterized in that, The first device is an earphone device; The method further includes: Spatial sound processing is performed based on the corrected data from the first sensor and / or the second sensor.

9. An attitude correction device, characterized in that, The device is used in a first apparatus, the device comprising: The acquisition module is used to acquire first initial sensor data of the first device when the first device is in a wearing state and there is a need for posture tracking, and to receive second initial sensor data sent by the second device, wherein the second initial sensor data is sent by the second device when it is in a wearing state. The first determining module is used to determine the initial relative attitude based on the first initial sensor data and the second initial sensor data; The acquisition module is also used to acquire first real-time sensor data of the first device and receive second real-time sensor data sent by the second device when the attitude correction cycle is reached during the attitude tracking process. The first determining module is further configured to determine the attitude difference between the real-time relative attitude and the initial relative attitude as the attitude drift amount, wherein the attitude drift amount is used to characterize the attitude drift of the first device relative to the second device; The first attitude correction module performs attitude correction based on the attitude drift amount to obtain the corrected first sensor data of the first device and / or the second sensor data of the second device.

10. An attitude correction device, characterized in that, The device is used in a terminal device, and the device includes: The second determining module is used to receive first initial sensor data sent by the first device and receive second initial sensor data sent by the second device when the first device is in a wearing state and there is a need for posture tracking. The second initial sensor data is sent by the second device when it is in a wearing state. Based on the first initial sensor data and the second initial sensor data, the initial relative attitude is determined; During attitude tracking, when the attitude correction cycle is reached, the system receives first real-time sensor data sent by the first device and second real-time sensor data sent by the second device. Based on the first real-time sensor data and the second real-time sensor data, the real-time relative attitude is determined; The attitude difference between the real-time relative attitude and the initial relative attitude is determined as the attitude drift, which is used to characterize the attitude drift of the first device relative to the second device. The second attitude correction module is used to perform attitude correction based on the attitude drift amount to obtain the corrected first sensor data of the first device and / or the second sensor data of the second device.

11. A chip, characterized in that, The chip includes a processor, the processor being configured to: When the first device is in a wearing state and there is a need for posture tracking, the first initial sensor data of the first device is acquired, and the second initial sensor data sent by the second device is received. The second initial sensor data is sent by the second device when it is in a wearing state. Based on the first initial sensor data and the second initial sensor data, the initial relative attitude is determined; During attitude tracking, when the attitude correction cycle is reached, the first real-time sensor data of the first device is acquired, and the second real-time sensor data sent by the second device is received. Based on the first real-time sensor data and the second real-time sensor data, the real-time relative attitude is determined; The attitude difference between the real-time relative attitude and the initial relative attitude is determined as the attitude drift, which is used to characterize the attitude drift of the first device relative to the second device. Attitude correction is performed based on the attitude drift amount to obtain the corrected first sensor data of the first device and / or the second sensor data of the second device.

12. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the attitude correction method as described in any one of claims 1 to 8.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the attitude correction method as described in any one of claims 1 to 6, or to implement the attitude correction method as described in any one of claims 7 to 8.

14. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the attitude correction method as described in any one of claims 1 to 6, or to perform the attitude correction method as described in any one of claims 7 to 8.

Images