Magnetometer calibration method and device of augmented reality equipment and augmented reality equipment

Abstract

The embodiment of the invention discloses a magnetometer calibration method and device for augmented reality equipment and the augmented reality equipment, and the method comprises the steps: carrying out the collection of original data through a magnetometer of the augmented reality equipment in response to the head movement of a target user wearing the augmented reality equipment, acquiring gyroscope data corresponding to the original data through a gyroscope of the augmented reality equipment; generating a target to-be-solved function for minimizing measurement noise based on a magnetometer measurement model, wherein the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise; the target to-be-solved function is solved through the original data and the gyroscope data, target magnetometer parameters are obtained, and the target magnetometer parameters are solving results for the magnetometer parameters; and calibrating the magnetometer based on the target magnetometer parameter. The difficulty of user operation during calibration is simplified, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of extended reality, and specifically to a magnetometer calibration method and apparatus for an extended reality device, and an extended reality device. Background Art

[0002] Extended Reality (XR) is a comprehensive term that covers virtual reality (VR), augmented reality (AR) and mixed reality (MR). XR technology expands human perception and interaction methods, interacting with the real world through digital technology to create virtual, augmented and mixed reality experiences. XR provides a more intuitive and convenient content presentation experience through virtual reality scenes or extended reality scenes, and is developing rapidly in entertainment, games, education, medical care, enterprise production and social communication, showing great potential and influence.

[0003] Accurate calibration of the magnetometer of an extended reality device is very important. The accuracy of the magnetometer sensor affects many functions and user experience of the extended reality device, including positioning navigation, gesture interaction, and motion analysis.

[0004] However, the existing magnetometer calibration relies heavily on the diversity and richness of the measurement data. It requires movement and data collection at various angles based on the "8-figure calibration" principle, and even relies on gravity parameters in a "static" state, making the calibration operation difficult and the user experience needs to be improved. Summary of the invention

[0005] The embodiments of the present application provide a magnetometer calibration method and apparatus for an extended reality device, an extended reality device, and a computer-readable storage medium, which can reduce the difficulty of magnetometer calibration, facilitate magnetometer calibration, and improve user experience.

[0006] The present application provides a magnetometer calibration method for an extended reality device, which is applied to the extended reality device. The method includes: In response to a head movement of a target user wearing the extended reality device, raw data is acquired through a magnetometer of the extended reality device, and gyroscope data corresponding to the raw data is acquired through a gyroscope of the extended reality device; generating a target function to be solved for minimizing measurement noise based on a magnetometer measurement model, wherein the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise; Solving the target function to be solved by using the original data and the gyroscope data to obtain target magnetometer parameters, wherein the target magnetometer parameters are solution results for the magnetometer parameters; The magnetometer is calibrated based on the target magnetometer parameters.

[0007] Accordingly, an embodiment of the present application further provides a magnetometer calibration device for an extended reality device, which is applied to the extended reality device, and the device includes: an acquisition module, configured to acquire raw data through a magnetometer of the extended reality device in response to a head movement of a target user wearing the extended reality device, and acquire gyroscope data corresponding to the raw data through a gyroscope of the extended reality device; A construction module, for generating a target function to be solved for minimizing measurement noise based on a magnetometer measurement model, wherein the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise; A solution module, used for solving the target function to be solved by using the original data and the gyroscope data to obtain target magnetometer parameters, wherein the target magnetometer parameters are solution results for the magnetometer parameters; A calibration module is used to calibrate the magnetometer based on the target magnetometer parameters.

[0008] Optionally, in some embodiments of the present application, a target function to be solved for minimizing measurement noise is generated based on a magnetometer measurement model, including: determining an initial measurement noise function for representing magnetometer measurement noise according to the magnetometer measurement model; Performing an identity transformation on the initial measurement noise function to obtain a minimized measurement noise function; The magnetometer parameters in the minimized measurement noise function are processed in a block manner to obtain a target function to be solved.

[0009] Optionally, in some embodiments of the present application, the gyroscope data includes angular velocity data, and the target function to be solved is solved by using the original data and the gyroscope data to obtain target magnetometer parameters, including: Calculate a time period rotation vector according to the angular velocity data of the sampling period, and convert the time period rotation vector into a time period rotation matrix; Multiplying the time period rotation matrices of each sampling period to obtain the rotation information corresponding to the head movement; The target function to be solved is solved based on the preset constraints through the original data and the rotation information to obtain the target magnetometer parameters.

[0010] Optionally, in some embodiments of the present application, before solving the target function to be solved by using the original data and the gyroscope data to obtain the target magnetometer parameters, the method further includes: Performing bidirectional filtering on the original data to obtain target data, or, The target function to be solved is used to perform random sampling iterative optimization processing on the original data to obtain target data, or, Performing bidirectional filtering on the original data to obtain filtered data, and performing random sampling iterative optimization processing on the filtered data using the target function to be solved to obtain target data; The step of solving the target function to be solved by using the original data and the gyroscope data to obtain target magnetometer parameters includes: The target function to be solved is solved by using the target data and the gyroscope data to obtain target magnetometer parameters.

[0011] Optionally, in some embodiments of the present application, bidirectional filtering is performed on the original data to obtain target data, including: Determining a calibration frequency of the magnetometer, wherein the calibration frequency refers to a movement frequency of the head movement; Determining a target order and a target frequency according to the calibration frequency; The original data is subjected to bidirectional filtering processing according to the target order and the target frequency to obtain target data.

[0012] Optionally, in some embodiments of the present application, the target function to be solved is used to perform random sampling iterative optimization processing on the original data to obtain target data, including: Generate at least one set of test data corresponding to the original data according to a preset ratio, wherein each set of test data includes a first set of data randomly selected based on the preset ratio and a second set of data in the original data except the first set of data; For each group of test data, the magnetometer parameters to be optimized of the magnetometer are calculated according to the first set of data in the test data and by the target function to be solved, and the target pure data with a measurement error less than an error threshold in the second set of data of the test data is determined according to the magnetometer parameters to be optimized; The second set of data containing the most target pure data is screened out, and the first set of data corresponding to the second set of data and the target pure data of the second set of data are used as target data.

[0013] Optionally, in some embodiments of the present application, after calibrating the magnetometer based on the target magnetometer parameter, the method further includes: In response to obtaining real-time magnetometer data measured by the magnetometer, calculating real-time error data corresponding to the real-time magnetometer data by using the target magnetometer parameters; Calculating the difference between the real-time magnetometer data and the real-time error data to obtain actual magnetometer data; A magnetic application is performed based on the actual magnetometer data, wherein the magnetic application includes at least one of positioning navigation, gesture interaction, motion analysis, or environment detection.

[0014] In a third aspect, an embodiment of the present application further provides an extended reality device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps in the magnetometer calibration method of the extended reality device are implemented.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the magnetometer calibration method of the above-mentioned extended reality device are implemented.

[0016] In a fifth aspect, the embodiments of the present application further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0017] The extended reality device of the embodiment of the present application responds to the head movement of the target user wearing the extended reality device, collects raw data through the magnetometer of the extended reality device, and collects gyroscope data corresponding to the raw data through the gyroscope of the extended reality device, generates a target function to be solved for minimizing measurement noise based on a magnetometer measurement model, the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise, solves the target function to be solved through the raw data and the gyroscope data, obtains target magnetometer parameters, the target magnetometer parameters are the solution results of the magnetometer parameters, and the magnetometer is calibrated based on the target magnetometer parameters.

[0018] The magnetometer parameters are solved based on a magnetometer measurement model including magnetometer parameters and gyroscope parameters, so that the magnetometer calibration is realized in combination with the gyroscope.

[0019] In the related prior art, when a separate magnetometer is used for calibration, it is necessary to collect magnetometer data based on the "8-character standard" to obtain more diverse and rich magnetometer data, so as to meet the accuracy requirements of the separate magnetometer calibration. However, the embodiment of the present application combines gyroscope parameters to calibrate the magnetometer parameters, and there is no need to collect magnetometer data based on the "8-character standard", which reduces the requirements for the data relied on for calibration, that is, simplifies the requirements for device movement during magnetometer calibration, simplifies the difficulty of user operation during calibration, and improves user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a schematic diagram of a scenario in which an extended reality device provided in an embodiment of the present application executes a magnetometer calibration method of the extended reality device; Figure 2 It is a flowchart of a magnetometer calibration method for an extended reality device provided in an embodiment of the present application; Figure 3 is a coordinate schematic diagram of a magnetometer calibration method for an extended reality device provided in an embodiment of the present application; Figure 4 It is a schematic diagram of the movement sequence of the extended reality device when collecting raw data in the magnetometer calibration method of the extended reality device provided in an embodiment of the present application; Figure 5 is a schematic diagram of a burr signal contained in the raw data collected by the magnetometer provided in an embodiment of the present application; Figure 6 It is a structural schematic diagram of a magnetometer calibration device for an extended reality device provided in an embodiment of the present application; Figure 7 It is a schematic diagram of the structure of the extended reality device provided in an embodiment of the present application.

[0022] Description of Figure Numbers: 10-extended reality device; 201-acquisition module; 202-construction module; 203-solution module; 204-calibration module; 301-processor; 302-memory; 303-power supply; 304-input unit. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0024] The embodiments of the present application provide a magnetometer calibration method and device for an extended reality device, an extended reality device, and a computer-readable storage medium. Specifically, the embodiments of the present application provide a magnetometer calibration device suitable for an extended reality device, and the extended reality device includes but is not limited to a head-mounted display, wearable glasses, and the extended reality device can also be an integrated extended reality device built into a computing processing unit, or a split extended reality device external to the computing processing unit. Among them, the extended reality device includes but is not limited to an airborne optical display system, i.e., a head-up display system, applied to vehicles such as aircraft, automobiles, and ships, for example, an AR-HUD (Augmented reality hud-up display) mounted on an intelligent networked car, an extended reality application applied to handheld mobile devices such as mobile phones, laptops, and tablets (such as extended reality games, virtual tourism, telemedicine, or virtual experiments, etc.), and a near-eye display system applied to wearable devices such as head-mounted displays and smart glasses.

[0025] See also Figure 1 , Figure 1 is a schematic diagram of a scenario in which an extended reality device according to an embodiment of the present application performs a magnetometer calibration method of the extended reality device, such as Figure 1 , taking the extended reality device as optical transmission wearable smart AR glasses as an example, the specific execution process of the extended reality device executing the magnetometer calibration method of the extended reality device is as follows: The extended reality device 10 responds to the head movement of the target user wearing the extended reality device, obtains raw data through the magnetometer of the extended reality device, and obtains gyroscope data corresponding to the raw data through the gyroscope of the extended reality device, and generates a target function to be solved for minimizing measurement noise based on the magnetometer measurement model, wherein the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise, solves the target function to be solved through the raw data and gyroscope data to obtain target magnetometer parameters, and calibrates the magnetometer of the extended reality device 10 based on the target magnetometer parameters.

[0026] Among them, the magnetometer parameters are solved based on the magnetometer measurement model including magnetometer parameters and gyroscope parameters, and the magnetometer calibration is realized by combining with the gyroscope. When the magnetometer parameters are calibrated by combining with the gyroscope parameters, it is not necessary to calibrate the magnetometer separately. The magnetometer data must be collected based on the "8-character standard", which reduces the requirements for the data relied on for calibration, that is, simplifies the requirements for device movement during magnetometer calibration, simplifies the difficulty of user operation during calibration, and improves the user experience.

[0027] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0028] See also Figure 2 , Figure 2 A flowchart of a magnetometer calibration method for an extended reality device provided in an embodiment of the present application. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown in the accompanying drawings. Specifically, the magnetometer calibration method for an extended reality device is applied to an extended reality device, and the specific process of the method is as follows: 101. In response to a head movement of a target user wearing an extended reality device, raw data is acquired through a magnetometer of the extended reality device, and gyroscope data corresponding to the raw data is acquired through a gyroscope of the extended reality device.

[0029] It should be noted that head movement refers to low-frequency movements of the target user's head when wearing an extended reality device, such as raising and shaking the head, etc., where low frequency refers to a slower frequency of head movement, that is, the user's head makes some simple movements of raising and shaking the head without causing discomfort to the user.

[0030] It should be noted that the head movement drives the worn extended reality device to move, and then the original data and gyroscope data are collected by the magnetometer and gyroscope installed in the extended reality device respectively.

[0031] It should be noted that the raw data is measured by a magnetometer (also called a magnetometer sensor). Generally speaking, the raw data contains measurement errors, namely measurement noise, which means that there is a deviation in the relative posture of the measured virtual display screen and the real object, resulting in the problem of misalignment between the virtual object and the real environment when the extended reality device renders the virtual object, that is, the virtual object no longer fits the real environment.

[0032] The high-precision calibration of the magnetometer can ensure the accurate alignment of virtual objects and the real environment during long-term use, achieving a smooth virtual-real fusion experience. In extended reality devices, the accuracy of the magnetometer sensor is also directly related to the reliability of the navigation system, ball sports detection, swimming posture detection and other motion state judgment accuracy.

[0033] 102. Generate a target function to be solved for minimizing measurement noise based on a magnetometer measurement model, wherein the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise.

[0034] Among them, by using the magnetometer measurement model constructed by combining gyroscope parameters and magnetometer parameters, it is helpful to realize the calibration of the magnetometer in combination with gyroscope data, simplify the complexity of the raw data required for magnetometer calibration, and simplify the operation difficulty of the user when calibrating the magnetometer of the extended reality device. For example, when wearing an extended reality device, the user only needs to make basic movements of raising his head and shaking his head to collect effective raw data and gyroscope data.

[0035] For example, in an embodiment of the present application, the magnetometer measurement model includes: ,in, is the uncalibrated magnetometer reading, corresponding to the raw data collected by the magnetometer. is the unit constant magnetic vector in the local inertial coordinate system, for The rotation from the coordinate system to the IMU coordinate system is as follows: Figure 3 As shown, The coordinate system refers to the magnetometer coordinate system, and the magnetometer readings on the device are obtained based on this coordinate system. The coordinate system refers to the local inertial coordinate system at the initial moment. The coordinate system refers to the coordinate system of the device itself, which is also equivalent to the device IMU coordinate system. It changes with time and the movement of the device. It can be obtained recursively by integrating the gyroscope, that is, ,in, The rotation within the sampling time can be obtained by multiplying the angular velocity measured by the gyroscope by the sampling time, and then the rotation can be integrated using the Rodrigues formula. It consists of multiple parts, such as the magnetometer soft magnetic error matrix and the internal parameter matrix. , rotation between the magnetometer coordinate system and the device coordinate system , specifically, . is the measurement noise of the magnetometer, is the hard magnetic parameter of the magnetometer. Specifically, and It is the magnetometer parameter in the embodiment of the present application, that is, the parameter that needs to be solved to obtain the target magnetometer parameter.

[0036] The target function to be solved is obtained by transforming the magnetometer measurement model. is the measurement noise of the magnetometer, then the target function to be solved is obtained by minimizing the residual error through the magnetometer measurement model representation. For example, the target function to be solved can be expressed as: , wherein the parameters involved in the target function to be solved are consistent with the parameters in the magnetometer measurement model, therefore, the definition or description of the same parameter names will not be repeated here.

[0037] Among them, by constructing a target function to be solved based on the magnetometer measurement model, it is convenient to calculate the solution results corresponding to the magnetometer parameters by solving the function.

[0038] 103. The target function to be solved is solved by using the original data and the gyroscope data to obtain target magnetometer parameters. The target magnetometer parameters are the solution results for the magnetometer parameters.

[0039] Correspondingly, after the target function to be solved is constructed, the target function to be solved can be solved based on the original data collected by the magnetometer and the gyroscope data collected by the gyroscope to obtain the solution results corresponding to the magnetometer parameters, that is, the target magnetometer parameters.

[0040] The raw data collected by the magnetometer and the gyroscope data are data that are associated at all times, that is, they are both data collected when the target user's head moves.

[0041] 104. Calibrate the magnetometer based on the target magnetometer parameters.

[0042] It can be understood that after the target magnetometer parameters are obtained, the magnetometer of the extended reality device can be calibrated based on the target magnetometer parameters to obtain more accurate magnetometer data.

[0043] For example, in the actual application of the magnetometer, if the real-time magnetometer data is collected by the magnetometer, the real-time error data can be calculated or estimated based on the target magnetometer parameters obtained by the solution (similarly, the associated gyroscope data needs to be collected and combined with the real-time magnetometer data for solution), and then the actual magnetometer data is calculated by the difference between the real-time magnetometer data and the real-time error data, and then the magnetic application can be performed based on the actual magnetometer data. Among them, the magnetic application includes but is not limited to at least one of positioning navigation, gesture interaction, motion analysis or environmental detection.

[0044] In summary, the embodiments of the present application solve the magnetometer parameters based on the magnetometer measurement model including the magnetometer parameters and the gyroscope parameters, thereby realizing the calibration of the magnetometer in combination with the gyroscope.

[0045] In the related prior art, when a separate magnetometer is used for calibration, it is necessary to collect magnetometer data based on the "8-character standard" to obtain more diverse and rich magnetometer data, so as to meet the accuracy requirements of the separate magnetometer calibration. However, the embodiment of the present application combines gyroscope parameters to calibrate the magnetometer parameters, and there is no need to collect magnetometer data based on the "8-character standard", which reduces the requirements for the data relied on for calibration, that is, simplifies the requirements for device movement during magnetometer calibration, simplifies the difficulty of user operation during calibration, and improves user experience.

[0046] Optionally, in order to facilitate solving the target function to be solved, the target function to be solved may also be obtained by performing a series of processes such as identity transformation and block merging on the magnetometer measurement model. That is, in the embodiment of the present application, the step of “generating a target function to be solved for minimizing measurement noise based on the magnetometer measurement model” includes: determining an initial measurement noise function for representing magnetometer measurement noise according to a magnetometer measurement model; Performing an identity transformation on the initial measurement noise function to obtain a minimized measurement noise function; The magnetometer parameters in the function that minimizes the measurement noise are processed in blocks to obtain the target function to be solved.

[0047] In the embodiment of the present application, the initial measurement noise function is a function representation of the measurement noise in the magnetometer measurement model. For example, if the magnetometer measurement model is: , then the initial measurement noise function is expressed as: , further, by multiplying both sides of the initial measurement noise function , we get the function that minimizes the measurement noise: , and, according to the Kronecker product Further transformation gives the target function to be solved: , wherein the parameters involved in the above-mentioned initial measurement noise function, minimized measurement noise function and target function to be solved are consistent with the parameters in the aforementioned magnetometer measurement model, therefore, the definition or description of the same parameter name will not be repeated here.

[0048] Furthermore, in an embodiment of the present application, the target function to be solved can be solved based on preset constraints to obtain target magnetometer parameters, that is, optionally, in some embodiments of the present application, the gyroscope data includes angular velocity data, and the step of "solving the target function to be solved by using the original data and the gyroscope data to obtain the target magnetometer parameters" includes: Calculate the period rotation vector according to the angular velocity data of the sampling period, and convert the period rotation vector into a period rotation matrix; Multiply the time period rotation matrices of each sampling period to obtain the rotation information corresponding to the head movement; Through the original data and rotation information, the target function to be solved is solved based on the preset constraints to obtain the target magnetometer parameters.

[0049] For example, in the embodiment of the present application, according to the solution idea of ​​the Rayleigh-Quotient problem, the target function to be solved is constrained to || ||=1 to solve and obtain the target magnetometer parameters.

[0050] The rotation information refers to the rotation between the current moment and the initial moment of the head movement, such as the rotation angle or the rotation vector, which is obtained by multiplying the time period rotation matrices of each time period, wherein the time period rotation matrix of each time period is obtained by converting the rotation vector of each time period based on the Rodrigues formula, and the rotation vector of each time period is obtained by multiplying the gyroscope angular velocity of each time period by the time period.

[0051] In summary, referring to the above target function to be solved, it should be noted that: , is a known quantity, among which, is the magnetometer reading, that is, the raw data collected by the magnetometer, which is directly used as the observed quantity to obtain the set. For the rotation between the extended reality device and the initial posture at each moment, that is, the rotation between the current moment and the initial moment of the head movement mentioned above, the angular velocity of the gyroscope can be multiplied by the time in each sampling time period (1 / gyroscope sampling frequency) to obtain a rotation vector for the sampling time period, and then converted into a rotation matrix using the Rodrigues formula. The rotation matrices of all sampling time periods are multiplied together to obtain the rotation matrix from time 0 to time k. Coordinate system to device coordinate system rotation.

[0052] In this process, there is no need to keep the device still, nor to rely on external devices (the gyroscope is set inside the extended reality device). It appears as a solution variable (actually a constant vector) and a constraint condition, and does not need to be pre-calculated. The modulus of 1 appears as an assumption, which is naturally true in a stable geomagnetic local area without external magnetic field interference. The calculation of also only depends on the observation of the gyroscope, including the accuracy of the gyroscope itself and the accuracy of the sampling time, which can be satisfied by using a consumer-grade IMU. It can also be seen from the formula of the target function to be solved that when the device rotates sufficiently, the accuracy and success rate of the solution are higher. After multiple actual tests, it is only necessary to rotate the device around at least two axes to a sufficient angle. For head-mounted extended reality devices (such as AR and VR), only two actions, nodding and shaking the head, can complete the calibration when wearing it.

[0053] For example, Figure 4 As shown, the coordinate axis direction XYZ of the extended reality device and the movement sequence of the extended reality device when calibrating the magnetometer are shown, for example, first rotating in the Yaw direction (shaking the head), around the Y axis of the device, including: Step 1: Rotate counterclockwise from 0 degrees (at this time in posture 1) to -60 degrees (at this time in posture 2); Step 2: Rotate clockwise from -60 degrees back to 0 degrees, and then rotate clockwise to +60 degrees (at this time it is in posture 3); Step 3: Rotate counterclockwise from +60 degrees back to 0 degrees (now in posture 1).

[0054] Rotate (nod) in the Pitch direction, around the device's X axis, including: Step 4: Rotate counterclockwise from 0 degrees (at this time in posture 1) to -40 degrees (at this time in posture 4); Step 5: Rotate clockwise from -40 degrees back to 0 degrees, and then rotate clockwise to +40 degrees (at this point it is in posture 5); Step 6: Rotate counterclockwise from +40 degrees back to 0 degrees (now in posture 1).

[0055] That is, through the above simple shaking and nodding movements, effective raw data required for magnetometer calibration can be collected.

[0056] In summary, the embodiment of the present application solves the magnetometer parameters based on the magnetometer measurement model including the magnetometer parameters and the gyroscope parameters, and realizes the calibration of the magnetometer in combination with the gyroscope. In the related prior art, when a separate magnetometer is used for calibration, it is necessary to collect magnetometer data based on the "8-character standard" to obtain more diverse and rich magnetometer data, so as to meet the accuracy requirements of the separate magnetometer calibration. The embodiment of the present application calibrates the magnetometer parameters in combination with the gyroscope parameters, and there is no need to collect magnetometer data based on the "8-character standard", which reduces the requirements for the data relied on for calibration, that is, simplifies the requirements for device movement during magnetometer calibration, simplifies the difficulty of user operation during calibration, and improves user experience.

[0057] Furthermore, since the raw data of the magnetometer is easily interfered by the external environment during collection, on the one hand, the magnetic field interference caused by the fluctuation of the environmental magnetic field, for example, there is a changing current or a ferromagnetic object near the extended reality device, which causes the magnetic field of the current calibration environment of the extended reality device to be unstable, so that a small part of the values ​​detected by the magnetometer does not conform to the hard magnetic error model when there is no magnetic field interference; on the other hand, environmental vibration interference or electrostatic interference, for example, the magnetometer signal noise suddenly deviates from the normal value of a frame of data (such as a burr signal), or introduces a high-frequency vibration noise, such as Figure 5 , Figure 5 Shows the glitch signal caused by static electricity interference or strong current changes nearby ( Figure 5 protrusions or small peaks on the middle curve, etc.), where Figure 5 Here, X, Y and Z represent three coordinate directions respectively, and the values ​​of the line segments corresponding to X, Y and Z are the magnetic field intensities in the three coordinate directions respectively.

[0058] Therefore, the raw data collected by the magnetometer will contain some data that are interfered by the environment, which are also called outliers. In order to improve the accuracy of the target magnetometer parameter calculation, before calculating the target magnetometer parameters based on the raw data, the outliers can be reduced by denoising or the influence of the outliers on the target magnetometer parameter calculation can be reduced. That is, optionally, in some embodiments of the present application, before the step of "solving the target function to be solved by the raw data and the gyroscope data to obtain the target magnetometer parameters", the method also includes: Perform bidirectional filtering on the original data to obtain the target data, or, Use the target function to be solved to perform random sampling iterative optimization on the original data to obtain the target data, or, Perform bidirectional filtering on the original data to obtain filtered data, and use the target function to be solved to perform random sampling iterative optimization on the filtered data to obtain the target data; The target function to be solved is solved through the original data and gyroscope data to obtain the target magnetometer parameters, including: The target function to be solved is solved by using the target data and the gyroscope data to obtain the target magnetometer parameters.

[0059] Among them, outliers contained in the raw data obtained by measurement are removed through bidirectional filtering, such as removing burr signals or high-frequency vibration noise in the raw data. Bidirectional filtering is used to effectively smooth the data and reduce the impact of environmental interference on the calibration results.

[0060] For example, in an embodiment of the present application, the bidirectional filtering includes bidirectional low-pass filtering, and the raw data is subjected to bidirectional low-pass filtering based on the calibration frequency of the magnetometer, that is, optionally, in some embodiments of the present application, the step of "performing bidirectional filtering on the raw data to obtain target data" includes: Determine the calibration frequency of the magnetometer, which refers to the frequency of head movement; Determine the target order and target frequency according to the calibration frequency; The original data is subjected to bidirectional filtering according to the target order and target frequency to obtain the target data.

[0061] Among them, the calibration action of the magnetometer itself is relatively low-frequency. For example, when wearing an extended reality device, the raw data of the magnetometer is obtained by raising or shaking the head. The frequency of raising the head and shaking the head is relatively slow, mainly concentrated in 0.5Hz-2Hz (corresponding to the calibration frequency). Therefore, the 100Hz raw data of the magnetometer can be processed by a 2nd-order (target order) and 10Hz (target frequency) bidirectional low-pass filter.

[0062] Specifically, is the magnetometer data of the nth frame before filtering, is the nth frame of magnetometer data after filtering, then the bidirectional filtering process includes: 1) Determine the sparseness of the filter transfer function as: ,in, , , , , Obtained by filter design (such as Butterworth filter design).

[0063] 2) Determine the impulse response function: ,in, is a discrete unit pulse function, n corresponds to the current frame, k represents the number of frames before the current frame, such as nk represents the number of frames before the current frame.

[0064] 3) Bidirectional filtering: Ⅰ. Forward, .

[0065] Ⅱ. Reverse direction, .

[0066] 4) Output the filtered magnetometer signal (i.e., filtered data): .

[0067] In addition, when the original data is randomly sampled and iteratively optimized based on the target function to be solved to obtain the target data, the original data is divided into a calculation set and a test set through random sampling data, the solution corresponding to the magnetometer parameters is calculated through the calculation set, and the solution result is tested through the test set to filter the target data from the original data according to the test result.

[0068] That is, optionally, in some embodiments of the present application, the step of “using the target function to be solved to perform random sampling iterative optimization processing on the original data to obtain the target data” includes: Generate at least one set of test data corresponding to the original data according to a preset ratio, wherein each set of test data includes a first set of data randomly selected based on the preset ratio and a second set of data in the original data except the first set of data; For each group of test data, the magnetometer parameters to be optimized of the magnetometer are calculated according to the first set of data in the test data by using the target function to be solved, and the target pure data with a measurement error less than an error threshold in the second set of data of the test data is determined according to the magnetometer parameters to be optimized; The second set of data containing the most target pure data is screened out, and the first set of data corresponding to the second set of data and the target pure data of the second set of data are used as target data.

[0069] For example, the random sampling iterative optimization process specifically includes: 1. Randomly and evenly extract 40% of the data from the original data collected by the magnetometer and the rotation matrix data obtained by the gyroscope integration at the corresponding moment; 2. Use this 40% data to adjust the magnetometer parameters of the magnetometer ( and ) for calibration; 3. Using the above calibration and And the remaining 60% of the data calculate the measurement error: , and the residual is less than Mark the data as target pure data (or inliers), and count the number of target pure data (or inliers); 4. Repeat steps 1-3 above to find a group of 40% data with the largest number of target pure data (or inliers), and re-solve the target function to be solved together with the number of all target pure data (or inliers) in this group (together to obtain the target data) to obtain the target magnetometer parameters.

[0070] Correspondingly, after the target magnetometer parameters are obtained, the magnetometer of the extended reality device can be calibrated based on the target magnetometer parameters.

[0071] Furthermore, in an embodiment of the present application, the original data is first subjected to bidirectional filtering to remove the second type of outliers (corresponding to: data generated by environmental vibration interference or electrostatic interference) to obtain filtered data, thereby eliminating the influence of the data generated by environmental vibration interference or electrostatic interference on the calibration accuracy of the magnetometer. Then, the filtered data is processed by random sampling iterative optimization to obtain target data, thereby screening data that conforms to the magnetometer error model and optimizing the data used to solve the target function to be solved, which is equivalent to eliminating outliers still contained in the filtered data, such as data generated by magnetic field interference caused by environmental magnetic field fluctuations (also called first type outliers).

[0072] Among them, the embodiment of the present application increases the probability of finding the best data by trying different data sets of the same proportion through random sampling.

[0073] In order to facilitate better implementation of the magnetometer calibration method of the extended reality device of the present application, the present application also provides a magnetometer calibration device of the extended reality device based on the magnetometer calibration method of the extended reality device. The meaning of the terms is the same as that in the magnetometer calibration method of the extended reality device, and the specific implementation details can refer to the description in the method embodiment.

[0074] See also Figure 6 , Figure 6 : is a schematic diagram of the structure of a magnetometer calibration device for an extended reality device provided in an embodiment of the present application, wherein the magnetometer calibration device for an extended reality device is applied to the extended reality device, and the device may be specifically as follows: An acquisition module 201 is used for acquiring raw data through a magnetometer of the extended reality device in response to a head movement of a target user wearing the extended reality device, and acquiring gyroscope data corresponding to the raw data through a gyroscope of the extended reality device; A construction module 202 is used to generate a target function to be solved for minimizing measurement noise based on a magnetometer measurement model, wherein the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise; A solution module 203 is used to solve the target function to be solved by using the original data and the gyroscope data to obtain target magnetometer parameters, where the target magnetometer parameters are the solution results for the magnetometer parameters; The calibration module 204 is used to calibrate the magnetometer based on the target magnetometer parameters.

[0075] Optionally, in some embodiments of the present application, a target function to be solved for minimizing measurement noise is generated based on a magnetometer measurement model, including: determining an initial measurement noise function for representing magnetometer measurement noise according to a magnetometer measurement model; Performing an identity transformation on the initial measurement noise function to obtain a minimized measurement noise function; The magnetometer parameters in the function that minimizes the measurement noise are processed in blocks to obtain the target function to be solved.

[0076] Optionally, in some embodiments of the present application, the gyroscope data includes angular velocity data, and the target function to be solved is solved by using the original data and the gyroscope data to obtain the target magnetometer parameters, including: Calculate the period rotation vector according to the angular velocity data of the sampling period, and convert the period rotation vector into a period rotation matrix; Multiply the time period rotation matrices of each sampling period to obtain the rotation information corresponding to the head movement; Through the original data and rotation information, the target function to be solved is solved based on the preset constraints to obtain the target magnetometer parameters.

[0077] Optionally, in some embodiments of the present application, before solving the target function to be solved by using the original data and the gyroscope data to obtain the target magnetometer parameters, the method further includes: Perform bidirectional filtering on the original data to obtain the target data, or, Use the target function to be solved to perform random sampling iterative optimization on the original data to obtain the target data, or, Perform bidirectional filtering on the original data to obtain filtered data, and use the target function to be solved to perform random sampling iterative optimization on the filtered data to obtain the target data; The target function to be solved is solved through the original data and gyroscope data to obtain the target magnetometer parameters, including: The target function to be solved is solved by using the target data and the gyroscope data to obtain the target magnetometer parameters.

[0078] Optionally, in some embodiments of the present application, bidirectional filtering is performed on the original data to obtain target data, including: Determine the calibration frequency of the magnetometer, which refers to the frequency of head movement; Determine the target order and target frequency according to the calibration frequency; The original data is subjected to bidirectional filtering according to the target order and target frequency to obtain the target data.

[0079] Optionally, in some embodiments of the present application, the target function to be solved is used to perform random sampling iterative optimization processing on the original data to obtain the target data, including: Generate at least one set of test data corresponding to the original data according to a preset ratio, wherein each set of test data includes a first set of data randomly selected based on the preset ratio and a second set of data in the original data except the first set of data; For each group of test data, the magnetometer parameters to be optimized of the magnetometer are calculated according to the first set of data in the test data by using the target function to be solved, and the target pure data with a measurement error less than an error threshold in the second set of data of the test data is determined according to the magnetometer parameters to be optimized; The second set of data containing the most target pure data is screened out, and the first set of data corresponding to the second set of data and the target pure data of the second set of data are used as target data.

[0080] Optionally, in some embodiments of the present application, after calibrating the magnetometer based on the target magnetometer parameters, the method further includes: In response to the magnetometer measurement, real-time magnetometer data is obtained, and real-time error data corresponding to the real-time magnetometer data is calculated using target magnetometer parameters; The difference between the real-time magnetometer data and the real-time error data is calculated to obtain the actual magnetometer data; A magnetic application is performed based on actual magnetometer data, wherein the magnetic application includes at least one of positioning navigation, gesture interaction, motion analysis, or environment detection.

[0081] In the embodiment of the present application, the acquisition module 201 obtains raw data through the magnetometer of the extended reality device in response to the head movement of the target user wearing the extended reality device, and obtains gyroscope data corresponding to the raw data through the gyroscope of the extended reality device. The construction module 202 generates a target function to be solved for minimizing measurement noise based on the magnetometer measurement model. The magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise. The solution module 203 solves the target function to be solved through the raw data and the gyroscope data to obtain target magnetometer parameters. The target magnetometer parameters are the solution results of the magnetometer parameters. The calibration module 204 calibrates the magnetometer based on the target magnetometer parameters.

[0082] Among them, the embodiment of the present application solves the magnetometer parameters based on a magnetometer measurement model including magnetometer parameters and gyroscope parameters, thereby realizing the calibration of the magnetometer in combination with the gyroscope.

[0083] In the related prior art, when a separate magnetometer is used for calibration, it is necessary to collect magnetometer data based on the "8-character standard" to obtain more diverse and rich magnetometer data, so as to meet the accuracy requirements of the separate magnetometer calibration. However, the embodiment of the present application combines gyroscope parameters to calibrate the magnetometer parameters, and there is no need to collect magnetometer data based on the "8-character standard", which reduces the requirements for the data relied on for calibration, that is, simplifies the requirements for device movement during magnetometer calibration, simplifies the difficulty of user operation during calibration, and improves user experience.

[0084] In addition, the present application also provides an extended reality device, such as Figure 7 As shown, it shows a schematic diagram of the structure of the extended reality device involved in this application, specifically: The extended reality device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will appreciate that Figure 7 The structure of the extended reality device shown in the figure does not constitute a limitation on the extended reality device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. The processor 301 is the control center of the extended reality device. It uses various interfaces and lines to connect various parts of the entire extended reality device. By running or executing software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, it performs various functions of the extended reality device and processes data, thereby monitoring the extended reality device as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0085] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and data processing by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the extended reality device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0086] The extended reality device also includes a power supply 303 for supplying power to each component. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 303 can also include one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0087] The extended reality device may further include an input unit 304, which may be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0088] Although not shown, the extended reality device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the extended reality device will load the executable file corresponding to the process of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application program stored in the memory 302, thereby implementing the steps in the magnetometer calibration method of any extended reality device provided in the embodiments of the present application.

[0089] The extended reality device of the embodiment of the present application responds to the head movement of the target user wearing the extended reality device, collects raw data through the magnetometer of the extended reality device, and collects gyroscope data corresponding to the raw data through the gyroscope of the extended reality device, generates a target function to be solved for minimizing measurement noise based on a magnetometer measurement model, the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise, solves the target function to be solved through the raw data and gyroscope data, obtains target magnetometer parameters, the target magnetometer parameters are the solution results of the magnetometer parameters, and the magnetometer is calibrated based on the target magnetometer parameters.

[0090] The magnetometer parameters are solved based on a magnetometer measurement model including magnetometer parameters and gyroscope parameters, so that the magnetometer calibration is realized in combination with the gyroscope.

[0091] In the related prior art, when a separate magnetometer is used for calibration, it is necessary to collect magnetometer data based on the "8-character standard" to obtain more diverse and rich magnetometer data, so as to meet the accuracy requirements of the separate magnetometer calibration. However, the embodiment of the present application combines gyroscope parameters to calibrate the magnetometer parameters, and there is no need to collect magnetometer data based on the "8-character standard", which reduces the requirements for the data relied on for calibration, that is, simplifies the requirements for device movement during magnetometer calibration, simplifies the difficulty of user operation during calibration, and improves user experience.

[0092] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0093] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0094] To this end, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in the magnetometer calibration method of any extended reality device provided in the present application.

[0095] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0096] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0097] Since the instructions stored in the computer-readable storage medium can execute the steps in the magnetometer calibration method of any extended reality device provided in the present application, the beneficial effects that can be achieved by the magnetometer calibration method of any extended reality device provided in the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0098] The above is a detailed introduction to the magnetometer calibration method, device, extended reality device and computer-readable storage medium of an extended reality device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0099] It should be noted that in the specific implementation of the present application, related data such as magnetometer data (raw data) and gyroscope data are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A magnetometer calibration method for an extended reality device, characterized in that: Applied to an extended reality device, the method comprises: In response to a head movement of a target user wearing the extended reality device, raw data is acquired through a magnetometer of the extended reality device, and gyroscope data corresponding to the raw data is acquired through a gyroscope of the extended reality device; generating a target function to be solved for minimizing measurement noise based on a magnetometer measurement model, wherein the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise; Solving the target function to be solved by using the original data and the gyroscope data to obtain target magnetometer parameters, wherein the target magnetometer parameters are solution results for the magnetometer parameters; The magnetometer is calibrated based on the target magnetometer parameters.

2. The magnetometer calibration method of the extended reality device according to claim 1, characterized in that: The method of generating a target function to be solved for minimizing measurement noise based on a magnetometer measurement model includes: determining an initial measurement noise function for representing magnetometer measurement noise according to the magnetometer measurement model; Performing an identity transformation on the initial measurement noise function to obtain a minimized measurement noise function; The magnetometer parameters in the minimized measurement noise function are processed in a block manner to obtain a target function to be solved.

3. The magnetometer calibration method of the extended reality device according to claim 1, characterized in that: The gyroscope data includes angular velocity data, and the target function to be solved is solved by using the original data and the gyroscope data to obtain target magnetometer parameters, including: Calculate a time period rotation vector according to the angular velocity data of the sampling period, and convert the time period rotation vector into a time period rotation matrix; Multiplying the time period rotation matrices of each sampling period to obtain the rotation information corresponding to the head movement; The target function to be solved is solved based on the preset constraints through the original data and the rotation information to obtain the target magnetometer parameters.

4. The magnetometer calibration method of the extended reality device according to claim 1, characterized in that: Before solving the target function to be solved by using the original data and the gyroscope data to obtain the target magnetometer parameters, the method further includes: Performing bidirectional filtering on the original data to obtain target data, or, The target function to be solved is used to perform random sampling iterative optimization processing on the original data to obtain target data, or, Performing bidirectional filtering on the original data to obtain filtered data, and performing random sampling iterative optimization processing on the filtered data using the target function to be solved to obtain target data; The step of solving the target function to be solved by using the original data and the gyroscope data to obtain target magnetometer parameters includes: The target function to be solved is solved by using the target data and the gyroscope data to obtain target magnetometer parameters.

5. The magnetometer calibration method of the extended reality device according to claim 4, characterized in that: The performing of bidirectional filtering on the original data to obtain target data includes: Determining a calibration frequency of the magnetometer, wherein the calibration frequency refers to a movement frequency of the head movement; Determining a target order and a target frequency according to the calibration frequency; The original data is subjected to bidirectional filtering processing according to the target order and the target frequency to obtain target data.

6. The magnetometer calibration method of the extended reality device according to claim 4, characterized in that: The step of performing random sampling iterative optimization processing on the original data using the target function to be solved to obtain target data includes: Generate at least one set of test data corresponding to the original data according to a preset ratio, wherein each set of test data includes a first set of data randomly selected based on the preset ratio and a second set of data in the original data except the first set of data; For each group of test data, the magnetometer parameters to be optimized of the magnetometer are calculated according to the first set of data in the test data and by the target function to be solved, and the target pure data with a measurement error less than an error threshold in the second set of data of the test data is determined according to the magnetometer parameters to be optimized; The second set of data containing the most target pure data is screened out, and the first set of data corresponding to the second set of data and the target pure data of the second set of data are used as target data.

7. The magnetometer calibration method of the extended reality device according to claim 1, characterized in that: After calibrating the magnetometer based on the target magnetometer parameters, the method further includes: In response to obtaining real-time magnetometer data measured by the magnetometer, calculating real-time error data corresponding to the real-time magnetometer data by using the target magnetometer parameters; Calculating the difference between the real-time magnetometer data and the real-time error data to obtain actual magnetometer data; A magnetic application is performed based on the actual magnetometer data, wherein the magnetic application includes at least one of positioning navigation, gesture interaction, motion analysis, or environment detection.

8. A magnetometer calibration device for an extended reality device, characterized in that: Applied to an extended reality device, the device comprises: an acquisition module, configured to acquire raw data through a magnetometer of the extended reality device in response to a head movement of a target user wearing the extended reality device, and acquire gyroscope data corresponding to the raw data through a gyroscope of the extended reality device; A construction module, for generating a target function to be solved for minimizing measurement noise based on a magnetometer measurement model, wherein the magnetometer measurement model is constructed based on magnetometer parameters, gyroscope parameters and measurement noise; A solution module, used for solving the target function to be solved by using the original data and the gyroscope data to obtain target magnetometer parameters, wherein the target magnetometer parameters are solution results for the magnetometer parameters; A calibration module is used to calibrate the magnetometer based on the target magnetometer parameters.

9. An extended reality device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the magnetometer calibration method of the extended reality device as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps in the magnetometer calibration method of the extended reality device according to any one of claims 1 to 7 are implemented.

Images