Pose information determination method and apparatus, electronic device, and non-transitory storage medium

Abstract

The application discloses a pose information determination method and device, electronic equipment and a nonvolatile storage medium. The method comprises the following steps: obtaining first frame sensor data from an inertial measurement unit in a target device; determining first timestamp information of the first frame sensor data and second timestamp information of second frame sensor data, wherein the second frame sensor data is sensor data obtained before the first frame sensor data; determining an interval duration between the first frame sensor data and the second frame sensor data according to the first timestamp information and the second timestamp information; and determining first target pose information of the target device according to the first frame sensor data, the second frame sensor data and the interval duration. The application solves the technical problem that the positioning effect of the inertial measurement unit in the mobile device is poor due to the high frame rate requirement of the inertial measurement unit in the related art.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and more specifically, to a method, apparatus, electronic device, and non-volatile storage medium for determining pose information. Background Technology

[0002] Currently, in related technologies, when determining the pose information of a device through an inertial measurement unit (IMU), there are usually high requirements for the frequency of data generated by the IMU, which leads to high requirements for the device configuration. However, due to size limitations, mobile terminal devices such as mobile phones usually cannot guarantee the data generation frequency of the IMU, resulting in poor positioning performance.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a pose information determination method, apparatus, electronic device, and non-volatile storage medium to at least solve the technical problem that the poor positioning effect of inertial measurement units in mobile devices is caused by the high frame rate requirement of inertial measurement units when performing pose positioning through inertial measurement units in related technologies.

[0005] According to one aspect of the embodiments of this application, a pose information determination method is provided, comprising: acquiring a first frame of sensor data from an inertial measurement unit in a target device; determining a first timestamp information of the first frame of sensor data and a second timestamp information of a second frame of sensor data, wherein the second frame of sensor data is sensor data acquired before acquiring the first frame of sensor data, and the first frame of sensor data and the second frame of sensor data are two consecutive frames of measurement data generated by the inertial measurement unit; determining an interval duration between the first frame of sensor data and the second frame of sensor data based on the first timestamp information and the second timestamp information; and determining a first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame of sensor data, the second frame of sensor data, and the interval duration.

[0006] Optionally, the step of determining the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame sensor data, the second frame sensor data, and the interval duration includes: determining the first target pose information based on the first frame sensor data and the second frame sensor data when the interval duration is not greater than a preset interval duration; performing interpolation calculation on the first frame sensor data and the second frame sensor data to obtain a third frame sensor data when the interval duration is greater than the preset interval duration, and inserting the third frame sensor data between the first frame sensor data and the second frame sensor data; and determining the first target pose information based on the first frame sensor data, the second frame sensor data, and the third frame sensor data.

[0007] Optionally, after determining the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame sensor data, the second frame sensor data, and the interval duration, the pose information determination method further includes: determining the gravity direction vector corresponding to the first frame sensor data based on the accelerometer data in the first frame sensor data; correcting the first target pose information based on the gravity direction vector to obtain the second target pose information; and displaying the second target pose information to the target object.

[0008] Optionally, the step of correcting the first target pose information based on the gravity direction vector to obtain the second target pose information includes: determining a first correction vector based on the first target pose information and the gravity direction vector; determining a second correction vector based on the first correction vector and the gravity direction vector; determining orientation angle information based on the first correction vector and the gravity direction vector; and correcting the rotation direction information in the first target pose information based on the orientation angle information and the second correction vector to obtain the second target pose information.

[0009] Optionally, the step of correcting the rotation direction information in the first target pose information based on the orientation angle information and the second correction vector includes: obtaining the target rotation vector based on the orientation angle information and the second correction vector; determining the corrected rotation information based on the target rotation vector; and correcting the rotation direction information based on the corrected rotation information.

[0010] Optionally, after obtaining the first frame of sensor data from the inertial measurement unit in the target device, the pose information determination method further includes: performing low-pass filtering on the first frame of sensor data to filter out high-frequency noise in the first frame of sensor data.

[0011] Optionally, the first frame of sensor data and the second frame of sensor data include image data, gyroscope data, accelerometer data and orientation sensor data.

[0012] According to another aspect of the embodiments of this application, a pose information determination device is also provided, comprising: a first processing module, configured to acquire a first frame of sensor data from an inertial measurement unit in a target device; a second processing module, configured to determine a first timestamp information of the first frame of sensor data and a second timestamp information of the second frame of sensor data, wherein the second frame of sensor data is sensor data acquired before acquiring the first frame of sensor data, and the first frame of sensor data and the second frame of sensor data are two consecutive frames of measurement data generated by the inertial measurement unit; a third processing module, configured to determine the interval duration between the first frame of sensor data and the second frame of sensor data based on the first timestamp information and the second timestamp information; and a fourth processing module, configured to determine a first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame of sensor data, the second frame of sensor data and the interval duration.

[0013] According to another aspect of the embodiments of this application, a terminal device is also provided, including an inertial measurement unit and a processor, wherein the processor is configured to: acquire a first frame of sensor data from the inertial measurement unit in the terminal device; determine a first timestamp information of the first frame of sensor data and a second timestamp information of the second frame of sensor data, wherein the second frame of sensor data is sensor data acquired before acquiring the first frame of sensor data, and the first frame of sensor data and the second frame of sensor data are two consecutive frames of measurement data generated by the inertial measurement unit; determine an interval duration between the first frame of sensor data and the second frame of sensor data based on the first timestamp information and the second timestamp information; and determine a first target pose information of the terminal device at the time point corresponding to the first timestamp information based on the first frame of sensor data, the second frame of sensor data and the interval duration.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, wherein a program is stored in the non-volatile storage medium, and the program controls the device where the non-volatile storage medium is located to execute a pose information determination method when it runs.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a pose information determination method during runtime.

[0016] In this embodiment, a method is employed whereby a first frame of sensor data is acquired from an inertial measurement unit (IMU) in the target device; a first timestamp of the first frame of sensor data and a second timestamp of the second frame of sensor data are determined, wherein the second frame of sensor data is sensor data acquired before the first frame of sensor data is acquired, and the first and second frames of sensor data are two consecutive frames of measurement data generated by the IMU; the interval between the first and second frames of sensor data is determined based on the first and second timestamps; and the first target pose information of the target device at the time point corresponding to the first timestamp is determined based on the first frame of sensor data, the second frame of sensor data, and the interval. By inserting a third frame of data between the first and second frames when the interval between them is too long, the frame rate of the sensor data is increased, thereby improving the positioning accuracy of the pose information. This solves the technical problem of poor positioning performance of the IMU in mobile devices due to the high frame rate requirement of the IMU when performing pose positioning. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a schematic diagram of the structure of a computer terminal (mobile terminal) according to an embodiment of this application;

[0019] Figure 2 This is a flowchart illustrating a pose information determination method according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of a data transmission process in an inertial measurement unit according to an embodiment of this application;

[0021] Figure 4 This is a schematic flowchart of a data preprocessing process provided according to an embodiment of this application;

[0022] Figure 5a This is a comparative schematic diagram of a device orientation angle provided according to an embodiment of this application;

[0023] Figure 5b This is a comparative schematic diagram of another device orientation angle provided according to an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of a pose information determination device provided according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] To enable navigation in complex scenarios, the application of AR navigation technology is constantly expanding. In AR navigation, when the device performing the navigation is a mobile phone or other terminal device, the phone is constantly moving, requiring real-time, rapid, and accurate calculation of its 6D pose information. However, due to the highly variable lighting conditions and complex low-texture environments in complex scenarios such as underground parking lots, inertial measurement units (IMUs) are typically used to determine the device's pose information.

[0028] However, when using inertial measurement units (IMUs) to determine pose information for positioning in related technologies, the frame rate requirements for the IMU are relatively high, leading to higher demands on the hardware configuration of the equipment. Therefore, it is often impossible to accurately determine the pose information of mobile devices such as smartphones, affecting navigation performance.

[0029] To address this issue, relevant solutions are provided in the embodiments of this application, which are described in detail below.

[0030] According to an embodiment of this application, a method embodiment for determining pose information is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a pose information determination method is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0032] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0033] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the pose information determination method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the pose information determination method of the application described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0035] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0036] Under the above operating environment, this application provides a method for determining location information, such as... Figure 2 As shown, the method includes the following steps:

[0037] Step S202: Obtain the first frame of sensor data from the inertial measurement unit in the target device;

[0038] In the technical solution provided in step S202, after obtaining the first frame of sensor data from the inertial measurement unit in the target device, the pose information determination method further includes: performing low-pass filtering on the first frame of sensor data to filter out high-frequency noise in the first frame of sensor data. By performing low-pass filtering on the sensor data, the original high-frequency noise in the sensor can be removed, thereby improving the reliability of the sensor data.

[0039] In some embodiments of this application, the process of acquiring sensor data is as follows: Figure 3 As shown, after the inertial sensor in the device generates data, the system interface can be called through the front-end program loaded in the device to obtain the raw data generated by the inertial sensor. Then, the data is initially processed, and after processing, the data format is encapsulated. Then, the relevant thread is waited to call the encapsulated data to determine the pose information.

[0040] The initial processing of raw data from the sensor includes address decoding and low-pass filtering. Address decoding refers to reading the data from memory, and this step also includes judging the reliability of the data, such as whether the data format is correct and whether the data is empty.

[0041] Step S204: Determine the first timestamp information of the first frame of sensor data and the second timestamp information of the second frame of sensor data, wherein the second frame of sensor data is sensor data obtained before acquiring the first frame of sensor data, and the first frame of sensor data and the second frame of sensor data are two consecutive frames of measurement data generated by the inertial measurement unit.

[0042] In some embodiments of this application, the first frame of sensor data and the second frame of sensor data include image data, gyroscope data, accelerometer data, and orientation sensor data. The aforementioned image data includes environmental data of the device's surrounding environment, such as environmental images captured by a camera on a mobile phone or other terminal device.

[0043] As an alternative implementation, when determining pose information using data from inertial sensors, the frame or multiple frames of data that have been generated the longest in the sliding window can be deleted first, thereby ensuring that the number of data in the sliding window remains within a preset range. Then, it is determined whether interpolation is needed in the sliding window.

[0044] In some embodiments of this application, the data processing flow is as follows: Figure 4 As shown, it includes the following steps:

[0045] Step S402: Read sensor data by address decoding;

[0046] Step S404: Perform low-pass filtering on the sensor data;

[0047] Step S406: Update the sliding window and delete one or more frames of data that were generated a long time ago;

[0048] Step S408: Determine whether the interval between the latest acquired frame of sensor data and the previous frame of sensor data is greater than a preset time interval (e.g., 30ms).

[0049] Step S410: If the interval is greater than the preset interval, interpolate the latest frame of sensor data with the previous frame of sensor data, and update the sliding window again.

[0050] Step S206: Determine the interval between the first frame of sensor data and the second frame of sensor data based on the first timestamp information and the second timestamp information;

[0051] Specifically, timestamp information can be used to determine the generation time of each frame of sensor data.

[0052] Step S208: Based on the first frame of sensor data, the second frame of sensor data, and the interval duration, determine the first target pose information of the target device at the time point corresponding to the first timestamp information.

[0053] In the technical solution provided in step S208, the step of determining the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame sensor data, the second frame sensor data, and the interval duration includes: determining the first target pose information based on the first frame sensor data and the second frame sensor data when the interval duration is not greater than a preset interval duration; performing interpolation calculation on the first frame sensor data and the second frame sensor data to obtain the third frame sensor data when the interval duration is greater than the preset interval duration, and inserting the third frame sensor data between the first frame sensor data and the second frame sensor data; and determining the first target pose information based on the first frame sensor data, the second frame sensor data, and the third frame sensor data.

[0054] When performing interpolation calculations, mean square interpolation methods, including linear interpolation such as mean square interpolation, can be used.

[0055] Specifically, after data processing steps such as interpolation are completed, pose information can be calculated based on sensor data. Here, the predictive partitioning method of the inertial measurement unit can be used to estimate the device's rotation over a period of time.

[0056] When performing the expected score calculation, it is first necessary to model the measurement models of the gyroscope and the accelerometer. The measurement model of the gyroscope is as follows:

[0057]

[0058] In the above measurement model of the gyroscope, This represents the angular velocity measurement of the device under test relative to a reference object, as measured by the gyroscope. b represents the true angular velocity of the device under test. g (t) represents the gyroscope's zero bias, which is the gyroscope's offset over time. Its derivative follows a Gaussian distribution. η g (t) represents the measurement noise.

[0059] The accelerometer's measurement model is as follows:

[0060]

[0061] In the above accelerometer measurement model, This indicates the acceleration measurement value from the accelerometer. This represents the product of the scale factor and the target rotation matrix, where the target rotation matrix is ​​the rotation matrix between the IMU coordinate system and the N / A coordinate system, aω represents the acceleration of the device under test in the inertial frame, and gω represents the gravitational acceleration b. a (t) represents the time-varying offset of the gyroscope and accelerometer, whose derivative follows a Gaussian distribution, η a (t) represents the measurement noise.

[0062] Based on the above measurement model, the prediction formula for the IMU model can be obtained. This prediction formula can be used to predict the angular velocity, acceleration, etc., of the next frame based on data such as angular velocity and acceleration from any given frame. The following formula is the prediction formula for the IMU model:

[0063]

[0064] In the above formula, t1 represents the first time point, and t2 represents the second time point. This indicates the position of the target device at time t1. This indicates the position of the target device at time t2. This represents the velocity of the target device at time t1. This represents the velocity of the target device at time t2. This represents the rotation angle of the target device at time t1. Let t2 represent the rotation angle of the target device at time t2, and g represent the acceleration. Represents the target rotation matrix. This represents the measurement value from the accelerometer. This represents the measurement value of the gyroscope. This represents the random walk error of the accelerometer. This represents the random walk error of the gyroscope. () represents the natural exponential function.

[0065] In some embodiments of this application, after determining the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame sensor data, the second frame sensor data, and the interval duration, the pose information determination method further includes: determining the gravity direction vector corresponding to the first frame sensor data based on the accelerometer data in the first frame sensor data; correcting the first target pose information based on the gravity direction vector to obtain the second target pose information; and displaying the second target pose information to the target object.

[0066] As an optional implementation, the step of correcting the first target pose information based on the gravity direction vector to obtain the second target pose information includes: determining a first correction vector based on the first target pose information and the gravity direction vector; determining a second correction vector based on the first correction vector and the gravity direction vector; determining orientation angle information based on the first correction vector and the gravity direction vector; and correcting the rotation direction information in the first target pose information based on the orientation angle information and the second correction vector to obtain the second target pose information.

[0067] Specifically, the steps of correcting the rotation direction information in the first target pose information based on the orientation angle information and the second correction vector include: obtaining the target rotation vector based on the orientation angle information and the second correction vector; determining the corrected rotation information based on the target rotation vector; and correcting the rotation direction information based on the corrected rotation information.

[0068] In some embodiments of this application, after calculating the pose using pre-integration, the pose is often inaccurate due to sensor errors and accumulated calculation errors. This manifests in AR navigation as a large deviation in the displayed pose, sometimes even drifting to a position far from the actual location. To address this issue, we employ a method for calculating the direction of gravity as a true direction reference to correct the calculated pose result, thereby obtaining a more realistic and accurate pose. The entire correction process is as follows:

[0069] First, determine the gravity direction vector V2. Then, based on the pose information Q obtained from the corrected measurement and the gravity direction vector V2, determine the first corrected direction vector V1. The specific formula is as follows:

[0070] V1 = Q V2

[0071] Then, the second correction direction vector V3 is determined based on the first correction direction vector V1 and the gravity direction vector V2, using the following formula:

[0072] V3 = V1.cross(V2)

[0073] In the above formula, .corss() represents the cross product operation between V1 and V2.

[0074] The rotation direction angle can then be calculated using the following formula:

[0075] V_angle_temp = V1.dot(V2)

[0076] V_angle = acos(V_angle_temp)

[0077] In the above formula, V_angle_temp represents the dot product of V1 and V2, V_angle is the direction angle information mentioned above, and acos() represents the inverse cosine function.

[0078] After obtaining the orientation angle information, the target rotation vector needs to be determined based on the orientation angle information and the second correction vector information. Then, the correction rotation information is obtained based on the target rotation vector. Finally, the rotation direction is corrected based on the correction rotation information to obtain the corrected second target pose information. The specific formula is as follows:

[0079] Initialize the rotation vector (i.e., the target rotation vector):

[0080] rotate_vecotr=(acos(Q V2.dot(V2)), Q V2.cross(V2))

[0081] Calculate the corrected rotation information: q_temp = rotate_vector

[0082] Calculate the corrected rotational parameters: Q_new = q_temp Q;

[0083] Construct the pose information of the second target based on Q_new.

[0084] The process involves acquiring a first frame of sensor data from the inertial measurement unit (IMU) of the target device; determining a first timestamp of the first frame of sensor data and a second timestamp of the second frame of sensor data, wherein the second frame of sensor data is sensor data acquired before acquiring the first frame of sensor data, and the first and second frames of sensor data are two consecutive frames of measurement data generated by the IMU; determining the interval between the first and second frames of sensor data based on the first and second timestamps; and determining the first target pose information of the target device at the time point corresponding to the first timestamp based on the first frame of sensor data, the second frame of sensor data, and the interval. By inserting a third frame of data between the first and second frames when the interval between them is too long, the frame rate of the sensor data is increased, thereby improving the positioning accuracy of the pose information. This solves the technical problem of poor positioning performance of the IMU in mobile devices due to the high frame rate requirement of the IMU when performing pose positioning.

[0085] Furthermore, the pose information determination method provided in this application can effectively improve the accuracy of pose information positioning. For example... Figure 5a and Figure 5b As shown, where, Figure 5a This is a comparison between the measured and true orientation angle values ​​of the device obtained without using the method provided in this application. Figure 5b It is a comparison between the measured value and the true value of the orientation angle of the device obtained after using the method provided in this application.

[0086] This application provides a pose information determination device. Figure 6 This is a structural schematic diagram of the pose information determination device, as shown below. Figure 6As shown, the device includes: a first processing module 60, used to acquire a first frame of sensor data from an inertial measurement unit in a target device; a second processing module 62, used to determine a first timestamp information of the first frame of sensor data and a second timestamp information of the second frame of sensor data, wherein the second frame of sensor data is sensor data acquired before acquiring the first frame of sensor data, and the first frame of sensor data and the second frame of sensor data are two consecutive frames of measurement data generated by the inertial measurement unit; a third processing module 64, used to determine the interval duration between the first frame of sensor data and the second frame of sensor data based on the first timestamp information and the second timestamp information; and a fourth processing module 66, used to determine the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame of sensor data, the second frame of sensor data and the interval duration.

[0087] In some embodiments of this application, after obtaining the first frame of sensor data from the inertial measurement unit in the target device, the first processing module 60 is further configured to: perform low-pass filtering on the first frame of sensor data to filter out high-frequency noise in the first frame of sensor data.

[0088] In some embodiments of this application, the first frame of sensor data and the second frame of sensor data include image data, gyroscope data, accelerometer data and orientation sensor data.

[0089] In some embodiments of this application, the step of the fourth processing module 66 determining the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame of sensor data, the second frame of sensor data, and the interval duration includes:

[0090] If the interval is not greater than the preset interval, the first target pose information is determined based on the first frame sensor data and the second frame sensor data.

[0091] If the interval is longer than the preset interval, interpolation is performed on the first frame sensor data and the second frame sensor data to obtain the third frame sensor data, and the third frame sensor data is inserted between the first frame sensor data and the second frame sensor data; the first target pose information is determined based on the first frame sensor data, the second frame sensor data and the third frame sensor data.

[0092] In some embodiments of this application, after the fourth processing module 66 determines the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame sensor data, the second frame sensor data, and the interval duration, the pose information determination method further includes: determining the gravity direction vector corresponding to the first frame sensor data based on the accelerometer data in the first frame sensor data; correcting the first target pose information based on the gravity direction vector to obtain the second target pose information; and displaying the second target pose information to the target object.

[0093] In some embodiments of this application, the fourth processing module 66 corrects the first target pose information based on the gravity direction vector to obtain the second target pose information, including: determining a first correction vector based on the first target pose information and the gravity direction vector; determining a second correction vector based on the first correction vector and the gravity direction vector; determining orientation angle information based on the first correction vector and the gravity direction vector; and correcting the rotation direction information in the first target pose information based on the orientation angle information and the second correction vector to obtain the second target pose information.

[0094] In some embodiments of this application, the fourth processing module 66 corrects the rotation direction information in the first target pose information based on the orientation angle information and the second correction vector, including: obtaining the target rotation vector based on the orientation angle information and the second correction vector; determining the corrected rotation information based on the target rotation vector; and correcting the rotation direction information based on the corrected rotation information.

[0095] It should be noted that each module in the above-mentioned pose information determination device can be a program module (for example, a set of program instructions to implement a certain function) or a hardware module. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0096] This application provides a terminal device, including an inertial measurement unit (IMU) and a processor. The processor is configured to: acquire a first frame of sensor data from the IMU in the terminal device; determine a first timestamp information of the first frame of sensor data and a second timestamp information of the second frame of sensor data, wherein the second frame of sensor data is sensor data acquired before acquiring the first frame of sensor data, and the first frame of sensor data and the second frame of sensor data are two consecutive frames of measurement data generated by the IMU; determine the interval duration between the first frame of sensor data and the second frame of sensor data based on the first timestamp information and the second timestamp information; and determine a first target pose information of the terminal device at the time point corresponding to the first timestamp information based on the first frame of sensor data, the second frame of sensor data, and the interval duration.

[0097] This application provides a non-volatile storage medium storing a program. During program execution, the device containing the non-volatile storage medium performs the following pose information determination method: acquiring a first frame of sensor data from an inertial measurement unit (IMU) in the target device; determining a first timestamp of the first frame of sensor data and a second timestamp of the second frame of sensor data, wherein the second frame of sensor data is sensor data acquired before acquiring the first frame of sensor data, and the first and second frames of sensor data are two consecutive frames of measurement data generated by the IMU; determining the interval between the first and second frames of sensor data based on the first and second timestamps; and determining the first target pose information of the target device at the time point corresponding to the first timestamp based on the first frame of sensor data, the second frame of sensor data, and the interval.

[0098] This application provides an electronic device, including a memory and a processor. The processor is used to run a program stored in the memory, wherein, during program execution, the following pose information determination method is performed: acquiring a first frame of sensor data from an inertial measurement unit in a target device; determining a first timestamp information of the first frame of sensor data and a second timestamp information of a second frame of sensor data, wherein the second frame of sensor data is sensor data obtained before acquiring the first frame of sensor data, and the first frame of sensor data and the second frame of sensor data are two consecutive frames of measurement data generated by the inertial measurement unit; determining the interval duration between the first frame of sensor data and the second frame of sensor data based on the first timestamp information and the second timestamp information; and determining the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame of sensor data, the second frame of sensor data, and the interval duration.

[0099] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0104] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of determining pose information, characterized by, include: The first frame of sensor data is acquired from the inertial measurement unit in the target device; Determine the first timestamp information of the first frame sensor data and the second timestamp information of the second frame sensor data, wherein the second frame sensor data is sensor data obtained before acquiring the first frame sensor data, and the first frame sensor data and the second frame sensor data are two consecutive frames of measurement data generated by the inertial measurement unit; The time interval between the first frame of sensor data and the second frame of sensor data is determined based on the first timestamp information and the second timestamp information; Based on the first frame of sensor data, the second frame of sensor data, and the interval duration, the first target pose information of the target device at the time point corresponding to the first timestamp information is determined; The step of determining the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame sensor data, the second frame sensor data, and the interval duration includes: When the interval duration is not greater than the preset interval duration, the first target pose information is determined based on the first frame sensor data and the second frame sensor data. If the interval duration is longer than a preset interval duration, interpolation calculation is performed on the first frame sensor data and the second frame sensor data to obtain the third frame sensor data, and the third frame sensor data is inserted between the first frame sensor data and the second frame sensor data; the first target pose information is determined based on the first frame sensor data, the second frame sensor data and the third frame sensor data.

2. The pose information determination method according to claim 1, characterized in that, After the step of determining the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame sensor data, the second frame sensor data, and the interval duration, the pose information determination method further includes: Based on the accelerometer data in the first frame of sensor data, determine the gravity direction vector corresponding to the first frame of sensor data; The first target pose information is corrected based on the gravity direction vector to obtain the second target pose information; The pose information of the second target is displayed to the target object.

3. The pose information determination method according to claim 2, characterized in that, The step of correcting the first target pose information based on the gravity direction vector to obtain the second target pose information includes: A first correction vector is determined based on the first target pose information and the gravity direction vector; The second correction vector is determined based on the first correction vector and the gravity direction vector; The orientation angle information is determined based on the first correction vector and the gravity direction vector; The rotation direction information in the first target pose information is corrected based on the orientation angle information and the second correction vector to obtain the second target pose information.

4. The pose information determination method according to claim 3, characterized in that, The step of correcting the rotation direction information in the first target pose information based on the orientation angle information and the second correction vector includes: The target rotation vector is obtained based on the orientation angle information and the second correction vector; The corrected rotation information is determined based on the target rotation vector; The rotation direction information is corrected based on the corrected rotation information.

5. The pose information determination method according to claim 1, characterized in that, After acquiring the first frame of sensor data from the inertial measurement unit in the target device, the pose information determination method further includes: The first frame of sensor data is subjected to low-pass filtering to remove high-frequency noise from the first frame of sensor data.

6. The pose information determination method according to claim 1, characterized in that, The first frame of sensor data and the second frame of sensor data include image data, gyroscope data, accelerometer data and orientation sensor data.

7. A pose information determination device, characterized in that, include: The first processing module is used to acquire the first frame of sensor data from the inertial measurement unit in the target device; The second processing module is used to determine the first timestamp information of the first frame sensor data and the second timestamp information of the second frame sensor data, wherein the second frame sensor data is sensor data obtained before the first frame sensor data is acquired, and the first frame sensor data and the second frame sensor data are two consecutive frames of measurement data generated by the inertial measurement unit. The third processing module is used to determine the interval between the first frame sensor data and the second frame sensor data based on the first timestamp information and the second timestamp information. The fourth processing module is used to determine the first target pose information of the target device at the time point corresponding to the first timestamp information based on the first frame sensor data, the second frame sensor data and the interval duration. The fourth processing module is further configured to: determine the first target pose information based on the first frame sensor data and the second frame sensor data when the interval duration is not greater than a preset interval duration; perform interpolation calculation on the first frame sensor data and the second frame sensor data to obtain a third frame sensor data when the interval duration is greater than the preset interval duration; and insert the third frame sensor data between the first frame sensor data and the second frame sensor data; and determine the first target pose information based on the first frame sensor data, the second frame sensor data, and the third frame sensor data.

8. A terminal device, characterized in that, Includes an inertial measurement unit and a processor, wherein, The processor is configured to: acquire a first frame of sensor data from the inertial measurement unit in the terminal device; determine a first timestamp information of the first frame of sensor data and a second timestamp information of the second frame of sensor data, wherein the second frame of sensor data is sensor data acquired before acquiring the first frame of sensor data, and the first frame of sensor data and the second frame of sensor data are two consecutive frames of measurement data generated by the inertial measurement unit; determine the interval duration between the first frame of sensor data and the second frame of sensor data based on the first timestamp information and the second timestamp information; and determine the first target pose information of the terminal device at the time point corresponding to the first timestamp information based on the first frame of sensor data, the second frame of sensor data, and the interval duration. The processor is further configured to: determine the first target pose information based on the first frame sensor data and the second frame sensor data when the interval duration is not greater than a preset interval duration; perform interpolation calculation on the first frame sensor data and the second frame sensor data to obtain a third frame sensor data when the interval duration is greater than the preset interval duration, and insert the third frame sensor data between the first frame sensor data and the second frame sensor data; and determine the first target pose information based on the first frame sensor data, the second frame sensor data, and the third frame sensor data.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a program, wherein when the program is executed, it controls the device containing the non-volatile storage medium to execute the pose information determination method according to any one of claims 1 to 6.

10. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the pose information determination method according to any one of claims 1 to 6.

Images