Timestamp synchronization method and device, equipment and storage medium

Abstract

The invention discloses a timestamp synchronization method, comprising: acquiring a plurality of first angular velocity values of a target IMU in a plurality of preset timestamp intervals, the timestamp intervals at least comprising two timestamps, and each timestamp corresponding to one first angular velocity value; determining a first sequence corresponding to a plurality of first angular velocity values in each timestamp interval; acquiring a plurality of second angular velocity values of the reference IMU according to the plurality of timestamp intervals; obtaining a search interval, wherein the search interval comprises a plurality of time intervals; shifting the timestamps corresponding to the plurality of second angular velocity values according to each time interval to obtain a second sequence corresponding to the plurality of shifted second angular velocity values; and determining a correlation coefficient according to the first sequence and each second sequence, and determining a target offset time delay from the plurality of time intervals according to the correlation coefficient, the target offset time delay being used for performing timestamp synchronization on the reference IMU and the target IMU.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a time synchronization method, apparatus, device and storage medium. Background Art

[0002] With the advancement of technology, more and more devices are equipped with multiple inertial measurement units (IMUs) to improve performance and working accuracy, such as augmented reality (AR) devices, virtual reality (VR) devices, and self-propelled robot platforms. Due to the different installation positions and directions of different IMUs, as well as reasons such as line hardware, different IMUs have different trigger times, resulting in asynchronous timestamps of IMU data. Summary of the invention

[0003] The main purpose of the present application is to provide a time synchronization method, apparatus, device and storage medium for synchronizing timestamps of data from multiple IMUs.

[0004] In a first aspect, an embodiment of the present application provides a timestamp synchronization method, which is applied to an electronic device, wherein the electronic device includes a plurality of IMUs, wherein the IMUs include a reference IMU and a target IMU, and the method includes:

[0005] Acquire a plurality of first angular velocity values ​​of the target IMU within a plurality of preset timestamp intervals, wherein the timestamp interval includes at least two timestamps, and each timestamp corresponds to a first angular velocity value;

[0006] Determine a first sequence corresponding to a plurality of first angular velocity values ​​within each of the timestamp intervals;

[0007] Acquire multiple second angular velocity values ​​of the reference IMU according to the multiple timestamp intervals; acquire a search interval corresponding to the offset delay, the search interval including multiple time intervals;

[0008] offsetting the timestamps corresponding to the plurality of second angular velocity values ​​respectively according to each of the time intervals to obtain second sequences corresponding to the plurality of offset second angular velocity values;

[0009] A correlation coefficient is determined according to the first sequence and each of the second sequences, and a target offset delay is determined from a plurality of the time intervals according to the correlation coefficient, wherein the target offset delay is used to perform timestamp synchronization on the reference IMU and the target IMU.

[0010] In a second aspect, an embodiment of the present application provides a timestamp synchronization device, the timestamp synchronization device is installed in an electronic device, the electronic device includes a plurality of IMUs, the IMUs include a reference IMU and a target IMU, the timestamp synchronization device includes: a first acquisition module, a first calculation module, a second acquisition module, a second calculation module and a time synchronization module;

[0011] A first acquisition module, configured to acquire a plurality of first angular velocity values ​​of the target IMU within a plurality of preset timestamp intervals, wherein the timestamp interval includes at least two timestamps, and each timestamp corresponds to a first angular velocity value;

[0012] A first calculation module, used for determining a first sequence corresponding to a plurality of first angular velocity values ​​within each of the timestamp intervals;

[0013] A second acquisition module is used to acquire a plurality of second angular velocity values ​​of the reference IMU according to a plurality of timestamp intervals; and acquire a search interval corresponding to an offset delay, wherein the search interval includes a plurality of time intervals;

[0014] A second calculation module, configured to respectively shift the timestamps corresponding to the plurality of second angular velocity values ​​according to each of the time intervals, so as to obtain a second sequence corresponding to the plurality of shifted second angular velocity values;

[0015] A time synchronization module is used to determine a correlation coefficient based on the first sequence and each of the second sequences, and to determine a target offset delay from a plurality of the time intervals based on the correlation coefficient, wherein the target offset delay is used to perform timestamp synchronization on the reference IMU and the target IMU.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor;

[0017] The memory is used to store computer programs;

[0018] The processor is used to execute the computer program and implement any one of the timestamp synchronization methods provided in the embodiments of the present application when executing the computer program.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor enables the processor to implement any one of the timestamp synchronization methods provided in the embodiments of the present application.

[0020] The present application discloses a timestamp synchronization method, which is applied to an electronic device. The electronic device includes multiple IMUs, and the IMUs include a reference IMU and a target IMU. The reference IMU is used to perform time synchronization on the target IMU. The method includes: obtaining multiple first angular velocity values ​​of the target IMU within multiple preset timestamp intervals, wherein the timestamp interval includes at least two timestamps, and each timestamp corresponds to a first angular velocity value; determining a first sequence corresponding to the multiple first angular velocity values ​​within each timestamp interval; obtaining multiple second angular velocity values ​​of the reference IMU according to the multiple timestamp intervals; obtaining a search interval corresponding to an offset delay, and the search interval includes multiple time intervals; offsetting the timestamps corresponding to the multiple second angular velocity values ​​according to each time interval, so as to obtain a second sequence corresponding to the multiple offset second angular velocity values; determining a correlation coefficient according to the first sequence and each second sequence, and determining a target offset delay from the multiple time intervals according to the correlation coefficient, and the target offset delay is used to perform timestamp synchronization on the reference IMU and the target IMU. In the above process, the first modulus mean value in each timestamp interval is obtained, and multiple first modulus mean values ​​constitute a first sequence. After the timestamps of the second angular velocity values ​​of the reference IMU are offset according to multiple time intervals, multiple groups of offset second angular velocity values ​​are obtained, thereby obtaining the second modulus mean value of each group of second angular velocity values ​​in each timestamp interval, and multiple second modulus mean values ​​constitute a second sequence. The second sequence and the time interval correspond one to one. After determining the correlation coefficient according to the first sequence and each second sequence, the time interval as the target offset delay can be inferred based on the correlation coefficient. Through the above method, the electronic device can autonomously calculate the target offset delay without the aid of external auxiliary calibration equipment, and synchronize the timestamps of the reference IMU and the target IMU through the target offset delay, thereby improving the efficiency of timestamp synchronization of the data of multiple IMUs. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 is a schematic flow chart of a timestamp synchronization method provided in an embodiment of the present application;

[0023] Figure 2 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0024] Figure 3 is a schematic flow chart of a timestamp offset method provided in an embodiment of the present application;

[0025] Figure 4 is a schematic flow chart of a second sequence generation method provided in an embodiment of the present application;

[0026] Figure 5 is a schematic block diagram of a timestamp synchronization device provided in an embodiment of the present application;

[0027] Figure 6 It is a schematic block diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0030] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0031] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0032] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0033] An inertial measurement unit (IMU) is an electronic device that measures acceleration and angular velocity through a combination of sensors. An IMU usually includes a three-axis accelerometer and a three-axis gyroscope. The three-axis accelerometer is used to measure the three-axis acceleration of an electronic device in its coordinate system, and the three-axis gyroscope is used to measure the three-axis angular velocity of an electronic device in its coordinate system.

[0034] The measurement models of the three-axis gyroscope are:

[0035] ω m (t) = w (t) + b g (t)+n ω (t)

[0036] Among them, ω m (t) is the measured gyroscope angular velocity, w(t) is the actual gyroscope angular velocity, b g Indicates the Bias of the gyroscope, n ω (t) is the Gaussian white noise of the gyroscope.

[0037] As the cost of IMU continues to drop, more and more electronic devices are equipped with multiple IMUs, such as AR devices, VR products, and autonomous robots. Usually, the installation positions and directions of multiple IMUs on electronic devices are different, and due to the influence of hardware circuits, the timestamps of the data of multiple IMUs are not synchronized.

[0038] When aligning the timestamps of multiple IMU data of an electronic device, it is usually necessary to use an internal camera device and an external auxiliary calibration device (manipulator, calibration plate, etc.) to complete the timestamp alignment operation. This method involves a large amount of data calculation, requires a long calculation time and manual assistance, and does not support real-time automatic timestamp alignment of the device.

[0039] To this end, an embodiment of the present application provides a time synchronization method, an electronic device, and a storage medium for simplifying the process of time synchronization of multiple IMUs, so that the electronic device can automatically perform timestamp alignment in real time.

[0040] In an embodiment of the present application, the electronic device includes multiple IMUs. The IMU with the highest sampling frequency among the multiple IMUs can be used as a reference IMU, and other IMUs other than the reference IMU can be used as target IMUs to be evaluated. When performing time synchronization, the offset delay between each target IMU and the reference IMU is calculated separately to achieve timestamp alignment of the multiple target IMUs with the reference IMU.

[0041] For example, the time delay between the target IMU and the reference IMU is t d :

[0042] t target =t base +t d

[0043] t d =t target -t base

[0044] Among them, tbase is the timestamp of the reference IMU data, t target is the timestamp of the target IMU data, t d is the offset delay.

[0045] Assume that the external parameters between the reference IMU and the target IMU are (R target , T target ), R target is the rotation matrix of the electronic device, T target is the translation matrix of the electronic device. The angular velocity values ​​of the reference IMU and the target IMU have the following relationship:

[0046] ω target (t) = R target ω base (t+t d )

[0047] ω target (t) is the first angular velocity value of the target IMU at time t, ω base (t+t d ) is the reference IMU at t+t d The second angular velocity value at the moment, wherein the first angular velocity value and the second angular velocity value are both vectors.

[0048] Since the rotation matrix R target It does not affect the size of the modulus of the angular velocity value. The modulus is the length of the vector of the angular velocity value. Therefore, we can get:

[0049] ||ω target (t)||=‖ω base (t+t d )‖

[0050] Among them, ||ω target (t)|| is the modulus of the first angular velocity value of the target IMU at time t, ‖ω base (t+t d )‖ is the reference IMU at (t+t d ). By using the equal relationship between the modulus length of the offset second angular velocity value and the modulus length of the corresponding first angular velocity value, the correlation between the modulus length of the second angular velocity value offset according to the plurality of offset delays and the modulus length of the first angular velocity value is verified, so as to obtain an ideal offset delay from the plurality of offset delays.

[0051] See also Figure 1 , Figure 1 is a schematic flow chart of a timestamp synchronization method provided by an embodiment of the present application. Figure 1 As shown, the specific steps of the timestamp synchronization method include: S101-S105.

[0052] S101. Acquire a plurality of first angular velocity values ​​of a target IMU within a plurality of preset timestamp intervals, wherein the timestamp interval includes at least two timestamps, and each timestamp corresponds to a first angular velocity value.

[0053] Exemplarily, the data sequence ω of the gyroscope collected by the target IMU is obtained. target (t), the data sequence of the gyroscope consists of angular velocity values ​​with timestamps. During the analysis process, the data of the reference IMU and the target IMU in the same preset target timestamp interval are selected for analysis. The target timestamp interval is divided into multiple preset timestamp intervals, and the target IMU includes at least two first angular velocity values ​​in each timestamp interval, that is, each timestamp interval covers at least the timestamps of the two first angular velocity values.

[0054] If the constant sampling period of the target IMU is [t k ,t k+1 ), t k is the timestamp of the first angular velocity value of the current frame, t k+1 is the timestamp of the first angular velocity value of the next frame, then the timestamp interval is [t k ,t k+1 In the timestamp interval [t k ,t k+1 ] at least includes the first angular velocity value of the current frame and the first angular velocity value of the next frame.

[0055] It should be noted that, in the embodiment of the present application, the timestamp interval may further include multiple first angular velocity values, and different sizes of the timestamp intervals may be set according to different calculation accuracies.

[0056] S102: Determine a first sequence corresponding to a plurality of first angular velocity values ​​in each timestamp interval.

[0057] Exemplarily, the first modulus mean corresponding to each time stamp interval is calculated according to the multiple first angular velocity values ​​in each time stamp interval. k ,t k+1 ]The corresponding first modulus length mean is:

[0058]

[0059] in, for [t k ,t k+1 ]The mean value of the first modulus length in the time period, ω target (t k ) is the first angular velocity value of the current frame, ω target (t k+1 ) is the first angular velocity value of the next frame.

[0060] make If k=1, 2, 3, ..., 8, then there are eight timestamp intervals and eight corresponding first modulus means. The first modulus means are sorted from small to large according to k corresponding to the timestamp intervals, and the first sequence X is obtained as (x1, x2, x3, ..., x8).

[0061] S103, obtaining multiple second angular velocity values ​​of the reference IMU according to multiple timestamp intervals; obtaining a search interval corresponding to the offset delay, the search interval including multiple time intervals.

[0062] Exemplarily, multiple timestamp intervals constitute a target timestamp interval. Since the timestamp of the first angular velocity value obtained in the above step is within the target timestamp interval, multiple second angular velocity values ​​with timestamps within the target timestamp interval are obtained to ensure that there is a correlation between the first angular velocity value and the second angular velocity value.

[0063] In the embodiment of the present application, the time interval may be a value that increases in increments according to a preset gradient. For example, the time interval is t r , the search interval is [n*t r ], t r The size can be set according to the required accuracy, and n is a preset value, for example, n=-2, -1, 0, 1 and 2, so as to realize the calculation of the offset delay at different accuracies.

[0064] S104: offset the timestamps corresponding to the multiple second angular velocity values ​​according to each time interval, so as to obtain second sequences corresponding to the multiple offset second angular velocity values.

[0065] Exemplarily, multiple second angular velocity values ​​before the offset constitute the original angular velocity group of the reference IMU. The timestamps of multiple second angular velocity value pairs in the original angular velocity group are offset according to each time interval to generate multiple offset second angular velocity values, and the multiple offset second angular velocity values ​​obtained by offsetting according to the same time interval constitute an offset angular velocity group. Compared with the original angular velocity group and the multiple offset angular velocity groups, the second angular velocity value and the corresponding offset second angular velocity value are the same, but the timestamps are different.

[0066] After obtaining the offset angular velocity group, the second modulus mean values ​​of multiple offset second angular velocity values ​​in each time stamp interval are obtained, a group of second modulus mean values ​​is obtained corresponding to each offset angular velocity group, and a second sequence is generated.

[0067] Exemplarily, the sequence of the second angular velocity values ​​of the offset angular velocity group is ω base (t+t d ), in [t k ,tk+1 ]The corresponding second mode length mean value within the time range is:

[0068]

[0069] in, for [t k ,t k+1 ]The mean value of the second mode length in the time period, ω base (t+t d ) is the timestamp in [t k ,t k+1 ]The second angular velocity value after the offset within the time period, N base For timestamp in [t k ,t k+1 ]The number of second angular velocity values ​​after the offset within the time period.

[0070] make If k = 1, 2, 3, ..., 8, then there are eight timestamp intervals, and the corresponding second modulus length means are also eight. The second modulus length means are sorted from small to large according to the k corresponding to the timestamp interval to obtain the second sequence for

[0071] S105. Determine a correlation coefficient according to the first sequence and each second sequence, and determine a target offset delay from a plurality of time intervals according to the correlation coefficient, wherein the target offset delay is used to perform timestamp synchronization on a reference IMU and a target IMU.

[0072] Exemplarily, the number of first mean modulus lengths in the first sequence is the same as the number of second mean modulus lengths in the second sequence. According to the correspondence between the first mean modulus length, the second mean modulus length and the timestamp interval, each first mean modulus length corresponds to a second mean modulus length. For example, x1 corresponds to The formula for calculating the correlation coefficient is:

[0073]

[0074] in, is the correlation coefficient corresponding to each second sequence, is the average of multiple first modulus means in the first sequence, is the average of multiple second modulus means in the second sequence.

[0075] After obtaining the correlation coefficient corresponding to each second sequence, the target correlation coefficient is determined according to the multiple correlation coefficients, for example, the largest correlation coefficient is used as the target correlation coefficient. Then, the corresponding second sequence and the time interval corresponding to the second sequence are inversely deduced according to the target correlation coefficient, and the time interval is used as the target offset delay, so as to achieve a time delay offset of the timestamp of the first angular velocity value of the target IMU according to the target offset delay, so as to align the timestamps of the data of the reference IMU and the target IMU.

[0076] In the embodiment of the present application, the technical solution of directly comparing the modulus of a single first angular velocity value and the modulus of a single offset second angular velocity value is not adopted, but the first modulus mean and the second modulus mean corresponding to multiple timestamp intervals are first obtained, and then the target offset delay is determined according to the correlation between the first sequence composed of the multiple first modulus mean and the second sequence composed of the multiple second modulus mean. In this way, the number of samples of data involved in the calculation can be increased, the influence of data fluctuation can be reduced, and the accuracy of the results can be improved.

[0077] In the above process, the first modulus mean value in each timestamp interval is obtained, and multiple first modulus mean values ​​constitute a first sequence. After the timestamps of the second angular velocity values ​​of the reference IMU are offset according to multiple time intervals, multiple groups of offset second angular velocity values ​​are obtained, thereby obtaining the second modulus mean value of each group of second angular velocity values ​​in each timestamp interval, and multiple second modulus mean values ​​constitute a second sequence. The second sequence and the time interval correspond one to one. After determining the correlation coefficient according to the first sequence and each second sequence, the time interval as the target offset delay can be inferred based on the correlation coefficient. Through the above method, the electronic device can autonomously calculate the target offset delay without the aid of external auxiliary calibration equipment, and synchronize the timestamps of the reference IMU and the target IMU through the target offset delay, thereby improving the efficiency of timestamp synchronization of the data of multiple IMUs.

[0078] In order to more clearly introduce the technical solution of the present application, the technical solution of the present application will be described in detail through the following examples. It should be noted that the following examples are used to supplement the technical solution of the present application and are not intended to limit the present application.

[0079] In some embodiments, a first sequence corresponding to multiple first angular velocity values ​​in each timestamp interval is determined, and specific implementation steps include: obtaining the first modulus mean values ​​corresponding to multiple first angular velocity values ​​in each timestamp interval; and generating the first sequence according to the multiple first modulus mean values.

[0080] Exemplarily, the timestamp of the first angular velocity value is read, and the first modulus mean of the first angular velocity value whose timestamp falls within the timestamp interval is calculated. For example, the timestamp intervals are [0:00, 0:10] and [0:10, 0:20], and the timestamps of the plurality of first angular velocity values ​​are 0:03, 0:04, 0:09, 0:12, 0:15, 0:18 and 0:21, respectively. Then, the first modulus mean of the timestamp interval [0:00, 0:10] is calculated based on the first angular velocity values ​​with timestamps of 0:03, 0:04 and 0:09, and the first modulus mean of the timestamp interval [0:10, 0:20] is calculated based on the first angular velocity values ​​with timestamps of 0:12, 0:15 and 0:18. Then, according to the order of the timestamp intervals, the corresponding first modulus means are arranged to obtain a first sequence. For example, the first modulus means in the timestamp interval [0:00, 0:10] are arranged before the first modulus means in the timestamp interval [0:10, 0:20].

[0081] In some embodiments, see Figure 3 , Figure 3 The following is a schematic flow chart of a timestamp offset method provided by an embodiment of the present application. Figure 3 As shown, the specific implementation steps of "offsetting the timestamps corresponding to the multiple second angular velocity values ​​according to each time interval" in step S104 include: S1041-S1042.

[0082] S1041. Add each time interval to the timestamps of the multiple second angular velocity values ​​to obtain a shifted second angular velocity value.

[0083] S1042: Divide the shifted second angular velocity values ​​into a plurality of shifted angular velocity groups according to the time intervals, wherein the shifted angular velocity groups correspond to the time intervals in a one-to-one manner.

[0084] Exemplarily, each time interval is added to the timestamps of multiple second angular velocity values, and the offset second angular velocity values ​​obtained after adding the same time interval are divided into the same offset angular velocity group, and each time interval generates a corresponding offset angular velocity group.

[0085] Specifically, a time interval is added to multiple second angular velocity values ​​to obtain a group of offset angular velocity groups. For example, the timestamps of the multiple second angular velocity values ​​are 0:00, 0:01, 0:02, and 0:03, respectively. When added to the time interval "0:01", the timestamps of the offset angular velocity group obtained are 0:01, 0:02, 0:03, and 0:04; when added to the time interval "0:02", the timestamps of the offset angular velocity group obtained are 0:02, 0:03, 0:04, and 0:05. The offset only changes the timestamps of the second angular velocity values, and does not change the magnitude of the second angular velocity values.

[0086] In the present application, the timestamp of the second angular velocity value of the reference IMU is hypothetically offset by multiple preset time intervals to obtain multiple offset angular velocity groups, and then the correlation between each offset angular velocity group and multiple first angular velocity values ​​is calculated, so that the offset angular velocity group closest to the first angular velocity value can be inferred based on the calculation result of the correlation, and then the time interval used to generate the offset angular velocity group is used as the target offset delay. In this way, the electronic device can autonomously calculate the offset delay between the reference IMU and the target IMU without the assistance of external devices, thereby timestamping the data of its reference IMU and target IMU.

[0087] In some embodiments, see Figure 4 , Figure 4 A schematic flow chart of a timestamp offset provided by an embodiment of the present application is shown. Figure 4 As shown, the specific implementation steps of "obtaining a second sequence corresponding to a plurality of offset second angular velocity values" in step S104 include: S1043-S1045.

[0088] S1043. In each offset angular velocity group, obtain a plurality of offset second angular velocity values ​​in each time stamp interval.

[0089] S1044: Determine a second modulus mean value according to a plurality of offset second angular velocity values ​​in each time stamp interval.

[0090] S1045. Generate a second sequence according to the multiple second modulus length averages, where the second sequence corresponds to the offset angular velocity group one by one.

[0091] In the embodiment of the present application, a corresponding second sequence is determined according to each offset angular velocity group. In the offset angular velocity group, if any timestamp interval contains a timestamp of the offset second angular velocity value, then these offset second angular velocity values ​​are used to determine the second modulus mean corresponding to the timestamp interval, and the second modulus mean of all timestamp intervals generates a second sequence. The order of the second modulus mean in the second sequence is the same as the order of the corresponding timestamp intervals. The order of the timestamp intervals is from front to back according to the corresponding time order of the timestamp intervals. For example, the timestamp interval [12:00, 12:10] is arranged after the timestamp interval [12:10, 12:20].

[0092] In the above process, the number of second modulus means in the second sequence is the same as the number of timestamp intervals, that is, the same as the number of first modulus means in the first sequence. For the same timestamp interval, the first modulus mean corresponds to a second modulus mean, thereby calculating the correlation between the first sequence and the second sequence. The first modulus mean and the second modulus mean are obtained by using multiple first angular velocity values ​​and multiple second angular velocity values ​​in the timestamp interval, respectively, so that the interference of data volatility can be reduced to improve the accuracy of the results.

[0093] In some embodiments, a target offset delay is determined from multiple time intervals based on a correlation coefficient, including: determining a target correlation coefficient based on the correlation coefficient; determining a target second sequence based on the target correlation coefficient; obtaining a target time interval corresponding to the target second sequence, and determining the target time interval as the target offset delay, where the target time interval is one of the time intervals.

[0094] For example, the search interval is [-2, -1, 0, 1, 2], and five second sequences are obtained according to the five values ​​in the search interval, namely, Y -2 , Y -1 , Y 0 , Y 1 and Y 2 According to Y -2 , Y -1 , Y 0 , Y 1 and Y 2 The five correlation coefficients calculated with X are r -2 、r -1 、r 0 、r 1 and r 2 , where r 1 When the five correlation coefficients are the largest, then r 1 As the target correlation coefficient. 1 The corresponding second sequence is Y 1 , Y 1 The corresponding time interval "1" is the target time interval, that is, the target offset delay t is determined. d = 1. The above process enables the electronic device to automatically obtain the offset delay in real time, so as to automatically perform time stamp alignment in real time.

[0095] In some embodiments, obtaining multiple second angular velocity values ​​of a reference IMU according to multiple timestamp intervals includes: determining a target timestamp interval according to the multiple timestamp intervals; and obtaining multiple second angular velocity values ​​of the reference IMU within the target timestamp interval.

[0096] For example, when acquiring multiple second angular velocity values ​​of the reference IMU, if only the second angular velocity values ​​with timestamps within multiple timestamp intervals are acquired, after the timestamps of the second angular velocity values ​​are time-delayed and offset, the timestamp interval at one end may have data missing, resulting in reduced accuracy of the calculation result. Therefore, it is also necessary to acquire a preset number of second angular velocity values ​​with timestamps adjacent to the timestamp intervals at the front and rear ends.

[0097] In one embodiment, the lower limit value of the target timestamp interval is less than the minimum lower limit value of the multiple timestamp intervals, and the upper limit value of the target timestamp interval is greater than the maximum upper limit value of the multiple timestamp intervals. For example, among the multiple timestamp intervals, the time range corresponding to each timestamp interval is [0:01, 0:02], [0:02, 0:03] and [0:03, 0:04] respectively, then the time range corresponding to the target timestamp interval can be [0:00, 0:05] or [0:00, 0:06].

[0098] It can be understood that if the timestamp interval is not a continuous timestamp interval, the time range corresponding to the target timestamp interval must include the time range between the lower limit values ​​and upper limit values ​​of multiple timestamp intervals, and the lower limit value of the target timestamp interval is less than the minimum lower limit value of the multiple timestamp intervals, and the upper limit value of the target timestamp interval is greater than the maximum upper limit value of the multiple timestamp intervals.

[0099] In another embodiment, the target timestamp interval composed of multiple timestamp intervals may further include two additional timestamp intervals, and the two additional timestamp intervals are adjacent to the timestamp intervals at the front and rear ends, respectively. When the multiple second angular velocity values ​​whose timestamps are obtained are within the target timestamp interval, they include multiple second angular velocity values ​​whose timestamps are within the timestamp interval, and multiple second angular velocity values ​​whose timestamps are within the two additional timestamp intervals. In this way, it is possible to prevent the problem of missing data in the timestamp interval at one end after the second angular velocity value is time-delayed and offset, so as to improve the accuracy of the calculation process.

[0100] In some embodiments, after determining the target offset delay from the search interval according to the correlation coefficient, it also includes: offsetting the timestamp of the first angular velocity value according to the target offset delay; determining the rotation matrix and translation matrix of the electronic device according to the offset first angular velocity value and the second angular velocity value corresponding to the offset first angular velocity value.

[0101] For example, see Figure 2 , Figure 2 A schematic diagram of an electronic device provided by an embodiment of the present application is shown. Figure 2 As shown, the electronic device 100 includes a reference IMU 101 and a target IMU 102. The external parameter between the reference IMU 101 and the target IMU 102 is (R target , Ttarget ), R target is the rotation matrix of the electronic device, T target is the translation matrix of the electronic device. The rotation matrix R target and the translation matrix T target , used to calculate the physical position relationship between the reference IMU and the target IMU, and also used to fuse or compare the data of the reference IMU and the target IMU.

[0102] Since there is a time delay between the reference IMU 101 and the target IMU 102, after the target IMU 102 is subjected to a time delay offset, the angular velocities of the reference IMU 101 and the target IMU 102 have the following relationship:

[0103] ω target (tt d )=R target ω base (t)

[0104] Among them, ω base (t) is the second angular velocity value of the reference IMU 101 at time t, ω target (tt d ) is the target IMU102 at tt d The first angular velocity value at the moment.

[0105] Thus, the offset first angular velocity value ω can be used to target (tt d ) and the second angular velocity value ω corresponding to the offset first angular velocity value base (t), find the rotation matrix R target . T target The solution of is also a conventional technical means, which will not be described here.

[0106] In some embodiments, obtaining a search interval corresponding to the offset delay, the search interval including multiple time intervals, includes: obtaining a preset basic interval and multiple interval coefficients; generating multiple time intervals according to the basic interval and the multiple interval coefficients.

[0107] For example, let t d The search interval is T[-n*t r , (-n+1)*t r , ..., (n-1)*t r , n*t r ], the basic interval is t r , n is the interval coefficient, and the search interval includes 2n+1 time intervals. r It can be set according to the accuracy requirement. If the accuracy requirement of the offset delay is high, t rCan be set to 0.1 (in ms). If the accuracy of the offset delay is low, t r Can be set to 1 (in ms), t r In this way, by setting different data precisions, different computing requirements are achieved, taking into account both computing precision and computing speed, so that the electronic device can adapt to different scenarios and reasonably allocate computing resources.

[0108] In some embodiments, before obtaining the first data sequence of the target IMU, the method further includes: obtaining the sampling frequencies of multiple IMUs; and determining the IMU with the largest sampling frequency as the reference IMU. In this way, the sampling density of the reference IMU is ensured to be greater than that of the target IMU, which helps to improve the accuracy of the result.

[0109] In other embodiments, an IMU specified by the user may be used as the reference IMU, or an IMU with a preset sampling frequency may be used as the reference IMU.

[0110] See also Figure 5 , see Figure 5 , Figure 5 The embodiment of the present application further provides a schematic block diagram of a timestamp synchronization device, wherein the timestamp synchronization device 300 is used to execute the above-mentioned timestamp synchronization method. The timestamp synchronization device can be configured in a server or an electronic device.

[0111] The server may be an independent server or a server cluster, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The electronic device may be a terminal device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a user digital assistant, and a wearable device.

[0112] like Figure 5 As shown, the timestamp synchronization device 300 includes: a first acquisition module 301 , a first calculation module 302 , a second acquisition module 303 , a second calculation module 304 and a time synchronization module 305 .

[0113] The first acquisition module 301 is used to acquire a plurality of first angular velocity values ​​of a target IMU within a plurality of preset timestamp intervals, wherein the timestamp interval includes at least two timestamps, and each timestamp corresponds to a first angular velocity value.

[0114] In some embodiments, before being used to obtain the first data sequence of the target IMU, the first acquisition module 301 is further used to: obtain the sampling frequencies of multiple IMUs; and determine the IMU with the largest sampling frequency as the reference IMU.

[0115] The first calculation module 302 is used to determine a first sequence corresponding to a plurality of first angular velocity values ​​in each time stamp interval.

[0116] In some embodiments, when the first calculation module 302 is used to determine the first sequence corresponding to multiple first angular velocity values ​​in each timestamp interval, it is specifically used to implement: obtaining the first modulus mean corresponding to the multiple first angular velocity values ​​in each timestamp interval; generating the first sequence according to the multiple first modulus mean values.

[0117] The second acquisition module 303 is used to acquire multiple second angular velocity values ​​of the reference IMU according to multiple timestamp intervals; and acquire a search interval corresponding to the offset delay, where the search interval includes multiple time intervals.

[0118] In some embodiments, when the second acquisition module 303 is used to obtain multiple second angular velocity values ​​of the benchmark IMU according to multiple timestamp intervals, it is specifically used to implement: determining a target timestamp interval according to multiple timestamp intervals; and obtaining multiple second angular velocity values ​​of the benchmark IMU within the target timestamp interval.

[0119] In some embodiments, the second acquisition module 303 is used to obtain the search interval corresponding to the offset delay, and when the search interval includes multiple time intervals, it is specifically used to: obtain a preset basic interval and multiple interval coefficients; generate multiple time intervals according to the basic interval and multiple interval coefficients.

[0120] The second calculation module 304 is used to shift the timestamps corresponding to the multiple second angular velocity values ​​according to each time interval, so as to obtain a second sequence corresponding to the multiple shifted second angular velocity values.

[0121] In some embodiments, when the second calculation module 304 is used to offset the timestamps corresponding to multiple second angular velocity values ​​according to each time interval, it is specifically used to implement: adding each time interval to the timestamps of the multiple second angular velocity values ​​to obtain the offset second angular velocity value; dividing the offset second angular velocity value into multiple offset angular velocity groups according to the time interval, and the offset angular velocity groups and the time intervals correspond one to one.

[0122] In some embodiments, when the second calculation module 304 is used to obtain a second sequence corresponding to multiple offset second angular velocity values, it is specifically used to implement: in each offset angular velocity group, respectively obtain multiple offset second angular velocity values ​​in each timestamp interval; determine the second modulus mean according to the multiple offset second angular velocity values ​​in each timestamp interval; generate a second sequence according to the multiple second modulus mean values, and the second sequence corresponds to the offset angular velocity group one by one.

[0123] The time synchronization module 305 is used to determine a correlation coefficient according to the first sequence and each second sequence, and determine a target offset delay from multiple time intervals according to the correlation coefficient, and the target offset delay is used to perform timestamp synchronization on the reference IMU and the target IMU.

[0124] In some embodiments, the time synchronization module 305 is used to determine the target offset delay from multiple time intervals based on the correlation coefficient, and is specifically used to implement: determining the target correlation coefficient based on the correlation coefficient; determining the target second sequence based on the target correlation coefficient; obtaining the target time interval corresponding to the target second sequence, and determining the target time interval as the target offset delay, where the target time interval is one of the time intervals.

[0125] In some embodiments, after being used to determine the target offset delay from multiple time intervals based on the correlation coefficient, the time synchronization module 305 is further used to: offset the timestamp of the first angular velocity value according to the target offset delay; determine the rotation matrix and translation matrix of the electronic device based on the offset first angular velocity value and the second angular velocity value corresponding to the offset first angular velocity value.

[0126] See also Figure 6 , Figure 6 The block diagram of an electronic device is shown. Figure 6 As shown, an embodiment of the present application proposes an electronic device 20, which includes a memory 21, a processor 22, a program stored in the memory and executable on the processor, and a data bus 23 for realizing connection and communication between the processor 21 and the memory 22. When the program is executed by the processor, the steps of the timestamp synchronization method as described in any one of the embodiments of the present application are realized.

[0127] Specifically, for example, the following Figure 1The specific steps shown are: obtaining multiple first angular velocity values ​​of the target IMU within multiple preset timestamp intervals, wherein the timestamp interval includes at least two timestamps, and each timestamp corresponds to a first angular velocity value; obtaining a first modulus mean corresponding to the multiple first angular velocity values ​​within each timestamp interval, and the multiple first modulus mean values ​​constitute a first sequence; obtaining multiple second angular velocity values ​​of the reference IMU according to the multiple timestamp intervals; obtaining a search interval corresponding to the offset delay, and the search interval includes multiple time intervals; offsetting the timestamps corresponding to the multiple second angular velocity values ​​according to each time interval, and obtaining a second modulus mean of the multiple offset second angular velocity values ​​within each timestamp interval, and the multiple second modulus mean values ​​constitute a second sequence; determining a correlation coefficient according to the first sequence and each second sequence; and determining the target offset delay from the search interval according to the correlation coefficient.

[0128] In some embodiments, when the processor is used to offset the timestamps corresponding to multiple second angular velocity values ​​according to each time interval, it is specifically used to implement: adding each time interval to the timestamps of the multiple second angular velocity values; dividing the offset second angular velocity values ​​into multiple offset angular velocity groups according to the time intervals, and the offset angular velocity groups and the time intervals correspond one to one.

[0129] In some embodiments, when the processor is used to obtain a second modulus mean of multiple offset second angular velocity values ​​in each timestamp interval, and the multiple second modulus mean values ​​constitute a second sequence, it is specifically used to implement: in each offset angular velocity group, respectively obtain a plurality of offset second angular velocity values ​​in each timestamp interval; determine the second modulus mean based on the plurality of offset second angular velocity values ​​in each timestamp interval, wherein the second sequence constituted by the plurality of second modulus mean values ​​corresponds one-to-one to the offset angular velocity group.

[0130] In some embodiments, the processor is used to determine the target offset delay from the search interval based on the correlation coefficient, specifically to implement: determining the target correlation coefficient based on the correlation coefficient; determining the corresponding second sequence based on the target correlation coefficient; obtaining the target time interval corresponding to the corresponding second sequence, and determining the target time interval as the target offset delay, the target time interval being one of the time intervals.

[0131] In some embodiments, when the processor is used to obtain multiple second angular velocity values ​​of the benchmark IMU according to multiple timestamp intervals, it is specifically used to implement: determining a target timestamp interval according to multiple timestamp intervals, the time range corresponding to the target timestamp interval is larger than the time ranges corresponding to the multiple timestamp intervals; obtaining multiple second angular velocity values ​​of the benchmark IMU within the target timestamp interval.

[0132] In some embodiments, after the processor is used to implement determining the target offset delay from the search interval based on the correlation coefficient, it is also specifically used to implement: offsetting the timestamp of the first angular velocity value according to the target offset delay; and determining the rotation matrix and translation matrix of the electronic device based on the offset first angular velocity value and the second angular velocity value corresponding to the offset first angular velocity value.

[0133] In some embodiments, when the processor is used to obtain a search interval corresponding to an offset delay, and the search interval includes multiple time intervals, it is specifically used to: obtain a preset basic interval and multiple interval coefficients; and generate multiple time intervals according to the basic interval and the multiple interval coefficients.

[0134] In some embodiments, before the processor is used to obtain the first data sequence of the target IMU, it is also specifically used to: obtain the sampling frequencies of multiple IMUs; and determine the IMU with the largest sampling frequency as the reference IMU.

[0135] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A timestamp synchronization method, characterized in that: Applied to an electronic device, the electronic device includes a plurality of IMUs, the IMUs include a reference IMU and a target IMU, the method includes: Acquire a plurality of first angular velocity values ​​of the target IMU within a plurality of preset timestamp intervals, wherein the timestamp interval includes at least two timestamps, and each timestamp corresponds to a first angular velocity value; Determine a first sequence corresponding to a plurality of first angular velocity values ​​within each of the timestamp intervals; Acquire multiple second angular velocity values ​​of the reference IMU according to the multiple timestamp intervals; acquire a search interval corresponding to the offset delay, the search interval including multiple time intervals; offsetting the timestamps corresponding to the plurality of second angular velocity values ​​respectively according to each of the time intervals to obtain second sequences corresponding to the plurality of offset second angular velocity values; A correlation coefficient is determined according to the first sequence and each of the second sequences, and a target offset delay is determined from a plurality of the time intervals according to the correlation coefficient, wherein the target offset delay is used to perform timestamp synchronization on the reference IMU and the target IMU.

2. The timestamp synchronization method according to claim 1, characterized in that: The determining a first sequence corresponding to a plurality of first angular velocity values ​​within each of the timestamp intervals comprises: Obtaining a first modulus average corresponding to a plurality of first angular velocity values ​​within each of the timestamp intervals; The first sequence is generated according to the multiple first modulus length means.

3. The timestamp synchronization method according to claim 1, characterized in that: The offsetting the timestamps corresponding to the plurality of second angular velocity values ​​respectively according to each of the time intervals includes: Adding each of the time intervals to the timestamps of the plurality of second angular velocity values ​​to obtain a shifted second angular velocity value; The shifted second angular velocity values ​​are divided into a plurality of shifted angular velocity groups according to the time intervals, and the shifted angular velocity groups correspond to the time intervals in a one-to-one manner.

4. The timestamp synchronization method according to claim 3, characterized in that: The obtaining of a second sequence corresponding to a plurality of offset second angular velocity values ​​comprises: In each of the offset angular velocity groups, a plurality of offset second angular velocity values ​​within each of the timestamp intervals are respectively acquired; Determine the second modulus mean value according to a plurality of second angular velocity values ​​after shifting in each of the timestamp intervals; The second sequence is generated according to a plurality of second modulus length averages, and the second sequence corresponds to the offset angular velocity group in a one-to-one manner.

5. The timestamp synchronization method according to claim 1, characterized in that: The determining the target offset delay from the plurality of time intervals according to the correlation coefficient comprises: determining a target correlation coefficient according to the correlation coefficient; Determine a target second sequence according to the target correlation coefficient; A target time interval corresponding to the target second sequence is acquired, and the target time interval is determined as a target offset delay, where the target time interval is one of the time intervals.

6. The timestamp synchronization method according to claim 1, characterized in that: The obtaining a plurality of second angular velocity values ​​of the reference IMU according to the plurality of timestamp intervals comprises: Determine a target timestamp interval according to the multiple timestamp intervals; Acquire multiple second angular velocity values ​​of the reference IMU within the target timestamp interval.

7. The timestamp synchronization method according to claim 1, characterized in that: After determining the target offset delay from the search interval according to the correlation coefficient, the method further includes: offsetting the timestamp of the first angular velocity value according to the target offset delay; A rotation matrix and a translation matrix of the electronic device are determined according to the offset first angular velocity value and the second angular velocity value corresponding to the offset first angular velocity value.

8. The time synchronization method according to claim 1, characterized in that: The search interval corresponding to the obtained offset delay includes multiple time intervals, including: Get the preset basic interval and multiple interval coefficients; A plurality of the time intervals are generated according to the basic interval and a plurality of the interval coefficients.

9. The time synchronization method according to claim 1, characterized in that: Before acquiring the first data sequence of the target IMU, the method further includes: Obtaining sampling frequencies of multiple IMUs; The IMU with the largest sampling frequency is determined as the reference IMU.

10. A timestamp synchronization device, characterized in that: Installed in an electronic device, the electronic device includes a plurality of IMUs, the IMUs include a reference IMU and a target IMU, and the timestamp synchronization device includes: A first acquisition module, configured to acquire a plurality of first angular velocity values ​​of the target IMU within a plurality of preset timestamp intervals, wherein the timestamp interval includes at least two timestamps, and each timestamp corresponds to a first angular velocity value; A first calculation module, used for determining a first sequence corresponding to a plurality of first angular velocity values ​​within each of the timestamp intervals; A second acquisition module is used to acquire a plurality of second angular velocity values ​​of the reference IMU according to a plurality of timestamp intervals; and acquire a search interval corresponding to an offset delay, wherein the search interval includes a plurality of time intervals; A second calculation module, configured to respectively shift the timestamps corresponding to the plurality of second angular velocity values ​​according to each of the time intervals, so as to obtain a second sequence corresponding to the plurality of shifted second angular velocity values; A time synchronization module is used to determine a correlation coefficient based on the first sequence and each of the second sequences, and to determine a target offset delay from a plurality of the time intervals based on the correlation coefficient, wherein the target offset delay is used to perform timestamp synchronization on the reference IMU and the target IMU.

11. An electronic device, characterized in that: The electronic device comprises a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the timestamp synchronization method according to any one of claims 1 to 7 when executing the computer program.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the timestamp synchronization method according to any one of claims 1 to 7.

Images