Sensor timestamp calibration method, device, smart terminal and storage medium

Abstract

This invention discloses a sensor timestamp calibration method, apparatus, smart terminal, and storage medium. Upon receiving an interrupt signal from the sensor, an interrupt timestamp is generated and the sequence number of the interrupt frame is obtained. Then, the sensor's data frames are read using a first-in-first-out (FIFO) mode to obtain a data frame set. The timestamps of the data frames in the data frame set are calibrated based on the interrupt frame sequence number and interrupt timestamp. Compared with existing technologies, this method can calibrate the timestamp of each data frame from the sensor.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a sensor timestamp calibration method, apparatus, smart terminal, and storage medium. Background Technology

[0002] In autonomous driving systems, multi-line LiDAR, cameras, and IMU (Inertial Measurement Unit) sensors are commonly used, and localization is achieved by fusing the timestamps from the IMU sensors. However, since cars travel at speeds of tens of miles per hour, even a time error of tens of milliseconds in the IMU sensor's timestamps can have a significant impact on the perception of the surrounding environment.

[0003] When acquiring data frames from IMU sensors, issues such as frame loss and time synchronization problems between the IMU sensor's timestamp and the main control terminal frequently occur. Existing timestamp synchronization methods only synchronize the IMU sensor's timestamp with the timestamps of other sensors, without calibrating the IMU sensor's timestamp, resulting in inaccurate IMU sensor timestamps. Summary of the Invention

[0004] The main objective of this invention is to provide a sensor timestamp calibration method, apparatus, smart terminal, and storage medium, aiming to solve the problem of inaccurate sensor timestamps.

[0005] To achieve the above objectives, a first aspect of the present invention provides a sensor timestamp calibration method, the method comprising:

[0006] When an interrupt signal from the sensor is received, an interrupt timestamp is generated and the sequence number of the interrupt frame is obtained; the sensor's data frames are read using a first-in-first-out (FIFO) mode to obtain a data frame set; based on the sequence number and interrupt timestamp of the interrupt frame, the timestamps of the data frames in the data frame set are calibrated.

[0007] Optionally, after obtaining the data frame set, the following may also be included:

[0008] Obtain the number of data frames in the sensor;

[0009] When the number is greater than or equal to a preset threshold, read the preset threshold number of data frames, save the data frames to the data frame set and calibrate the timestamps of the data frames in the data frame set, and return to obtain the number of data frames in the sensor for cyclic reading until the number is less than the preset threshold.

[0010] Optionally, calibrating the timestamps of data frames in the data frame set based on the sequence number and timestamp of the interrupted frame includes:

[0011] Obtain the sequence number difference between the sequence number of each data frame in the data frame set and the sequence number of the interrupted frame;

[0012] Based on the sensor's frame rate and the sequence number difference, the time difference between each data frame in the data frame set and the interrupt frame is obtained;

[0013] Based on the interruption timestamp and the time difference between each data frame and the interruption frame, the timestamp of each data frame in the data frame set is set to calibrate the timestamp of the data frames in the data frame set.

[0014] Optionally, it also includes: querying the number of data frames in the sensor at a preset waiting time interval; when the number is greater than or equal to the preset threshold, reading the preset threshold number of data frames in the sensor and calibrating the timestamp of the data frames.

[0015] Optionally, the preset waiting time can be determined based on the sensor's frame rate.

[0016] A second aspect of the present invention provides a sensor timestamp calibration apparatus, wherein the apparatus comprises:

[0017] The data reading module is used to read data frames from the sensor in a first-in-first-out (FIFO) mode to obtain a data frame set.

[0018] The calibration module is used to generate an interrupt timestamp and obtain the sequence number of the interrupt frame when an interrupt signal from the sensor is received; and to calibrate the timestamps of the data frames in the data frame set based on the sequence number and the interrupt timestamp of the interrupt frame.

[0019] Optionally, the data reading module includes a reading unit, which is used to obtain the number of data frames in the sensor; when the number is greater than or equal to a preset threshold, read the preset threshold number of data frames, save the data frames to the data frame set and calibrate the timestamps of the data frames in the data frame set, and return to obtain the number of data frames in the sensor for cyclic reading until the number is less than the preset threshold.

[0020] Optionally, the calibration module includes a frame sequence calibration unit, which is used to obtain the sequence number difference between the sequence number of each data frame in the data frame set and the sequence number of the interrupted frame; obtain the time difference between each data frame in the data frame set and the interrupted frame based on the sensor's frame rate and the sequence number difference; and set the timestamp of each data frame in the data frame set according to the interrupted timestamp and the time difference between each data frame and the interrupted frame, so as to calibrate the timestamp of the data frames in the data frame set.

[0021] A third aspect of the present invention provides a smart terminal, the smart terminal including a memory, a processor, and a sensor timestamp calibration program stored in the memory and executable on the processor, wherein the sensor timestamp calibration program, when executed by the processor, implements any of the steps of the sensor timestamp calibration method.

[0022] A fourth aspect of the present invention provides a computer-readable storage medium storing a sensor timestamp calibration program, wherein the sensor timestamp calibration program, when executed by a processor, implements any of the steps of the sensor timestamp calibration method described above.

[0023] As can be seen from the above, after obtaining the interrupt signal from the sensor, this invention generates an interrupt timestamp and obtains the sequence number of the interrupt frame. Then, it reads the sensor's data frames using a first-in-first-out (FIFO) mode to obtain a data frame set. The timestamps of the data frames in the data frame set are calibrated based on the interrupt frame sequence number and the interrupt timestamp. Because the timestamp of each data frame in the data frame set is calibrated based on the interrupt timestamp generated during the interrupt, the superposition and amplification of time deviations are prevented, ensuring accurate calibration of the timestamp of each data frame from the sensor. Furthermore, the FIFO mode allows for batch reading of sensor data frames, resulting in fast data frame reading speed and improved efficiency of the main control terminal. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a diagram illustrating the application scenario of sensor timestamp calibration provided in an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram illustrating the specific process of the sensor timestamp calibration method provided in this embodiment of the invention;

[0027] Figure 3 This is a schematic diagram illustrating the specific process of calibrating the timestamps of data frames in the data frame set;

[0028] Figure 4 This is a schematic diagram illustrating the specific process of intelligent reading;

[0029] Figure 5 This is a schematic diagram of the sensor timestamp calibration device provided in an embodiment of the present invention;

[0030] Figure 6 This is a block diagram illustrating the internal structure of a smart terminal provided in an embodiment of the present invention. Detailed Implementation

[0031] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0032] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0033] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0034] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0035] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0038] Various sensors and hardware devices (such as GPS, LiDAR, cameras, and inertial sensors) are used in autonomous driving systems, SLAM (Simultaneous Localization and Mapping), and AR navigation. There is inevitably a time lag between the time sensor data is collected and the time it is used by the main control terminal (such as the vehicle's electronic control unit) running the autonomous driving computing platform. Typically, the timestamps of individual sensor data are calibrated using the timestamps of the IMU sensor data as a benchmark to achieve consistency, before these sensor data can be fused and various application functions can be implemented. Therefore, timestamps are crucial for sensor data fusion, prediction, and decision-making modules. If an autonomous driving system suffers from trigger delays, transmission delays, or the lack of an accurate synchronized clock, it will have a significant impact on the perception of the surrounding environment and its own localization.

[0039] However, when reading data frames from the IMU sensor, the timestamp of the IMU sensor is often inaccurate. Therefore, it is necessary to calibrate the timestamp of the IMU sensor data frames according to the time of the main control terminal.

[0040] Common methods for the master terminal to read IMU sensor data frames include: FIFO (First Input First Out) mode, polling mode, and interrupt mode.

[0041] In FIFO mode, the master terminal queries the number of data frames in the IMU sensor registers at a set frequency. When the number of data frames stored in the registers reaches a preset threshold, it reads a preset threshold number of IMU sensor data frames in batches, resulting in fast reading speed and improved processing efficiency of the master terminal. However, since the time when the master terminal first acquires an IMU sensor data frame is used as a reference, the timestamps of subsequent IMU sensor data frames are calibrated based on this reference and the time interval between generating the IMU sensor data frames. When there is a time discrepancy between the master terminal and the IMU sensor, the time discrepancy will accumulate and increase as the number of IMU sensor data frame acquisitions increases, leading to inaccurate timestamps on the calibrated IMU sensor data frames.

[0042] In polling mode, the master control terminal reads IMU sensor data frame by frame at a certain frequency. Because the frequency at which the master control terminal acquires IMU sensor data is difficult to control—too high a frequency wastes the master control terminal's resources, while too low a frequency results in untimely acquisition of IMU sensor data, causing data overflow in the IMU sensor's registers, leading to frame drops, incorrect data frame sequence numbers, and inaccurate timestamps on calibrated data frames. Furthermore, in the multi-tasking platform (Linux / Android) of the autonomous driving system, the time spent switching between tasks can also cause untimely acquisition of IMU sensor data frames, similarly leading to data overflow in the IMU sensor's registers and frame drops.

[0043] In interrupt mode, the IMU sensor generates an interrupt signal every time it generates a data frame. Upon receiving this interrupt signal, the master terminal calls an interrupt function to read a frame of IMU sensor data. While the timestamps of the IMU sensor data frames can be calibrated promptly using the current master terminal time, preventing the accumulation and amplification of time deviations, the frequent interrupts caused by each frame's interrupt signal mean the master terminal might be in the middle of acquiring IMU sensor data and unable to respond to the next interrupt signal in time. This can lead to the next interrupt signal being blocked, preventing timely reading of IMU sensor data and resulting in dropped frames. Similarly, if an event is generated within the interrupt function to wake up other threads to acquire IMU sensor data, the same issue of delayed data acquisition can occur.

[0044] To address the aforementioned problems, this invention provides a sensor timestamp calibration method that combines the advantages of both FIFO and interrupt modes. When a sensor interrupts, the FIFO mode is used to read data frames in batches to obtain a data frame set. The timestamp of each data frame in the set is then calibrated based on the interrupt timestamp generated during the interruption. Since an interrupt is generated when the number of data frames in the sensor register reaches a preset threshold, there is no accumulation or increase in time deviations, ensuring the accuracy of the timestamp for each data frame. Furthermore, the interrupt triggering mechanism eliminates the need for frequent checks by the main control terminal, improving its processing efficiency. This method can be applied not only to IMU sensors but also to other types of sensors, such as line laser rangefinders and GPS sensors, provided the sensor has an interrupt mechanism and supports FIFO mode.

[0045] Exemplary methods

[0046] This invention provides a sensor timestamp calibration method, deployed on the autonomous driving computing platform of an unmanned driving system, to calibrate the timestamps of received IMU sensor data, ensuring timestamp accuracy when fusing data from other sensors. Of course, it can also be deployed on other types of master control terminals, such as smart terminals, microcontrollers, and MCUs (microcontroller units).

[0047] The main control terminal in this embodiment is as follows: Figure 1 As shown, the main control terminal is a microcontroller running an autonomous driving computing platform, and the IMU sensor is installed next to the vehicle engine. It includes: an intelligent polling and frame sequence prediction unit, a read / write unit (W / R), and an interrupt handling unit (IRQ). The read / write unit reads data frames from the IMU sensor, and the interrupt handling unit receives interrupt signals from the Interrupt Manager on the IMU sensor and sends signals to the read / write unit. The read / write unit connects to the IMU sensor's read / write interface, such as an RS232 / 422 / 485 serial port module. Through this interface, the number of data frames stored in the IMU sensor's registers (Regs Array) can be queried, and data frames can be read from the IMU sensor's registers.

[0048] Specifically, such as Figure 2 As shown, the sensor timestamp calibration method in this embodiment includes the following steps:

[0049] Step S100: When an interrupt signal from the sensor is received, an interrupt timestamp is generated and the sequence number of the interrupt frame is obtained; the sensor's data frames are read using a first-in-first-out (FIFO) mode to obtain a data frame set;

[0050] Specifically, the main control terminal pre-sets the interrupt threshold of the sensor by calling the sensor's interface, so that an interrupt signal is generated only when the number of data frames in the sensor's register reaches the preset threshold, rather than generating an interrupt signal for each data frame generated.

[0051] refer to Figure 1 As shown, assuming the preset threshold is 5, when the number of data frames stored in the sensor's register (Regs Array) reaches 5, the sensor's interrupt manager will generate an interrupt signal. The interrupt processing unit of the main control terminal can receive the interrupt signal and send a signal to the read / write unit to read the sensor data frames.

[0052] This invention discloses a novel mode combining interrupt mode and FIFO mode for reading IMU sensor data frames. Unlike the FIFO mode in related technologies, when the number of data frames in the sensor's register reaches a preset threshold, the sensor actively generates an interrupt signal to notify the master control terminal. The master control terminal then uses FIFO mode to read the data frames and calibrates the data frame timestamps based on the time the master control terminal receives the interrupt signal. This prevents time differences from being superimposed and amplified, and also controls the frequency of interrupt signal generation, improving the working efficiency of the master control terminal. In other words, it simultaneously utilizes the advantages of batch reading in FIFO mode and the timely notification advantage of interrupt mode, enabling the master control terminal to read data frames from the sensor in a timely and efficient manner.

[0053] Step S200: Based on the sequence number of the interrupted frame and the interrupted timestamp, calibrate the timestamps of the data frames in the data frame set.

[0054] Specifically, after receiving an interrupt signal, the master control terminal can directly use the current time as the interrupt timestamp, since the delay between the time the master control terminal receives the interrupt signal and the time the sensor generates the interrupt signal is negligible. It then retrieves the number of data frames stored in the sensor's register, which is the sequence number of the interrupt frame (i.e., the frame corresponding to the moment the interrupt occurs). Since a preset interrupt threshold is set for the sensor, the number of data frames in the sensor's register gradually increases from the beginning. When the preset threshold is reached, an interrupt signal is immediately generated, and the number of data frames in the sensor's register at this point is the preset threshold. Then, the same number of data frames as the preset threshold are read from the sensor to form a data frame set. Based on the relationship between the sequence number of each data frame in the data frame set and the sequence number of the interrupt frame, the time difference between the data frame and the interrupt frame is calculated. Using the interrupt timestamp and the time difference, the accurate timestamp of each data frame can be obtained and calibrated.

[0055] In this embodiment, the timestamps of the data frames in the calibration data frame set, such as Figure 3 As shown, the specific steps include the following:

[0056] Step S210: Obtain the sequence number difference between the sequence number of each data frame in the data frame set and the sequence number of the interrupt frame;

[0057] Specifically, since each data frame in the sensor's register is stored in chronological order, when the sensor's data frames are read and saved to the data frame set using FIFO mode, the order of the data frames in the data frame set is the same as the order stored in the sensor's register. The sequence number of each data frame in the data frame set is its sorting number, and the interrupt frame is the last frame in the data frame set. Therefore, the sequence number difference between each data frame's sequence number and the interrupt frame's sequence number can be obtained. For example, if the data frame set sequentially stores 5 data frames S1 to S5, then the data frame sequence numbers are 1 to 5, the interrupt frame's sequence number is 5, and the sequence number difference between data frame S2 and the interrupt frame is 3.

[0058] Step S220: Based on the sensor's frame rate and sequence number difference, obtain the time difference between each data frame and the interrupt frame in the data frame set;

[0059] Specifically, the sensor's frame rate indicates how many data frames the sensor generates per second. Based on the sequence number difference and the sensor's frame rate, the time difference between each data frame in the data frame set can be calculated. For example, in the example above, the sequence number difference between data frame S2 and the interrupt frame is 3. Assuming the sensor's frame rate is 10 frames / second, the time difference between data frame S2 and the interrupt frame can be calculated to be 0.3 seconds.

[0060] Step S230: Based on the interruption timestamp and the time difference between each data frame and the interruption frame, set the timestamp of each data frame in the data frame set to calibrate the timestamp of the data frames in the data frame set.

[0061] Specifically, the interrupt timestamp and the time difference corresponding to each data frame are obtained. The interrupt timestamp and the time difference are added together to obtain the corrected timestamp, and the timestamp of each data frame in the data frame set is calibrated. Assuming the interrupt timestamp is t, and the time differences of the above data frames S1 to S5 are 0.4, 0.3, 0.2, 0.1, and 0, then the timestamps of each data frame in the data frame set are t-0.4, t-0.3, t-0.2, t-0.1, and t, respectively.

[0062] The timestamp of each data frame in the data frame set can be calibrated simply and quickly by using the frame sequence calibration method described above.

[0063] In summary, by combining interrupt mode and FIFO mode, the timestamp of each data frame in the data frame set can be calibrated based on the interrupt timestamp generated during the interrupt, preventing the accumulation and amplification of time deviations and accurately calibrating the timestamp of each sensor data frame. Although some sensors timestamp the acquired data using their own clocks for use by downstream algorithms, it is still necessary to calibrate the timestamps of sensor data to ensure consistency across different sensor data. Taking IMU sensors and cameras in an autonomous driving system as an example, it is necessary to fuse IMU sensor data frames with camera images to estimate the vehicle's attitude / motion / pose. Although both the camera images and IMU sensor data frames have timestamps, with the camera image timestamp typically being the camera exposure time and the IMU sensor data frame timestamp being the IMU sensor data frame generation time, issues such as trigger delay, transmission delay, and lack of accurate clock synchronization can cause misalignment between the timestamps of camera images and IMU sensor data frames. By aligning the timestamps of the camera images with the accurate timestamps of the IMU sensor data frames, effective time synchronization can be maintained when fusing the IMU sensor data frames and camera image data.

[0064] The master control terminal reads sensor data in batches according to a preset threshold. An interrupt signal is generated when the total number of data frames in the sensor's register reaches the preset threshold. Normally, although the sensor continues to generate data frames during the reading process, after the interrupt signal is generated, the master control terminal promptly reads the preset threshold number of data frames from the sensor register. At this point, the total number of data frames in the sensor register is less than the preset threshold, ensuring the sensor can generate the next interrupt normally. However, in some cases, the master control terminal may be handling other tasks, leading to untimely data reading. This could result in the master control terminal reading the preset threshold number of data frames from the sensor register, only to find that the number of data frames in the sensor has exceeded the preset threshold. Furthermore, although the sensor generates another interrupt signal, it is lost because the master control terminal is still processing the previous interrupt signal. Subsequently, because the total number of data frames in the sensor register continues to exceed the preset threshold, the sensor stops generating interrupt signals, resulting in frame loss.

[0065] Therefore, in one embodiment, such as Figure 4 As shown, after the timestamps of the data frames in the calibration data frame set, it also includes:

[0066] Step S300: Obtain the number of data frames in the sensor;

[0067] Step S400: When the above quantity is greater than or equal to a preset threshold, read the preset threshold number of data frames, save the data frames to the data frame set and calibrate the timestamps of the data frames in the data frame set, and return to obtain the number of data frames in the sensor for cyclic reading until the above quantity is less than the preset threshold.

[0068] Specifically, after reading a set of data frames from the sensor's register (once a data frame is read from the sensor register, it is no longer stored in the register), the number of data frames in the sensor's register is checked. If the number of data frames is still greater than or equal to a preset threshold, it indicates that an interrupt signal loss has occurred. In this case, it is necessary to continue reading a preset threshold number of data frames, save the read data frames to the data frame set, and calibrate the timestamps of the data frames in the data frame set. After calibration, the number of data frames in the sensor's register is checked again. If it is still greater than or equal to the preset threshold, reading continues until the number of data frames in the sensor's register is less than the preset threshold. When calibrating the timestamps of the data frames in the data frame set read using the above method, the timestamp calibration can still be performed using the methods in steps S210 to S230. For example: Assuming that after receiving an interrupt signal, the sequence number of the interrupt frame is 5, the interrupt timestamp is t, and the sensor's frame rate is 10 frames / second, the sequence numbers of the data frames in the read data frame set will sequentially increase from 6 to 10. Then, the timestamps of each data frame in this data frame set will be t+0.1, t+0.2, t+0.3, t+0.4, and t+0.5, respectively. When reading data frames again, the sequence number of the data frames will start from 11, and so on. This continues until the number of data frames in the sensor's register is less than a preset threshold, at which point the starting sequence number of the data frames will be reset to 1.

[0069] By employing the aforementioned intelligent reading method, it can be ensured that sensor data frames are not lost. When the total number of data frames in the sensor register exceeds a preset threshold, the sensor will not generate an interrupt signal, causing the mechanism of triggering data frame reading based on the interrupt signal to fail. To prevent this situation, in some embodiments, the number of data frames in the sensor is queried at intervals according to a preset waiting time. When the number is greater than or equal to a preset threshold, the preset threshold number of sensor data frames is read, and the time when the main control terminal starts reading is set as the interrupt timestamp, the sequence number of the interrupt frame is set to 1, and the timestamp is calibrated using the methods in steps S210 to S230.

[0070] The preset waiting time can be determined based on the sensor's frame rate. For example, assuming the preset threshold is 10 frames and the sensor's frame rate is 10 frames per second, the preset waiting time can be set to 1.5 seconds. This will not affect the interrupt signal triggering mechanism and will ensure that the interrupt signal is not stopped being generated.

[0071] The interval query method described above can run as a separate thread on the main control terminal. By employing this method, stable and reliable acquisition of sensor data frames can be ensured without frame loss.

[0072] Exemplary device

[0073] like Figure 5 As shown, corresponding to the above-described sensor timestamp calibration method, this embodiment of the invention also provides a sensor timestamp calibration apparatus, the apparatus comprising:

[0074] The data reading module 600 is used to read the data frames of the sensor in a first-in-first-out mode to obtain a data frame set;

[0075] The calibration module 610 is used to generate an interrupt timestamp and obtain the sequence number of the interrupt frame when an interrupt signal from the sensor is received; and to calibrate the timestamp of the data frames in the data frame set based on the sequence number and the interrupt timestamp of the interrupt frame.

[0076] Optionally, the data reading module includes a reading unit, which is used to obtain the number of data frames in the sensor's register; when the number is greater than or equal to a preset threshold, read the preset threshold number of data frames, save the data frames to the data frame set and calibrate the timestamps of the data frames in the data frame set, and return to obtain the number of data frames in the sensor for cyclic reading until the number is less than the preset threshold.

[0077] Optionally, the calibration module includes a frame sequence calibration unit, which is used to obtain the sequence number difference between the sequence number of each data frame in the data frame set and the sequence number of the interrupted frame; obtain the time difference between each data frame in the data frame set and the interrupted frame based on the sensor's frame rate and the sequence number difference; and set the timestamp of each data frame in the data frame set according to the interrupted timestamp and the time difference between each data frame and the interrupted frame, so as to calibrate the timestamp of the data frames in the data frame set.

[0078] Specifically, in this embodiment, the specific functions of each module of the sensor timestamp calibration system can be referred to the corresponding description in the sensor timestamp calibration method, and will not be repeated here.

[0079] Based on the above embodiments, the present invention also provides a smart terminal, the principle block diagram of which can be as follows: Figure 6As shown. The aforementioned smart terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and sensor timestamp calibration program. The internal memory provides an environment for the operation of the operating system and sensor timestamp calibration program stored in the non-volatile storage medium. The network interface of the smart terminal is used for communication with external terminals via a network connection. When the sensor timestamp calibration program is executed by the processor, it implements the steps of any of the aforementioned sensor timestamp calibration methods. The display screen of the smart terminal can be a liquid crystal display (LCD) or an e-ink display.

[0080] Those skilled in the art will understand that Figure 6 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the smart terminal to which the present invention is applied. A specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0081] In one embodiment, a smart terminal is provided, the smart terminal including a memory, a processor, and a sensor timestamp calibration program stored in the memory and executable on the processor, wherein the sensor timestamp calibration program performs the following operation instructions when executed by the processor:

[0082] When an interrupt signal from the sensor is received, an interrupt timestamp is generated and the sequence number of the interrupt frame is obtained; the sensor's data frames are read using a first-in-first-out (FIFO) mode to obtain a data frame set; based on the sequence number and interrupt timestamp of the interrupt frame, the timestamps of the data frames in the data frame set are calibrated.

[0083] Optionally, after obtaining the data frame set, the following may also be included:

[0084] Obtain the number of data frames in the sensor;

[0085] When the number is greater than or equal to a preset threshold, read the preset threshold number of data frames, save the data frames to the data frame set and calibrate the timestamps of the data frames in the data frame set, and return to obtain the number of data frames in the sensor for cyclic reading until the number is less than the preset threshold.

[0086] Optionally, calibrating the timestamps of data frames in the data frame set based on the sequence number and timestamp of the interrupted frame includes:

[0087] Obtain the sequence number difference between the sequence number of each data frame in the data frame set and the sequence number of the interrupted frame;

[0088] Based on the sensor's frame rate and the sequence number difference, the time difference between each data frame in the data frame set and the interrupt frame is obtained;

[0089] Based on the interruption timestamp and the time difference between each data frame and the interruption frame, the timestamp of each data frame in the data frame set is set to calibrate the timestamp of the data frames in the data frame set.

[0090] Optionally, it also includes: querying the number of data frames in the sensor at a preset waiting time interval; when the number is greater than or equal to the preset threshold, reading the preset threshold number of data frames in the sensor and calibrating the timestamp of the data frames.

[0091] Optionally, the preset waiting time can be determined based on the sensor's frame rate.

[0092] This invention also provides a computer-readable storage medium storing a sensor timestamp calibration program. When the sensor timestamp calibration program is executed by a processor, it implements the steps of any of the sensor timestamp calibration methods provided in this invention.

[0093] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0095] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of the above modules or units is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0098] If the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0099] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not mean that the essence of the corresponding technical solutions deviates from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A sensor timestamp calibration method, characterized in that, The method includes: When an interrupt signal from the sensor is received, an interrupt timestamp is generated and the sequence number of the interrupt frame is obtained. An interrupt signal is generated when the number of data frames in the corresponding register of the sensor reaches a preset threshold. The data frames from the sensor are read using a first-in-first-out (FIFO) mode to obtain a data frame set; Based on the sequence number and interrupt timestamp of the interrupted frame, the timestamps of the data frames in the data frame set are calibrated. This calibration includes: calculating the time difference between the data frames and the interrupted frames based on the sequential relationship between the sequence number of each data frame and the sequence number of the interrupted frame; and calibrating the timestamps of the data frames in the data frame set based on the interrupted timestamp and the time difference. After calibrating the timestamps of the data frames in the data frame set, the process further includes: Obtain the number of data frames in the sensor; When the number is greater than or equal to a preset threshold, read the preset threshold number of data frames, save the data frames to the data frame set and calibrate the timestamps of the data frames in the data frame set, and return to obtain the number of data frames in the sensor for cyclic reading until the number is less than the preset threshold.

2. The sensor timestamp calibration method as described in claim 1, characterized in that, The calibration of the timestamps of data frames in the data frame set based on the sequence number and timestamp of the interrupted frame includes: Obtain the sequence number difference between the sequence number of each data frame in the data frame set and the sequence number of the interrupted frame; Based on the sensor's frame rate and the sequence number difference, the time difference between each data frame in the data frame set and the interrupt frame is obtained; Based on the interruption timestamp and the time difference between each data frame and the interruption frame, the timestamp of each data frame in the data frame set is set to calibrate the timestamp of the data frames in the data frame set.

3. The sensor timestamp calibration method as described in claim 1, characterized in that, Also includes: Query the number of data frames in the sensor at a preset waiting time interval; When the number is greater than or equal to a preset threshold, read the preset threshold number of data frames from the sensor and calibrate the timestamp of the data frames.

4. The sensor timestamp calibration method as described in claim 3, characterized in that, The preset waiting time is determined based on the sensor's frame rate.

5. A sensor timestamp calibration device, characterized in that, The device includes: The data reading module is used to read data frames from the sensor in a first-in-first-out mode to obtain a data frame set. An interrupt signal is generated when the number of data frames in the corresponding register of the sensor reaches a preset threshold. The calibration module is used to generate an interrupt timestamp and obtain the sequence number of the interrupt frame when an interrupt signal from the sensor is received. Based on the sequence number and interrupt timestamp of the interrupted frame, the timestamps of the data frames in the data frame set are calibrated. This calibration includes: calculating the time difference between the data frames and the interrupted frames based on the sequential relationship between the sequence number of each data frame and the sequence number of the interrupted frame; and calibrating the timestamps of the data frames in the data frame set based on the interrupted timestamp and the time difference. The data reading module includes a reading unit, which is used to obtain the number of data frames in the sensor; when the number is greater than or equal to the preset threshold, the module reads the preset threshold number of data frames, saves the data frames to the data frame set and calibrates the timestamps of the data frames in the data frame set, and returns the number of data frames in the sensor to perform cyclic reading until the number is less than the preset threshold.

6. The sensor timestamp calibration device as described in claim 5, characterized in that, The calibration module includes a frame sequence calibration unit, which is used to obtain the sequence number difference between the sequence number of each data frame in the data frame set and the sequence number of the interrupted frame. Based on the sensor's frame rate and the sequence number difference, the time difference between each data frame in the data frame set and the interrupt frame is obtained; Based on the interruption timestamp and the time difference between each data frame and the interruption frame, the timestamp of each data frame in the data frame set is set to calibrate the timestamp of the data frames in the data frame set.

7. A smart terminal, characterized in that, The smart terminal includes a memory, a processor, and a sensor timestamp calibration program stored in the memory and executable on the processor. When the sensor timestamp calibration program is executed by the processor, it implements the steps of the sensor timestamp calibration method as described in any one of claims 1-4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a sensor timestamp calibration program, which, when executed by a processor, implements the steps of the sensor timestamp calibration method as described in any one of claims 1-4.

Images