Frame synchronization method and system for multiple devices, electronic device and readable storage medium
By synchronizing the system clock and detecting signals, frame synchronization of a multi-device camera system is achieved, solving the problems of high hardware cost and poor flexibility in existing technologies, and improving data quality and model generalization ability.
Patent Information
- Application Number
- CN202611085923.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-21
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2046-07-21
AI Technical Summary
In multi-device camera systems, inconsistent frame synchronization leads to data misalignment, affecting the accuracy of subsequent fusion and analysis. Existing technologies require additional dedicated synchronization circuits, increasing hardware costs and wiring complexity, and limiting device flexibility and scalability.
By synchronizing the system clock, electronic devices generate detection signals at preset times. Frame synchronization correction is achieved by utilizing the existing system clock and second pulse detection mechanism without the need for additional hardware circuit design. Preset correspondences can be flexibly configured to adapt to the frame rate requirements of different devices.
It achieves frame synchronization of multiple camera units, reduces hardware costs and wiring complexity, improves flexibility and scalability between devices, ensures data quality and consistency, and enhances the accuracy of training data and the generalization ability of the model.
Smart Images

Figure CN122601805B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of computer technology, and specifically relates to a multi-device frame synchronization method, system, electronic device, and readable storage medium. Background Technology
[0002] In data acquisition scenarios, multi-device acquisition systems typically employ separate head-mounted and hand-mounted devices. The corresponding frames from multiple camera units connected to these devices must be exposed simultaneously to ensure consistent recording time for the same event. For example, if the corresponding frame from the head-mounted device's camera lags behind the corresponding frame from the hand-mounted device's camera when the hand-mounted device touches an object, the head-mounted device's camera will still record an image of the object not in contact. This asynchrony between the camera feeds from multiple devices leads to temporal misalignment of the acquired data, affecting the accuracy of subsequent fusion and analysis. Summary of the Invention
[0003] To overcome the problems existing in related technologies, this disclosure provides a multi-device frame synchronization method, system, electronic device, and readable storage medium.
[0004] In a first aspect, embodiments of this application provide a multi-device frame synchronization method, applied to a first electronic device, the first electronic device being used to collect multiple frames of data; the method includes: A detection signal is generated at a preset time, wherein the second electronic device also generates a detection signal at the preset time, the system clocks of the first electronic device and the second electronic device are synchronized, and the second electronic device is used to collect multiple frames of data; In response to the detection signal generated by the first electronic device, frame synchronization correction is performed on the first electronic device based on the trigger time corresponding to the detection signal and the trigger time of a specified frame. The specified frame is a frame in the frame sequence corresponding to the first electronic device, so that the trigger time of the frame corresponding to the first electronic device after frame synchronization correction satisfies a preset correspondence with the trigger time of the frame corresponding to the second electronic device.
[0005] Secondly, this application provides a frame synchronization system composed of multiple devices, including a first electronic device and a second electronic device. The first electronic device and the second electronic device are wearable devices. The first electronic device and the second electronic device are both wired to one or more camera units. The system clocks of the first electronic device and the second electronic device are synchronized. The trigger time of the frame corresponding to the first electronic device and the trigger time of the frame corresponding to the second electronic device have a preset correspondence. The first electronic device is configured to perform: A detection signal is generated at a preset time; In response to the detection signal, frame synchronization correction is performed based on the trigger time corresponding to the detection signal and the trigger time of a specified frame, wherein the specified frame is a frame in the frame sequence corresponding to the first electronic device; The second electronic device is configured to perform: A detection signal is generated at the preset time; In response to the detection signal, frame synchronization correction is performed based on the trigger time corresponding to the detection signal and the trigger time of a specified frame, wherein the specified frame is a frame in the frame sequence corresponding to the second electronic device.
[0006] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.
[0007] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.
[0008] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.
[0009] In this embodiment, the system clocks of the first electronic device and the second electronic device are synchronized so that each device generates a detection signal at a preset time. In response to the detection signal, frame synchronization determination and correction are performed based on the trigger time corresponding to the detection signal and the trigger time of the specified frame. This enables the corresponding frames of multiple camera units connected to multiple devices to be exposed at the same time, ensuring that the data collected by multiple devices are aligned in time and improving data quality. Attached Figure Description
[0010] Figure 1 This is a flowchart illustrating the steps of a multi-device frame synchronization method provided in this embodiment of the disclosure; Figure 2 This is a schematic diagram of the connection between a first electronic device and a camera unit provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of a frame trigger timestamp sequence provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram illustrating the specific steps of a multi-device frame synchronization method provided in this embodiment of the disclosure; Figure 5 This is a timing diagram illustrating a time synchronization method between a transmitting device and a receiving device provided in an embodiment of this disclosure; Figure 6 This is a flowchart illustrating the specific steps of a time synchronization method between a transmitting device and a receiving device provided in an embodiment of this disclosure; Figure 7 This is a structural diagram of an electronic device provided in an embodiment of this disclosure; Figure 8 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0012] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0013] In data acquisition scenarios, multi-device acquisition systems typically employ separate head-mounted and hand-mounted devices. The corresponding frames from multiple camera units connected to these devices must be exposed simultaneously to ensure consistent recording time for the same event. For example, if the corresponding frame from the head-mounted device's camera lags behind the corresponding frame from the hand-mounted device's camera when the hand-mounted device touches an object, the head-mounted device's camera will still record an image of the object not in contact. This asynchrony between the camera feeds from multiple devices leads to temporal misalignment of the acquired data, affecting the accuracy of subsequent fusion and analysis.
[0014] For example, in related technologies, frame synchronization between devices typically relies on time synchronization between multiple devices, and the frame synchronization hardwires on each device are synchronized with the 1PPS (pulses per second) signal through special hardware circuitry to ensure that the frame triggering times of each device are aligned. However, this approach requires the design of dedicated synchronization circuits on each device to hardware-level bind the frame synchronization hardwires with the 1PPS signal. This not only increases the hardware cost and wiring complexity of the devices, but also imposes uniform circuit specification requirements on the hardware design of each device, limiting the flexibility and scalability between different devices.
[0015] The multi-device frame synchronization method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.
[0016] Figure 1 This is a flowchart of the steps of a multi-device frame synchronization method provided in this embodiment, applied to a first electronic device.
[0017] like Figure 1 As shown, the method may include: Step 101: Generate a detection signal at a preset time, wherein the second electronic device also generates a detection signal at the preset time, and the system clocks of the first electronic device and the second electronic device are synchronized.
[0018] The first and second electronic devices can be used to acquire multiple frames of data, which can be point cloud data, image data, etc.
[0019] In this embodiment, each electronic device generates a detection signal when a preset time is reached. The detection signal can also be used to trigger an interrupt. The system clocks of the first and second electronic devices are synchronized through a time synchronization mechanism. The generation of the detection signal can be based on an external wired pulse signal or an internal operating system interrupt event. The detection signal provides a time reference point for alignment of each device; all devices generate the detection signal at the same preset time as a benchmark for determining whether the frame synchronization phase has drifted.
[0020] The first electronic device and the second electronic device refer to any two electronic devices participating in frame synchronization. The first electronic device and the second electronic device may each include a system-on-a-chip (SoC), a hardware clock, at least one camera unit, and a hardware interface for frame synchronization, etc. Multiple devices can form a device group; further, one device is designated as the master device, and the others are slave devices. For example, the first electronic device can be the master device, and the second electronic device can be a slave device.
[0021] In some embodiments, the preset time can refer to the moment when the system clock of each device reaches a fixed time point, such as the start time of each natural second (i.e., a whole second). Further, the detection signal can be set to be generated periodically, with the period being a fixed duration. That is, each electronic device generates a detection signal every fixed duration, and after reaching a preset time determined based on the fixed duration, each electronic device generates a detection signal. For example, it can be set to generate a detection signal every 1 second, so that each electronic device can generate a detection signal at whole seconds.
[0022] Of the two methods for generating the detection signal, if a hard-wired pulse output is used, the hardware clocks of the first and second electronic devices can output a level-changing pulse at a preset time, such as every whole second. This pulse is captured through the I / O port and triggers an interrupt. If an internal operating system interrupt event is used, the operating system actively generates a software interrupt at a preset time, such as every whole second, without requiring additional hardware support. The second electronic device can generate the detection signal in the same way as the first electronic device. Since the hardware clocks of the first and second electronic devices are synchronized, the times when the two devices generate the detection signal are aligned on the global time base. It can be understood that the detection signal not only provides a common time alignment reference point for all devices, but also serves as an interrupt trigger event. The detection signal can drive the active execution of frame synchronization phase determination at a preset time, such as every whole second, thereby achieving periodic closed-loop monitoring of the frame synchronization status.
[0023] Step 102: In response to the detection signal generated by the first electronic device, perform frame synchronization correction on the first electronic device based on the trigger time corresponding to the detection signal and the trigger time of the specified frame, wherein the specified frame is a frame in the frame sequence corresponding to the first electronic device, so that the trigger time of the frame corresponding to the first electronic device after frame synchronization correction and the trigger time of the frame corresponding to the second electronic device satisfy a preset correspondence.
[0024] In this embodiment of the present disclosure, when the detection signal triggers an interrupt, the first electronic device, within the interrupt context, obtains the trigger time corresponding to the detection signal and the frame synchronization event timestamp of the specified frame, using it as the trigger time of the specified frame. Based on the trigger time corresponding to the detection signal and the trigger time of the specified frame, it determines whether frame synchronization correction needs to be performed on the first electronic device. If correction is required, frame synchronization correction is performed on the first electronic device; otherwise, no correction is performed.
[0025] The trigger time of the frame corresponding to the first electronic device and the trigger time of the frame corresponding to the second electronic device have a preset correspondence. This preset correspondence can be a mapping rule followed between the trigger times of the frames of the two devices when the first and second electronic devices send frame trigger signals at their respective preset frequencies. Specifically, the trigger times of the first frame of the first electronic device and the first frame of the second electronic device are aligned, meaning the time anchor frames of the two devices are at the same time; for example, the first frame within each second pulse period can be triggered at the same time. For other frames after the first frame, each device can operate independently according to its own preset frame rate. The time anchor frame refers to the frame designated as the time alignment reference between the two devices in the preset correspondence. It can be the first frame in the frame sequence of each device (i.e., the frame with sequence number 1), or a frame with other preset sequence numbers.
[0026] In some embodiments, the preset correspondence is related to the preset frame rates of the first electronic device and the second electronic device. Provided that the trigger times of the first frame of the first electronic device and the first frame of the second electronic device are aligned, when the preset frame rates of the two devices are the same, the preset correspondence is that frames with the same sequence position in the frame sequences of the two devices are corresponding frames, and correspondingly, frames with the same sequence position in the frame sequences of the first electronic device and the second electronic device have the same trigger time; when the preset frame rates of the two devices are different, the preset correspondence is a frame index mapping rule based on the ratio of the preset frame rates of each device.
[0027] The specified frame can be the calculation anchor point for frame synchronization correction. The specified frame can be any frame before or after the trigger time of the detection signal. In some embodiments, the specified frame can be any frame within the current second pulse period (e.g., the frame corresponding to the most recent frame trigger signal before the trigger time of the detection signal).
[0028] Specifically, the determination of whether frame synchronization correction is required can include two methods: (1) obtaining the trigger time of the specified frame recorded by the most recent frame synchronization hard-wired trigger, and calculating the phase deviation between the trigger time of the specified frame and the trigger time corresponding to the detection signal; or, calculating the standard trigger time of the specified frame based on the preset frame rate and frame synchronization period of the first electronic device, and extending a standard trigger time range based on the standard trigger time as an allowable deviation window. Further, if the phase deviation exceeds the preset phase difference threshold, or the trigger time of the specified frame falls outside the standard time trigger range, it is determined that frame synchronization correction is required; otherwise, it is determined that correction is not required.
[0029] Accordingly, if the first electronic device and the second electronic device operate at the same preset frame rate, the first electronic device can obtain the trigger time of the kth frame of the specified frame. Then, the determination is made using any of the above determination methods, and the second electronic device also executes the same determination logic. Since the preset frame rates of the two devices are the same and the system clocks are synchronized, the determination results of the two devices should theoretically be consistent. In some embodiments, if the first electronic device and the second electronic device operate at different preset frame rates, the first electronic device can obtain the trigger time of any frame as the trigger time of the specified frame. Correspondingly, the second electronic device can also obtain the trigger time of any frame as the trigger time of the specified frame. In other embodiments, if the first electronic device and the second electronic device operate at different preset frame rates, the first electronic device can obtain the trigger time of the kth frame as the trigger time of the specified frame. Correspondingly, the second electronic device can select the mth frame corresponding to the kth frame as the specified frame according to the preset correspondence and obtain its trigger time. Then, each of them uses any of the above determination methods to make a determination. For example, (1) based on the phase deviation between the trigger time of the specified frame and the trigger time corresponding to the detection signal, it is compared with a preset phase difference threshold to determine whether correction is required. (2) Based on the preset frame rate and frame synchronization period of each device, independently calculate the standard trigger time of the specified frame for each device. Then compare the standard trigger time range obtained by expansion with the actual trigger time of the specified frame to determine whether correction is required.
[0030] In summary, in this embodiment of the present disclosure, by synchronizing the system clocks of the first electronic device and the second electronic device, the two devices generate detection signals at preset times and respond to the detection signals. Based on the trigger time corresponding to the detection signal and the trigger time of the specified frame, deviation judgment and correction are performed to enable the corresponding frames of multiple camera units connected to multiple devices to be exposed at the same time, ensuring that the data collected by multiple devices are aligned in time and improving data quality.
[0031] In some embodiments, the first electronic device includes a head-mounted data acquisition device equipped with sensors (e.g., cameras) for acquiring multi-frame data, and the second electronic device includes a hand-mounted data acquisition device (e.g., a gripper data acquisition device) equipped with sensors (e.g., cameras) for acquiring multi-frame data. A user wears both the head-mounted and hand-mounted data acquisition devices to perform a preset teaching task, such as folding clothes, pouring water, tidying up items, cooking, or other daily operations. The devices continuously acquire multimodal information, including motion data, image data, and tactile data, at a preset frame rate. This data is used to train an embodied large model, enabling a robot equipped with the trained large model to also perform specific tasks, such as folding clothes, pouring water, tidying up items, and cooking. Through the method provided in this disclosure, the first and second electronic devices can achieve more precise time synchronization, resulting in more accurate alignment of multi-frame data acquired by different devices, significantly improving the quality and consistency of the training data. The embodied large model trained with this high-quality data can more accurately establish the temporal mapping relationship between "visual perception—action execution," thereby improving the model's generalization ability and execution success rate in complex tasks.
[0032] This solution eliminates the need for dedicated hardware circuits on each device. Instead, it utilizes the existing system clock and second pulse detection mechanism. Each device simultaneously generates a detection signal, triggering the calculation and correction of frame synchronization phase deviation synchronously. This eliminates the need for additional synchronization circuit design, reducing hardware costs and wiring complexity. Furthermore, since the frame synchronization correction logic and preset correspondences are independent of specific hardware circuit specifications, different devices can flexibly configure preset correspondences according to their respective frame rate requirements. This supports mixed deployments with the same or different preset frame rates, free from the constraints of a unified circuit design, thus improving flexibility and scalability among different devices.
[0033] Optionally, embodiments of this disclosure may further include the following steps: Step 201: Send a frame trigger signal to one or more camera units connected to the first electronic device via a wired connection at a preset frame rate to synchronously trigger the one or more camera units; wherein the first electronic device and the second electronic device send the frame trigger signal at different preset frequencies; or the first electronic device and the second electronic device send the frame trigger signal at the same preset frequency.
[0034] In this embodiment of the disclosure, a first electronic device sends a frame trigger signal to one or more camera units connected to it via a wired connection to synchronously trigger each camera unit to perform exposure or image output. Correspondingly, the first electronic device can also send a frame trigger signal to one or more camera units connected to it via a wired connection to synchronously trigger each camera unit to perform exposure or image output. The first or second electronic device can connect one or more camera units based on one or more frame synchronization hard lines. The frame synchronization hard line can be a frame sync hard line used to transmit frame trigger signals between the device and the camera units. The frame synchronization hard line can connect a target pin to the camera unit, and the target pin can be the SoC sampling I / O of the first electronic device. The frame synchronization hard line can simultaneously trigger multiple camera units on one device to perform exposure or image output. Multiple devices and multiple camera units can also synchronously trigger exposure or synchronously trigger image output. For example, as shown... Figure 2 As shown, the first electronic device can be connected to multiple camera units (cam) via one or more frame synchronization hard lines. The first electronic device can trigger the detection signal through the Gigabit Media Access Controller (GMAC) interface, and the external frame trigger signal (Frame Sync) is connected to the SoC via hard line. The SoC performs back sampling detection, and after detecting a valid edge, it triggers an internal interrupt (INT) to notify the processor that a new frame has arrived and records the timestamp of the frame trigger signal.
[0035] For example, the SoC of the first electronic device outputs a periodic trigger signal on a frame synchronization hardline at a preset frame rate. This hardline is connected to the trigger input port of one or more camera units. When a camera unit receives a trigger edge, it synchronously performs exposure or image output operations at the same time. All camera units connected to the same device receive the same frame trigger signal, realizing multi-camera synchronization within the device.
[0036] The second electronic device can send frame trigger signals to its connected camera unit at the same or different preset frame rates. For example, assuming the first and second electronic devices send frame trigger signals at different preset frame rates, with the first device's preset frame rate being 30Hz and the second electronic device's preset frame rate being 6Hz, the first frame of the second electronic device corresponds to the first frame of the first electronic device, the second frame of the second electronic device corresponds to the fifth frame of the first electronic device, and so on. If the first and second electronic devices send frame trigger signals at the same preset frame rate of 30Hz, the first frame of the second electronic device corresponds to the first frame of the first electronic device, the second frame of the second electronic device corresponds to the second frame of the first electronic device, and so on.
[0037] In this embodiment, frame synchronization across multiple devices is decoupled into two independent layers: frame synchronization within each device and frame synchronization between devices. Wired connections ensure deterministic trigger latency and low jitter characteristics while guaranteeing frame synchronization across multiple camera units within the device itself. Furthermore, by having each device generate a detection signal at the same preset time and execute frame synchronization correction logic, frame synchronization between devices is ensured. These two mechanisms work together to guarantee both local frame synchronization within each device and global frame synchronization across multiple devices.
[0038] Optionally, embodiments of this disclosure may further include the following steps: Step 301: Obtain the frame trigger timestamp corresponding to the frame trigger signal.
[0039] In this embodiment, the frame trigger timestamp corresponding to the frame trigger signal is obtained. The rising edge time of the frame trigger signal can be obtained based on the target pin of the first electronic device and used as the frame trigger timestamp. The target pin can be a SoC sampling I / O of the first electronic device. The target pin of the first electronic device monitors the level changes on the frame synchronization hardline. When a frame trigger signal (e.g., a rising edge) is detected, the target pin captures the signal edge, triggers an internal interrupt (INT) to notify the processor that a new frame has arrived, and reads the current system clock count value, outputting this count value as the frame trigger timestamp.
[0040] Step 302: Store the frame trigger timestamp in the target data structure to obtain the frame trigger timestamp sequence.
[0041] In this embodiment, the first electronic device writes the frame trigger timestamp into a target data structure. The target data structure can be a circular buffer or a linked list; each time a new frame trigger timestamp is acquired, it can be appended to the end of the data structure in chronological order. The target data structure maintains a sequence of frame trigger timestamps arranged in ascending order of time, containing timestamps of multiple frame trigger events.
[0042] Step 303: Obtain the trigger time of the specified frame from the frame trigger timestamp sequence.
[0043] In this embodiment of the disclosure, in response to a detection signal, the trigger time of a specified frame is read from the target data structure. For example, any frame can be selected as the specified frame from the frame trigger timestamp sequence corresponding to the first electronic device, and the trigger time of the specified frame can be obtained from the frame trigger timestamp sequence according to the frame sequence number of the specified frame.
[0044] In this embodiment of the disclosure, the frame trigger timestamp is obtained by the first electronic device through back sampling, and the timestamp sequence is maintained in the target data structure, without the need for additional synchronization circuits, thus saving hardware costs.
[0045] Optionally, step 303 may include the following steps: Step 3031: Determine the trigger time of the specified frame based on the frame trigger timestamp corresponding to the position with the smallest time difference from the trigger time in the frame trigger timestamp sequence corresponding to the first electronic device.
[0046] In this embodiment of the disclosure, the first electronic device, in its interrupt handler, can perform a forward search in the frame trigger timestamp sequence, using the trigger time of the detection signal as a reference point. The search range is limited to all positions where the frame trigger timestamp is less than the trigger time; that is, the search is only conducted in historical timestamps prior to the trigger time of the detection signal. Within this range, the time difference between each timestamp and the trigger time is calculated, and the frame trigger timestamp corresponding to the position with the smallest time difference is selected and determined as the trigger time of the specified frame.
[0047] For example, such as Figure 3 As shown, assuming both the first and second electronic devices transmit frame trigger signals at a frequency of 30fps, and simultaneously, both devices capture the corresponding frame trigger timestamp for each frame trigger signal via SoC backsampling I / O, for example... Figure 3 The sequence t1 to t29. In response to the detection signal generated at the integer second, the first electronic device and the second electronic device can respectively use the frame trigger timestamp corresponding to the position in the corresponding frame trigger timestamp sequence that is before the trigger time and has the smallest time difference with the trigger time, i.e., t29, as the trigger time of the specified frame. Subsequently, t29 can be compared with the trigger time corresponding to the detection signal to determine whether frame synchronization correction is required for the first electronic device or the second electronic device.
[0048] In this embodiment of the disclosure, the trigger timestamp of the frame with the smallest time difference before the trigger time is selected as the trigger time of the specified frame, that is, the frame closest to the detection signal is selected. This ensures that the phase deviation calculation is based on the latest actual trigger event, avoids the accumulation of deviation distortion caused by selecting too far frames, and improves the real-time performance and judgment accuracy of frame synchronization correction.
[0049] Optionally, embodiments of this disclosure may further include the following steps: Step 401: For any frame trigger timestamp stored in the target data structure, if the difference between the frame trigger timestamp and the current time is greater than a preset duration threshold, delete the frame trigger timestamp from the target data structure.
[0050] In this embodiment, the first electronic device calculates the time difference between each timestamp and the current hardware clock time when writing a new frame trigger timestamp to the target data structure or traversing the existing timestamps in the target data structure at a fixed period. If the difference between a timestamp and the current time exceeds a preset duration threshold, the timestamp is determined to have expired and is deleted from the target data structure or marked as invalid, and will no longer participate in the subsequent acquisition and correction of the specified frame trigger time. The preset duration threshold can be a pre-set time window length, such as 1 second, to ensure that the target data structure always retains at least one second of valid frame trigger timestamps.
[0051] In this embodiment, by maintaining a preset duration threshold for the length of timestamp records and automatically deleting timeout timestamps, it is ensured that the acquired trigger time of a specified frame always comes from valid data within the preset duration threshold, avoiding the problem of inaccurate phase deviation calculation caused by using expired timestamps. Simultaneously, by constraining the preset duration threshold, data storage overhead can be reduced to some extent.
[0052] Optionally, embodiments of this disclosure may further include the following steps: Step 501: Upon receiving the detection signal, trigger an interrupt event in response to the detection signal.
[0053] In this embodiment of the present disclosure, when a detection signal arrives, the hardware or operating system kernel of the first electronic device responds to the detection signal and generates an interrupt event, causing the processor to jump to the interrupt handler for execution.
[0054] For example, when the detection signal is generated based on an external wired pulse signal, the hardware clock of the first electronic device outputs a high-level pulse at a preset time. When the rising edge of the pulse arrives, the first electronic device sends an interrupt request to the interrupt controller, the interrupt controller sends an interrupt signal to the processor, the processor suspends the current task, and jumps to the pre-registered second pulse interrupt handler.
[0055] When the detection signal is generated based on an internal interrupt event of the operating system, the operating system of the first electronic device will send a local timer interrupt to the processor core at a preset time. The processor responds to the interrupt and jumps to the pre-registered second pulse interrupt handler.
[0056] Accordingly, step 102 may include the following steps: Step 502: In the interruption event, based on the trigger time corresponding to the detection signal and the trigger time of the specified frame, perform frame synchronization correction on the first electronic device.
[0057] In this embodiment of the present disclosure, after the interrupt event is responded to by the processor, the first electronic device enters the interrupt handler. In the interrupt event, frame synchronization correction is performed on the first electronic device based on the trigger time corresponding to the detection signal and the trigger time of the specified frame.
[0058] In this embodiment, there is no need to add a dedicated synchronization circuit to each device to achieve frame synchronization. Instead, the existing system clock and interrupt mechanism are used to complete the correction determination in the interrupt, which eliminates the need for additional synchronization circuit design and reduces hardware cost and wiring complexity.
[0059] Optionally, embodiments of this disclosure may further include the following steps: Step 601: Configure signal control parameters; the signal control parameters include at least one of the following: frame period, duty cycle.
[0060] In this embodiment, the first electronic device can configure signal control parameters during the system initialization phase according to its own data acquisition task requirements. These signal control parameters include at least one of frame period and duty cycle, and may also include other timing parameters, such as phase parameters. After configuration, the frame synchronization hardline of the first electronic device generates a frame trigger signal according to the signal control parameters. For example, the signal control parameters can be written into the configuration register inside the first electronic device's SoC, causing the hardware output circuit to generate the frame trigger signal according to these parameters.
[0061] It is understandable that when the first electronic device and the second electronic device need to work at different preset frame rates, they can configure their own signal control parameters respectively without the need for unified hardware circuits, thereby realizing the device's ability to operate based on different frame rates at the parameter level.
[0062] Step 602: Based on the signal control parameters, send a frame trigger signal at a preset frame rate to one or more camera units connected to the first electronic device via a hard wire to synchronously trigger each camera unit.
[0063] In this embodiment of the disclosure, based on the signal control parameters configured in the first electronic device, frame trigger signals are periodically sent to one or more connected camera units at a preset frame rate via one or more frame synchronization hard lines. Upon receiving the frame trigger signal, each camera unit synchronously performs exposure or image output operations, thereby achieving local frame synchronization triggered by multiple camera units at the same time. The preset frequency can be determined based on the frame period in the signal control parameters. Sending frame trigger signals to one or more camera units connected to the first electronic device at a preset frame rate via hard lines means that the transmission frequency of the frame trigger signals is equal to the frequency corresponding to the frame period in the signal control parameters.
[0064] In one possible implementation, the phase parameter in the signal control parameters can be configured to be 0, so that the first electronic device sends a frame trigger signal at a preset frame rate to one or more camera units connected to the first electronic device via a hard wire from the whole second, so as to synchronously trigger each camera unit. For example, assuming the preset frame rate is 30fps, there are 30 frames (t0 to t29) in one second. The last frame t29 is output at 966.57ms, and there are 33.33ms left before the next whole second (1000ms), which is exactly equal to one frame period. Then the next frame t30 will be output at 1000ms (i.e., the next whole second), and will be aligned with the whole second again.
[0065] In this embodiment of the disclosure, by configuring signal control parameters including frame period and duty cycle, the first electronic device can automatically determine the preset frame rate according to the frame period, and trigger each camera unit synchronously at the frame rate through hard wire, thereby improving the flexibility of parameter configuration and adapting to different frame rate requirements.
[0066] Optionally, step 102 may include the following steps: Step 701: In response to the detection signal, if the first electronic device has enabled the output of a frame trigger signal, perform frame synchronization correction on the first electronic device based on the trigger time corresponding to the detection signal and the trigger time of the specified frame.
[0067] In this embodiment of the disclosure, in response to the detection signal, the first electronic device can first detect the enabled state of its own frame synchronization hardline output function. Specifically, the first electronic device can read the enable field in the SoC's internal configuration register and determine whether the enable field is enabled. If yes, it proceeds to the subsequent calibration process; if no, it exits the interrupt handler directly without performing any calibration operation.
[0068] It is understandable that if the enable field is not an enable value, it indicates that the first electronic device is not enabled to output the frame trigger signal. The signal control parameters of the first electronic device can be configured first, and then, based on the signal control parameters, the frame trigger signal can be sent to one or more camera units connected to the first electronic device at a preset frame rate through a hard wire to synchronously trigger each camera unit. That is, when the first electronic device is not enabled to output the frame trigger signal, steps 601-602 can be executed.
[0069] In this embodiment of the disclosure, before performing the determination and correction process, it is first detected whether the first electronic device has enabled the output frame trigger signal, which can avoid performing meaningless phase calculations.
[0070] Optionally, step 102 may include the following steps: Step 801: Determine the second preset phase threshold based on the alignment mode parameter in the signal control parameters and the frame period.
[0071] In this embodiment, the first electronic device, within its 1PPS interrupt handling routine, can read the alignment mode parameter and frame period from the signal control parameters, determine the reference phase threshold by looking up a table or according to a preset calculation rule, and then determine the second preset phase threshold. The alignment mode parameter refers to the parameter in the signal control parameters used to define the timing alignment relationship between the trigger time of the frame synchronization hardline and the 1PPS time. The value of the alignment mode parameter determines the calculation rule for the reference phase threshold. Different alignment mode parameters correspond to different calculation rules. Alignment modes include center alignment mode and edge alignment mode, etc.
[0072] For example, different alignment mode parameters can correspond to different mapping relationships. For instance, in center alignment mode, the reference phase threshold can be directly set to a fixed value of 5ms; in edge alignment mode, the reference phase difference threshold can be set to half of the frame period. Alternatively, different alignment mode parameters can correspond to different coefficients, and the reference phase difference threshold can be determined based on the coefficients and the frame synchronization period. This disclosure does not limit the method for determining the second preset phase threshold.
[0073] When the specified frame is any frame, the second preset phase threshold = reference phase threshold + specified frame sequence offset × frame period. Here, the specified frame sequence offset refers to the difference between the sequence position of the most recent frame and the sequence position of the specified frame. That is, when the specified frame is the most recent frame before the trigger time and has the smallest time difference with the trigger time, the specified frame sequence offset is 0, and the reference phase threshold can be directly determined as the second preset phase threshold.
[0074] For example, assuming a frame rate of 30fps, a frame period of 33.33ms, a center alignment mode, and a reference phase difference threshold of 5ms. When the specified frame is the most recent frame t29 (the 29th frame), the specified frame sequence offset = 0, and the second preset phase threshold = 5ms + 0 × 33.33ms = 5ms. When the specified frame is t1 (the 1st frame), there are 28 frames between t1 and t29. Therefore, the second preset phase threshold = 5ms + 28 × 33.33ms = 5ms + 933.24ms = 938.24ms.
[0075] Step 802: Calculate the absolute value of the difference between the trigger time of the specified frame and the trigger time corresponding to the detection signal, and use it as the target difference.
[0076] In this embodiment of the disclosure, the absolute value of the difference between the trigger time of the specified frame and the trigger time corresponding to the detection signal is calculated as the target difference.
[0077] Step 803: If the target difference is greater than the second preset phase threshold, adjust the phase parameter in the signal control parameters based on the target difference.
[0078] In this embodiment, the target difference is compared with a second preset phase threshold. If the target difference is greater than the second preset phase threshold, the phase deviation is determined to be out of tolerance and correction is required. If the target difference is not greater than the preset phase difference threshold, the phase deviation is determined to be within tolerance and no adjustment is performed.
[0079] In this embodiment, the trigger time of the detection signal (i.e., the second pulse moment) is used as the absolute time reference. The deviation between the trigger time of a specified frame and this reference is calculated as the target difference, and compared with a second preset phase threshold dynamically determined by the alignment mode and frame period. Utilizing the hardware-level time determinism of the second pulse, it is possible to accurately determine whether the phase is out of tolerance without additional synchronization circuitry. Furthermore, the second preset phase threshold is adaptively adjusted according to the alignment mode and frame period, supporting flexible deployment for devices with different frame rates and improving the accuracy and versatility of the correction determination.
[0080] Alternatively, step 102 may include the following steps: Step 901: Based on the frame period and the trigger time corresponding to the detection signal, determine the standard trigger time corresponding to the specified frame.
[0081] In this embodiment, a standard trigger time for a specified frame is determined based on the frame period and the trigger time corresponding to the detection signal. A standard trigger time range is formed by expanding outwards from the standard trigger time as the center and using a first preset phase threshold as half-width. This standard trigger time range defines the tolerance interval within which the trigger time of the specified frame is considered within acceptable limits; exceeding this interval indicates a phase error. It is determined whether the trigger time of the specified frame falls within the standard trigger time range. If it does not, frame synchronization correction can be performed on the first electronic device. By setting a dynamic tolerance range centered on the standard trigger time for judgment, consistency in the judgment scale for any specified frame is ensured, improving the accuracy and robustness of the correction judgment.
[0082] For example, the frame synchronization of the first electronic device can be performed by: adjusting the phase parameter in the signal control parameters based on the target difference between the trigger time of the specified frame and the standard trigger time corresponding to the specified frame; or, retransmitting the frame trigger signal.
[0083] In the second pulse interrupt handling routine, the first electronic device uses the trigger time corresponding to the detected signal (i.e., the whole second) as the time reference, and combines it with the frame period in the signal control parameters. Based on the sequence position of the specified frame within one second, it calculates the theoretical time at which the specified frame should be triggered under ideal synchronization, and uses this as the standard trigger time. The standard trigger time refers to the theoretical time at which the specified frame should be triggered under ideal synchronization. This time can be determined by the phase parameter, the frame period, and the sequence position of the specified frame within one second. With the first frame being t1, the calculation method is: Standard trigger time = Theoretical trigger time of the first frame in each second pulse period (determined according to the phase parameter in the signal configuration parameters) + (Specified frame sequence position - 1) × Frame period. The standard trigger time is a theoretical reference value that does not change with actual operational deviations, serving as a benchmark for determining whether the actual trigger time exceeds the tolerance. For example, the phase parameter represents the initial offset of the theoretical trigger time of the first frame in each second pulse period relative to the whole second reference. If the phase parameter in the signal configuration parameters is 5ms, then the initial offset of the theoretical trigger time of the first frame in each second pulse period relative to the integer second reference is 5ms. That is, the theoretical trigger time of the first frame is aligned with the trigger time (integer second) corresponding to the detection signal. The first frame is output at 5ms, and subsequent frames are triggered sequentially at frame period intervals. The theoretical trigger time of the last frame is (total number of frames). 1) × Frame period: The frame triggering time within the entire second pulse period is uniquely determined by the frame period and sequence number, and there is no additional initial phase offset.
[0084] For example, assuming the theoretical trigger time of the first frame t1 in the current second pulse cycle is a whole second + 5ms, such as 5ms, then when the specified frame is t2, the sequence number k=2, and the standard trigger time = 5ms + 1 × 33.33ms = 38.33ms, meaning that t2 should theoretically be triggered at 38.33ms. When the specified frame is t29, the sequence number k=29, and the standard trigger time = 5ms + 29 × 33.33ms = 971.57ms, meaning that t29 should theoretically be triggered at 971.57ms.
[0085] Step 902: Determine the standard trigger time range based on the standard trigger time and the first preset phase threshold.
[0086] In this embodiment of the disclosure, a standard trigger time range is determined based on a standard trigger time and a first preset phase threshold. The first preset phase threshold characterizes the allowable deviation value. The first preset phase threshold can be determined based on the alignment mode parameter and frame period in the signal control parameters. The first preset phase threshold can be the same as or different from a second preset phase threshold. The first preset phase threshold determines the width of the standard trigger time range, that is, the maximum range by which the actual trigger time of a specified frame deviates from the standard trigger time.
[0087] Step 903: In response to the fact that the trigger time of the specified frame is not within the standard trigger time range, determine the target difference between the standard trigger time and the trigger time of the specified frame.
[0088] In this embodiment of the present disclosure, the first electronic device compares the actual trigger time of a specified frame with a standard trigger time range. If the actual trigger time falls within the standard trigger time range, the synchronization state is determined to be normal, and no correction is performed. If the actual trigger time falls outside the standard trigger time range, the phase is determined to be out of tolerance, and the absolute difference between the standard trigger time and the actual trigger time corresponding to the specified frame is calculated as the target difference.
[0089] Step 904: Based on the target difference, adjust the phase parameter in the signal control parameters.
[0090] In this embodiment of the disclosure, the phase parameter in the signal control parameters is adjusted according to the target difference.
[0091] In this embodiment of the disclosure, the standard trigger time is used as the reference, and the determination reference is determined as a fixed theoretical ideal time. It will not fluctuate with the time jitter of the detection signal, making the frame synchronization correction determination more stable. Moreover, the target difference reflects the deviation direction and magnitude of the actual trigger time relative to the theoretical time, providing a clear directional guide for the adjustment of the phase parameters.
[0092] Optionally, the step 803 and step 904, "adjusting the phase parameter in the signal control parameters based on the target difference," may include: Step 1001: If the target trigger time is greater than the trigger time of the specified frame, update the phase parameter in the signal control parameter based on the sum of the original phase parameter and the target difference; the target trigger time is the trigger time corresponding to the detection signal or the standard trigger time.
[0093] In this embodiment of the disclosure, after determining that the target trigger time is greater than the specified frame trigger timestamp, the first electronic device adds the original phase parameter to the target difference to obtain a new phase parameter, and writes it back to the phase parameter field in the signal control parameters to complete the update. The original phase parameter refers to the current value of the phase parameter in the signal control parameters before the correction action is performed. The original phase parameter reflects the initial phase offset of the frame synchronization hard-wire trigger time relative to the reference time.
[0094] A target trigger time greater than the specified frame trigger timestamp indicates that the actual trigger time of the specified frame before correction was earlier than the target trigger time, meaning the frame synchronization hardline trigger was advanced. For example, if the target trigger time is a whole second 1000.000000s, and the actual trigger time of the specified frame is 999.999400s, the actual trigger is 600μs earlier than the target. After updating based on the sum of the differences between the original phase parameters and the target, the phase parameters are adjusted in the direction of increase. Since the trigger time will be delayed compared to before correction when the phase parameters increase, this adjustment direction compensates for the deviation of the actual trigger being advanced before correction, causing the trigger time of the frame synchronization hardline after correction to converge towards the target trigger time.
[0095] Step 1002: If the target trigger time is less than the trigger time of the specified frame, update the phase parameter in the signal control parameters based on the difference between the original phase parameter and the target difference.
[0096] In this embodiment of the disclosure, after determining that the target trigger time is less than the specified frame trigger timestamp, the first electronic device subtracts the target difference from the original phase parameter to obtain a new phase parameter, and writes it back to the phase parameter field in the signal control parameters to complete the update.
[0097] A target trigger time less than the specified frame trigger timestamp indicates that the actual trigger time of the specified frame before correction is later than the target trigger time, i.e., frame synchronization hardline trigger lag. For example, if the target trigger time is a whole second 1000.000000s, and the actual trigger time of the specified frame is 1000.000800s, the actual trigger lags behind the target by 800μs. After updating based on the difference between the original phase parameter and the target value, the phase parameter is adjusted in the decreasing direction. Since the trigger time will be earlier than before correction when the phase parameter decreases, this adjustment direction compensates for the deviation of the actual trigger lag before correction, causing the trigger time of the frame synchronization hardline after correction to converge towards the target trigger time.
[0098] In this embodiment, the phase parameter is automatically adjusted by addition or subtraction based on the relationship between the target trigger time and the actual trigger time. The direction is adaptive and the adjustment amount equals the deviation, achieving precise correction. Furthermore, the adjustment operation directly acts on the phase parameter without changing the frame period or duty cycle, eliminating the need for additional synchronization circuitry and improving the efficiency of phase deviation correction.
[0099] Optionally, step 102 may also include the following steps: Step 1101: Resend the frame trigger signal.
[0100] In this embodiment, after determining that frame synchronization correction is required, the first electronic device can send a shutdown command to the PWM generator to stop the current PWM output. Subsequently, when the trigger time corresponding to the detection signal (i.e., the N+1th minute of the second) arrives, a re-enable command is sent to the PWM generator to restore the PWM output based on the original signal control parameters. This enables the first electronic device to re-send frame trigger signals at a preset frame rate to one or more camera units connected to the first electronic device via hardwire, based on the signal control parameters, to synchronously trigger each camera unit. After the PWM generator is re-enabled, it restarts counting from the minute of the second. The deviation between the first rising edge pulse of its output and the re-enable time is usually less than the frame period. Therefore, the trigger time of the first frame trigger signal after re-enablement can be considered to be basically aligned with the minute of the second, thereby completing the frame synchronization phase correction.
[0101] It is understandable that when the first electronic device and the second electronic device simultaneously perform the operation of turning off and then on the PWM generator, the PWM output of each device is recalculated from the same moment. This is equivalent to the automatic completion of a frame synchronization and realignment among multiple devices, achieving global synchronization convergence without the need for communication between devices.
[0102] In this embodiment of the disclosure, by turning off the PWM output and then re-enabling it, the PWM is restarted from the whole second. Without modifying any parameters, the frame synchronization hard-wire trigger time can be forced to return to the whole second reference through hardware reset.
[0103] Optionally, embodiments of this disclosure may further include the following steps: Step 1201: When the first electronic device is a transmitting device, a target time is determined based on a preset delay, and in response to the arrival of the target time, target data is sent to the receiving device.
[0104] In this embodiment of the disclosure, when the first electronic device is a transmitting device, a second electronic device that needs to synchronize its system clock with the first electronic device can be identified as a receiving device. The transmitting device and the receiving device are connected wirelessly. The receiving device can be the party requiring clock correction, and includes a processor and a wireless communication module. The processor can include a system-on-a-chip (SoC), and the wireless communication module can be an ultra-wideband (UWB) module. The transmitting device can be the party initiating the time synchronization process, and includes a processor and a wireless communication module. The processor of the transmitting device can be a system-on-a-chip (SoC), and the wireless communication module of the transmitting device can be an ultra-wideband (UWB) wireless module.
[0105] In one possible implementation, the transmitting device can be the master device, and the receiving device can be the slave device.
[0106] In one possible implementation, the receiving device can be a device equipped with a UWB wireless module in a multi-sensor data acquisition system, such as a wearable hand device. The transmitting device can be a device equipped with a UWB wireless module in a multi-sensor data acquisition system, such as a wearable head device.
[0107] In one possible implementation, the system clock of the receiving device is used to generate the system time, and the local clock of the wireless communication module of the receiving device is used to generate the local time.
[0108] The target time is determined based on a preset delay. The transmitting device can calculate the future target time based on a pre-configured preset delay and the current time to ensure that the transmission timing is controllable. The preset delay can be a pre-set desired time offset between the current time and the target transmission time.
[0109] In response to reaching the target time, the transmitting device sends the target data to the receiving device. Specifically, when the local clock of the transmitting device's wireless communication module reaches the target time, the wireless communication module is triggered to send the target data carrying the system's transmission time.
[0110] After the transmitting device sends the target data, the target data is transmitted via the wireless air interface and received by the wireless communication module of the receiving device. The receiving device then determines its target response duration for the target data. Based on the determined target response duration, the receiving device corrects its system clock. Specifically, the receiving device can use an interrupt mechanism, utilizing the target response duration and the system interrupt time recorded when the interrupt response is completed, to determine the accurate system time when the target data arrives at the receiving device, which is then used as the system reception time.
[0111] For example, after the processor of the transmitting device calculates the target time, it can write the target time into a register in the wireless communication module, such as the DX_TIME register. Simultaneously, the processor of the transmitting device can send a delayed transmission command to the wireless communication module, which instructs the wireless communication module to trigger the transmission of the target data when the local clock reaches the target time.
[0112] Step 1202: If the first electronic device is a receiving device, receive the target data.
[0113] Step 1203: Determine the target response time of the receiving device for the target data; the target response time is related to the current task status of the receiving device.
[0114] Step 1204: Based on the target response duration, correct the system clock of the receiving device.
[0115] In this embodiment of the disclosure, when the first electronic device is a receiving device, it can receive target data sent by a transmitting device, such as any second electronic device. The target data is sent by the transmitting device at a target time, which is a local time determined by the transmitting device based on a preset delay. The target time may refer to a scheduled transmission time point configured by the processor of the transmitting device based on the local clock of the wireless communication module, that is, the time when the transmitting device expects the wireless communication module to trigger data transmission when its local time reaches the target time.
[0116] After the transmitting device sends the target data, the target data is transmitted over the wireless air interface and then received by the wireless communication module of the receiving device. For example, the wireless communication module of the receiving device (e.g., a UWB wireless module) receives the target data sent by the transmitting device through a radio frequency antenna.
[0117] The receiving device determines its target response duration for the target data. In some embodiments, the target response duration is used to characterize the time elapsed from when the receiving device receives the target data until the processor of the receiving device completes its interrupt response to the target data.
[0118] When the processor detects an interrupt trigger signal, it will look up the interrupt vector table according to the interrupt number corresponding to the interrupt source to obtain the entry address of the interrupt service routine (ISR) associated with the interrupt, and jump to that address to execute.
[0119] Because the processor of the receiving device operates using a task scheduling mechanism, different tasks have different processing priorities, or the currently executing instructions of the receiving device cannot be interrupted, and the processor's processing speed is affected by the current load, the time span from when the receiving device receives the target data to when the interrupt response is completed is not a fixed value, but an uncertain quantity that dynamically changes with the task scheduling state. The target response time is related to the task scheduling of the receiving device's processor.
[0120] For example, due to task scheduling, the current instruction of the receiving device cannot be interrupted. In this case, the receiving device needs to process the current instruction before responding to the target data. Alternatively, if the receiving device is currently idle, it can directly respond to the target data. Therefore, the response time is not fixed.
[0121] For example, due to task scheduling, the priority of the receiving device's response to the target data may be lower than the priority of the receiving device's current task. In this case, the receiving device needs to complete its current task before responding to the target data. Conversely, the priority of the receiving device's response to the target data may be higher than the priority of its current task, or the receiving device may be in an idle state. In this case, the receiving device can directly respond to the target data. Therefore, the response time is not fixed.
[0122] Therefore, the current task state of the receiving device may be idle or executing a task. The currently executing task may be interruptible or non-interruptible. For example, if the receiving device is currently executing a task that includes multiple instructions, and the currently executing task can be interrupted, then the receiving device needs to complete the current instructions before interrupting the current task. Therefore, due to task scheduling, the target response time is affected by the current task state, making this time non-deterministic.
[0123] For example, the target response duration can be determined based on the local time when the receiving device completes the interrupt response and the local time when the receiving device receives the radio frequency signal of the target data.
[0124] The receiving device calibrates its system clock based on the determined target response duration to eliminate the impact of the target response duration on the calibration. Specifically, the receiving device can use an interrupt mechanism, along with the target response duration and the system interrupt time recorded upon completion of the interrupt response, to determine the accurate system time at which the target data arrives at the receiving device, and use this as the system reception time.
[0125] Under ideal conditions where the system clocks of the transmitting and receiving devices are synchronized, the system reception time of the receiving device should equal the system time of the actual transmission of the target data by the transmitting device plus the air interface transmission delay. However, in reality, there may be discrepancies between the system clocks of the transmitting and receiving devices, requiring clock correction for synchronization. This can be achieved by comparing the calculated system reception time with the actual transmission time and air interface transmission delay of the transmitting device to determine the time deviation of the receiving device's system clock relative to the transmitting device's system clock. The receiving device can then correct its system clock based on this time deviation, thereby achieving system clock synchronization between the transmitting and receiving devices.
[0126] In this embodiment, when the first electronic device is a transmitting device, target data is sent to the receiving device. Furthermore, when the first electronic device is a receiving device, the target response time is actually determined by the receiving device upon receiving the target data, rather than relying on a fixed compensation value. Based on this target response time, the system clock of the receiving device is then corrected. The entire process can complete clock deviation calculation and correction solely based on the target data transmission between the transmitting and receiving devices and the measured target response time. No wired connection is required, and no dedicated time synchronization hardware module is needed. This improves the convenience of time synchronization and reduces its cost while ensuring accurate correction of the receiving device's system clock relative to the transmitting device's system clock.
[0127] Optionally, step 1202 may include the following steps: Step 1301: Receive the target data through the wireless communication module of the receiving device.
[0128] Accordingly, after step 1202, this embodiment of the disclosure may further include the following steps: Step 1302: Generate an interrupt trigger signal through the wireless communication module.
[0129] In this embodiment of the disclosure, after the wireless communication module of the receiving device receives the target data, the wireless communication module generates an interrupt trigger signal and sends the interrupt trigger signal to the processor of the receiving device. For example, after the wireless communication module completes the data packet verification, it can generate an interrupt trigger signal by pulling the IRQ pin high to trigger the processor of the receiving device to enter the interrupt handling process.
[0130] Accordingly, step 1203 may include the following steps: Step 1303: Determine the target response time based on the response time of the processor of the receiving device to the interrupt trigger signal.
[0131] In this embodiment of the disclosure, the receiving device determines the target response duration based on the processor's response time to the interrupt trigger signal. Specifically, the receiving device can obtain the local interrupt time when the interrupt response is completed, compare this local terminal time with the local reception time of the target data automatically locked by the wireless communication module, and the difference between the two is the target response duration.
[0132] In some embodiments, the local reception time corresponds to the instant when the target data hits the antenna, while the local interrupt time for interrupt response completion corresponds to the moment when the processor of the receiving device completes the interrupt processing. Therefore, the target response duration fully covers the entire delay from the arrival of data to the completion of the response in the receiving device, including the data processing delay inside the wireless communication module, the communication transmission delay of the interrupt signal from the wireless communication module to the processor, and the instruction execution delay of the processor executing the interrupt service routine.
[0133] In some embodiments, the processor of the receiving device may be a system-on-a-chip (SoC), and the wireless communication module of the receiving device may be an ultra-wideband (UWB) wireless module.
[0134] For example, due to task scheduling, the current instruction received by the receiving device cannot be interrupted. Therefore, the receiving device needs to process the current instruction before responding to the interrupt trigger signal. The size of the current instruction is not fixed. Thus, the response time is not fixed. Alternatively, if the receiving device is currently idle, it can directly respond to the interrupt trigger signal.
[0135] For example, due to task scheduling, the priority of the receiving device's response to the interrupt trigger signal may be lower than the priority of the receiving device's current task. In this case, the receiving device needs to complete its current task before responding to the interrupt trigger signal. Conversely, if the priority of the receiving device's response to the interrupt trigger signal may be higher than the priority of the receiving device's current task, or if the receiving device is currently idle, it can directly respond to the interrupt trigger signal. Therefore, the target response time is not a fixed duration.
[0136] Therefore, the current task state of the receiving device may be idle or executing a task. The currently executing task may be interruptible or non-interruptible. For example, if the receiving device is currently executing a task that includes multiple instructions, and the currently executing task can be interrupted, then the receiving device needs to complete the current instructions before interrupting the current task. Therefore, due to task scheduling, the target response time is affected by the current task state, making this time non-deterministic.
[0137] In this embodiment of the disclosure, the receiving device determines the target response time based on the response duration to the interrupt trigger signal. Since the target response time is related to task scheduling and cannot be predicted, the interrupt mechanism can measure the actual target response time, ensuring that the subsequently back-calculated data arrives at the receiving device system at an accurate time, thereby improving clock correction accuracy.
[0138] Optionally, step 1303 may include the following steps: Step 1401: Determine the target response duration based on the local reception time of the wireless communication module corresponding to the arrival of the target data at the receiving device and the local interruption time of the wireless communication module corresponding to the completion of the interrupt trigger signal response.
[0139] In this embodiment, the local reception time can be the local time of the receiving device when the target data arrives at the wireless communication module of the receiving device; the local interrupt time can be the local time read by the processor of the receiving device from the wireless communication module when the interrupt trigger signal response is completed. Based on the local interrupt time and the local reception time, the target response duration can be obtained.
[0140] In this embodiment, the local reception time is determined by hardware when the target data arrives at the receiving device and the RF hits the antenna. The local interrupt time is also directly obtained via SPI reading. Both are definite and obtainable time values, eliminating the need to estimate or decompose the delays of each stage. The target response duration is calculated based on these two values, ensuring that the target response duration accurately reflects the actual processing time of the receiving device from the arrival of data to the completion of the interrupt response.
[0141] Optionally, step 1401 may include the following steps: Step 1501: Obtain the local time when the target data arrives at the wireless communication module of the receiving device, as the local reception time.
[0142] In this embodiment, the wireless communication module of the receiving device automatically locks and records the local time corresponding to the target data received by the wireless communication module at the hardware level, and uses it as the local reception time. Specifically, after the preamble detection is completed and the SFD verification is passed, a time counter is captured at the start of the first symbol of the PHY header (PHR) in the target data (RMARKER), and the local reception time is determined. The local reception time is used to characterize the local time when the receiving device receives the target data. This process is automatically completed by the wireless communication module hardware without processor intervention, ensuring the real-time performance and accuracy of the time locking.
[0143] For example, after the wireless communication module of the transmitting device sends the target data, the radio frequency signal of the target data propagates through the air interface and hits the antenna of the wireless communication module of the receiving device. After the radio frequency signal hits the antenna, a local timestamp can be obtained as the local reception time. This local reception time can be the logical time of the start of a calibrated reception frame.
[0144] Step 1502: Obtain the local time of the wireless communication module corresponding to the completion of the interrupt trigger signal response, and use it as the local interrupt time.
[0145] In this embodiment, after the receiving device's wireless communication module completes packet verification of the target data, it pulls the interrupt request pin high to trigger an interrupt trigger signal. The receiving device's processor responds to the interrupt trigger signal, enters the interrupt handling process, and reads the wireless communication module's buffer via the SPI bus to obtain the target data. Once the processor completes reading the target data, it signifies the end of its response to the interrupt. The receiving device's processor can then read the local time of the wireless communication module via the SPI bus as the local interrupt time.
[0146] Step 1503: Determine the target response duration based on the local interrupt time, the local reception time, and the first delay time, wherein the first delay time is the time required for the processor to obtain the local interrupt time from the wireless communication module.
[0147] In this embodiment, the local time from when the processor issues an SPI read command to when it reads the wireless communication module is not instantaneous. Instead, it involves the following process: the processor executes the SPI read instruction, the SPI command is transmitted to the wireless communication module via the bus, the wireless communication module parses the command, and the read time register is used to obtain the local interrupt time. That is, this local interrupt time is the actual UWB local time when the SPI read is completed, and it is delayed relative to the true local time when the interrupt response is completed. Therefore, to determine the true local time when the interrupt response is completed, the local interrupt time can be subtracted from the first delay time. This eliminates the additional delay introduced by the SPI read process. The first delay time is the time required for the processor to obtain the local interrupt time from the wireless communication module.
[0148] It is understandable that the first delay time can be a relatively stable delay of the receiving device during data transmission and processing, and the actual value of the first delay time and the error of the measurement or estimation are within the allowable range.
[0149] In some embodiments, in addition to the first delay time, other delay times during data transmission and processing by the receiving device can be considered, such as instruction execution time, data processing time, or other data transmission time. These delay times are typically determined by inherent factors such as the physical characteristics of the hardware itself, data format, or data processing rules, and have relatively stable characteristics, not easily affected by load or operating environment. Therefore, these delay times can be measured or estimated through theoretical analysis or empirical statistics, and the error between their actual values and the measured or estimated values is within an acceptable range.
[0150] Therefore, the local interrupt time read by the processor is actually the value of the actual local time when the interrupt response is completed plus the first delay time, that is: local interrupt time = actual local time when the interrupt response is completed + first delay time.
[0151] The target response duration is determined based on the local interruption time, local reception time, and the first delay time. Since the target response duration is defined as the time from the completion of the interruption response to the arrival of the data, that is: Target response duration = Actual local time at the completion of the interruption response. Substituting the local reception time into the above formula, we get: Target response time = Local interruption time First delay time Local reception time. Thus, the target response time is the actual time elapsed from the arrival of the target data to the completion of the response to the interrupt trigger signal by the receiving device.
[0152] In this embodiment, by subtracting the first delay time from the difference between the local interrupt time and the local reception time, the error in obtaining the local interrupt time due to communication delay in the SPI reading process is eliminated. Furthermore, since the target response time cannot be preset to a fixed value due to factors such as task scheduling, this solution accurately obtains the actual target response time of the receiving device for the current target data by subtracting the first delay time from the difference between the local interrupt time and the local reception time. By using the interrupt mechanism as an anchor point to bypass the uncertainty caused by task scheduling, the flexibility and accuracy of obtaining the target response time are improved.
[0153] Optionally, step 1204 may include the following steps: Step 1601: Obtain the system transmission time from the target data; the system transmission time is determined based on the system clock of the device transmitting the target data.
[0154] In this embodiment of the disclosure, when the transmitting device transmits target data, it can do so by scheduling the transmission, so that the wireless communication module of the transmitting device transmits the target data to the receiving device when the scheduled transmission time, i.e., the target time, is reached in a local time. In order for the receiving device to obtain the system time when the transmitting device transmits the target data, the system time when the transmitting device transmits the target data, i.e., the system transmission time, can be calculated in advance and written into the target data.
[0155] Step 1602: Based on the system transmission time, the system time of the receiving device, and the target response duration, correct the system clock of the receiving device.
[0156] In this embodiment of the disclosure, by combining the known system transmission time of the transmitting end, the target response duration measured by the receiving end, and the current system time of the receiving end, the system clock of the receiving end can be corrected without relying on external time synchronization, thereby aligning the system time of the receiving device with the system time of the transmitting device. The system time of the receiving device is the time indicated by its system clock when the correction operation is performed.
[0157] In this embodiment of the disclosure, clock correction can be completed without external time synchronization by combining the system time of the receiving device and the target response time. All the time bases required for correction are obtained through a single data transmission and reception process, which simplifies the correction process and reduces dependence by eliminating the need for additional hardware synchronization mechanisms.
[0158] Optionally, step 1602 may include the following steps: Step 1701: Obtain the system time corresponding to the completion of the interrupt trigger signal response, and use it as the system interrupt time.
[0159] In this embodiment of the disclosure, the system time corresponding to the completion of the interrupt trigger signal response by the receiving device is obtained as the system interrupt time. For example, the processor of the receiving device can read and record the current system clock value after completing the interrupt response to the interrupt trigger signal, and use this value as the system interrupt time.
[0160] Step 1702: Based on the target response time and the system interruption time, determine the system reception time of the receiving device; the system reception time is the system time corresponding to when the target data arrives at the receiving device.
[0161] In this embodiment, the system reception time of the receiving device can be calculated based on the target response time and the system interruption time. Since the target response time is the actual time from data arrival to the completion of the interruption response, subtracting the target response time from the system interruption time allows for the reverse calculation of the system time corresponding to the data arrival time; that is: System reception time = System interruption time. Target response time. System reception time is the system time corresponding to when the target data arrives at the receiving device, used to characterize the actual arrival time of the data at the receiving end.
[0162] Step 1703: Based on the system transmission time and the system reception time, correct the system clock of the receiving device.
[0163] In this embodiment of the disclosure, the system clock of the receiving device is corrected based on the system transmission time and the system reception time. Since the system transmission time is the known transmission time on the transmitting device side, and the system reception time is the reception time calculated on the receiving device side, the time deviation between the receiving device and the transmitting device can be determined through the system transmission time and the system reception time, and then correction is completed based on the time deviation.
[0164] In this embodiment, the system reception time is calculated by back-calculating the accurate target response time and system interruption time, achieving a precise mapping of the data arrival time in the system time dimension, so that the system reception time accurately corresponds to the actual arrival time of the target data. Furthermore, by comparing the system reception time with the system transmission time, the accurate clock deviation can be obtained, ensuring the accuracy of clock correction.
[0165] Optionally, step 1703 may include the following steps: Step 1801: Based on the system transmission time, the air interface transmission delay corresponding to the target data, and the system reception time, determine the time deviation of the receiving device relative to the transmitting device.
[0166] In this embodiment of the disclosure, the time deviation between the receiving device and the sending device is determined based on the system transmission time, the air interface transmission delay corresponding to the target data, and the system reception time. The air interface transmission delay can be the transmission time from when the sending device sends the target data to when the receiving device receives the target data. The time deviation is the offset of the receiving device's system clock relative to the sending device's system clock.
[0167] For example, since the transmitting and receiving devices of this disclosure are close to each other, the transmission delay of the target data in the air is very small and can be ignored. Taking the moment when the wireless communication module of the receiving device begins to receive the first byte of the target data as the timing start point, and the moment when the wireless communication module recognizes the valid frame and generates an interrupt trigger signal as the end point, the amount of data transmitted between the timing start point and the timing end point is converted into bits and divided by the wireless transmission rate to obtain the air interface transmission delay.
[0168] For example, when the wireless transmission rate is 1 Mbps, the total length of the target data is 10 bytes, and the first 4 bytes are a preamble, and the wireless communication module recognizes the valid frame and generates an interrupt after receiving the preamble, with N set to 4, the air interface transmission delay = 4 × 8 / 1000000 = 32 μs. This calculation method uses the actual byte position where the interrupt occurs as the timing endpoint, ensuring that the air interface transmission delay reflects the actual transmission time from when the target data enters the wireless channel to when the receiver interrupt is triggered.
[0169] The system transmission time plus the air interface transmission delay equals the theoretical system time at the sending end when the target data arrives at the receiving device. Comparing this theoretical system time with the system reception time, the difference between the two is the time deviation of the receiving device relative to the sending device, i.e.: Time deviation = (System transmission time + Air interface transmission delay) System reception time.
[0170] Step 1802: Based on the time deviation, correct the system clock of the receiving device.
[0171] In this embodiment of the disclosure, the system clock of the receiving device is corrected based on the time deviation. The time deviation is applied as a correction amount to the system clock of the receiving device to align it with the system time of the transmitting end.
[0172] Correcting the system clock of a receiving device based on time deviation may include: obtaining the current system time of the receiving device; and determining the absolute difference between the current system time and the time deviation as the corrected system time of the receiving device.
[0173] Specifically, the current system time of the receiving device is obtained. For example, the receiving device reads the current time indicated by its system clock as the current system time. The current system time is then calculated with the time deviation to determine the corrected system time. For instance, if the time deviation is positive, it means that the receiving device's system clock is faster than the transmitting device's system clock, so the time deviation is subtracted from the current system time; if the time deviation is negative, it means that the receiving device's system clock is slower than the transmitting device's system clock, so the absolute value of the time deviation is added to the current system time. In this way, the time deviation can be compensated to the receiving device's current system time in one step to obtain the corrected system time, thereby completing the correction of the receiving device's system clock and keeping it synchronized with the transmitting device's system time.
[0174] In this embodiment, the air interface transmission delay is superimposed on the system transmission time to obtain the theoretical time when the data arrives at the receiving end from the perspective of the transmitting device. This time is then compared with the system reception time calculated by the receiving device. The resulting time deviation compensates for both the air interface transmission delay and the clock offset at the receiving end, so that the correction result fully reflects the accumulated error, thereby improving the clock synchronization accuracy to a certain extent.
[0175] Optionally, the first delay time can be determined by the following steps: Step 1901: Obtain the first rising edge time of the first trigger signal output by the wireless communication module of the receiving device when it receives the test data corresponding to the test command; wherein, after receiving the test command, the wireless communication module generates a test interrupt trigger signal.
[0176] In this embodiment, the first delay time can be obtained by the time acquisition device calibrating the receiving device. When the wireless communication module of the receiving device receives the test data corresponding to the test command, it outputs a first trigger signal, raising the level of the signal from a default low level to a high level. The time acquisition device acquires the first rising edge time corresponding to the first trigger signal. Specifically, it can monitor the level change of the first trigger signal, capture the rising edge transition moment of the signal, mark that moment as the first rising edge time, and store it locally. The time acquisition device can be integrated into the receiving device, and its function can be implemented by the processor and memory of the receiving device. The time acquisition device can also be an additional electronic device, including a processor and memory.
[0177] After receiving the test command, the wireless communication module verifies the test data. After verification, it sends a test interrupt trigger signal to the designated interrupt pin of the receiver's processor.
[0178] Step 1902: Obtain the second rising edge time of the second trigger signal output by the processor of the receiving device after responding to the test interrupt trigger signal.
[0179] In this embodiment of the disclosure, after the processor of the receiving device responds to the test interrupt trigger signal, it triggers the processor's interrupt response process. After completing the interrupt response process, a second trigger signal is output, and the level of the signal is pulled up from the default low level to the high level. The second rising edge time corresponding to the second trigger signal is obtained through a time acquisition device. Specifically, the level change of the second trigger signal can be monitored, and after capturing the rising edge transition time of the signal, this time is marked as the second rising edge time and stored locally.
[0180] Step 1903: Determine the first delay time based on the first rise time, the second rise time, and the response duration for the test data determined by the receiving device.
[0181] In this embodiment of the disclosure, the response time of the receiving device to the test data is determined based on the difference between the first local time corresponding to the wireless communication module when it receives the test data of the test instruction and the second local time of the wireless communication module read by the processor after completing the interrupt response process.
[0182] The difference between the acquired first rise time and second rise time is used to obtain the time interval between the first trigger signal and the second trigger signal. This time interval is used to characterize the externally observable total duration from the output of the first trigger signal by the wireless communication module to the output of the second trigger signal by the processor.
[0183] The first delay time is the difference between the response time of the receiving device to the test data and the total externally observable time.
[0184] It is understood that the calculation of the first delay time can be completed by the general computing unit inside the receiving device, or the collected time parameters can be uploaded to an external computing device to complete the final calculation. This disclosure does not limit this.
[0185] In this embodiment of the present disclosure, by testing the first rising edge time corresponding to the test data corresponding to the test instruction received by the wireless communication module, and the second rising edge time corresponding to the processor responding to the test interrupt trigger signal, combined with the internal response time calibration of the device, interference errors in the internal processing of the wireless communication module can be eliminated, thereby achieving calibration of the inherent delay in the processor interrupt response process.
[0186] Optionally, embodiments of this disclosure may further include the following steps: Step 2001: Obtain the local time of the wireless communication module of the transmitting device.
[0187] In this embodiment of the disclosure, when the first electronic device is a transmitting device, the transmitting device can obtain the current local time of its wireless communication module.
[0188] Accordingly, step 1201 may include the following steps: Step 2002: Determine the target time based on the current local time of the transmitting device and the preset delay.
[0189] In this embodiment of the disclosure, a preset delay is added to the local time to obtain the local time, which is then used as the target time. That is, the target time = the current local time + the preset delay.
[0190] In this embodiment of the disclosure, the transmission time is determined based on the local time of the wireless communication module, which avoids the conversion error between the processor system clock and the local time of the wireless communication module, and makes the target time precisely aligned with the actual transmission action of the wireless communication module.
[0191] In this embodiment of the disclosure, the non-deterministic processing delay caused by internal data scheduling and / or data processing can introduce random errors into time synchronization. By setting a preset delay, which is greater than the maximum possible value of the non-deterministic processing delay (e.g., 8ms), the preset delay can cover the non-deterministic processing delay, thereby eliminating random errors during time synchronization and making time synchronization more accurate.
[0192] Optionally, embodiments of this disclosure may further include the following steps: Step 2101: Obtain the system time of the processor of the transmitting device.
[0193] Step 2102: Determine the system transmission time of the processor based on the preset delay and the system time.
[0194] Step 2103: Determine the target data based on the system's transmission time.
[0195] In this embodiment of the disclosure, when the first electronic device is a transmitting device, the system time of the transmitting device is obtained. Specifically, the system time can be obtained by reading the current time value indicated by the system clock of the transmitting device through its processor.
[0196] The system transmission time is determined based on the preset delay and system time. The target data is then determined based on the system transmission time. The system transmission time can be written into the target data and sent along with it to the receiving device, allowing the receiving device to directly obtain the system time at the transmission moment.
[0197] Step 2102 may include the following steps: Step 2201: Determine the system transmission time based on the second delay time, the preset delay, and the system time, wherein the second delay time is the time required for the processor to obtain the local time of the wireless communication module from the wireless communication module.
[0198] In this embodiment, since the target data will be transmitted after a preset delay, the system time needs to be increased by the preset delay and decreased by the time required for the processor to read the current local time of the wireless communication module, i.e., the second delay time, to compensate for the advance of the system clock during the local time reading process. This is to obtain the true system time corresponding to the target time, i.e., the system transmission time. Thus, by eliminating the second delay time when determining the system transmission time, a more accurate true system time of the processor corresponding to the target time of the wireless communication module when the target data is transmitted can be obtained. (System transmission time)
[0199] In this embodiment of the disclosure, the system transmission time is determined in advance based on a preset delay and the system time, that is, the system time corresponding to the transmission of the target data, and the target data is determined based on the system transmission time. This allows the receiving device to directly obtain the system time of the sending device when the target data is transmitted, which is convenient for time synchronization with the system transmission time as a reference.
[0200] Optionally, the second delay time can be determined based on the following steps: Step 2301: Obtain the third rising edge time of the third trigger signal corresponding to the test command sent by the processor of the transmitting device to the wireless communication module of the transmitting device.
[0201] In this embodiment, the second delay time can be obtained by testing and calibrating the transmitting device using a time acquisition device. The processor sends a test command to the wireless communication module of the transmitting device and outputs a third trigger signal. The time acquisition device pulls the level of this signal from a default low level to a high level. The time acquisition device obtains the third rising edge time corresponding to the third trigger signal. Specifically, it can monitor the level change of the third trigger signal, capture the rising edge transition moment of the signal, mark this moment as the third rising edge time, and store it locally. The time acquisition device can be integrated into the transmitting device, and the processor and memory of the transmitting device implement the function of the time acquisition device. The time acquisition device can also be an additional electronic device, including a processor and memory.
[0202] Step 2302: Obtain the fourth rising edge time corresponding to the fourth trigger signal output by the wireless communication module of the transmitting device before transmitting the test command at the test time; the test time is determined based on a preset test delay.
[0203] In this embodiment, after the processor issues the test command, it immediately reads the current third local time of the wireless communication module, adds this third local time to the preset test delay, calculates the corresponding test moment, and synchronously sends it to the wireless communication module. The wireless communication module continuously times under the drive of its internal clock. When it reaches the instantaneous node with a preset lead time to the test moment, it pulls the GPIO pin level corresponding to the pre-configured fourth trigger signal from the default low level to the high level. The time acquisition device synchronously monitors the level state of this GPIO pin, captures the instantaneous moment of the level transition from low to high, marks this moment as the fourth rising edge time, and stores it locally to obtain the termination timing reference for the transmission process.
[0204] Step 2303: Determine the second delay time based on the third rising edge time, the preset test delay, and the fourth rising edge time.
[0205] In this embodiment of the disclosure, the difference between the acquired fourth rising edge time and the third rising edge time is used to obtain the total externally observable duration from the processor outputting the third trigger signal to the wireless communication module outputting the fourth trigger signal. The difference between this total externally observable duration and the pre-configured preset test delay is the second delay time.
[0206] It is understood that the calculation of the second delay time can be completed by the general computing unit inside the transmitting device, or the collected time parameters can be uploaded to an external computing device to complete the final calculation. This disclosure does not limit this.
[0207] In this embodiment of the disclosure, by collecting the rising edge time of two trigger signals and combining it with a preset test delay to complete the calibration, the interference caused by the reserved waiting delay can be eliminated, and the inherent delay of the wireless transmission link can be calibrated, thereby reducing the system error of the delay measurement at the transmitting device end.
[0208] For example, Figure 4 A schematic diagram illustrating the specific steps of a multi-device frame synchronization method is shown, as follows: Figure 4As shown, system clock synchronization is performed between the first and second electronic devices. The system awaits an external enable command and, in response, configures signal control parameters. A detection signal is generated at fixed intervals, assuming a detection signal is generated every 1 second (one full second). After the first electronic device generates a detection signal, it triggers an interrupt event in response. The system acquires the frame trigger timestamp sequence recorded by the first electronic device and selects the most recent frame trigger timestamp as the trigger time for the specified frame. Based on the trigger time of the specified frame and the trigger time of the detection signal, it determines whether frame synchronization correction is needed. If so, frame synchronization correction is performed by retransmitting the frame trigger signal or adjusting the phase parameters in the signal control parameters; otherwise, it waits for the next detection signal.
[0209] Figure 5 A timing relationship diagram of a time synchronization method provided in this embodiment of the disclosure is shown below. Figure 5 As shown, the transmitting device includes a main SoC and a main UWB, and the receiving device includes a slave SoC and a slave UWB. The transmitting device can periodically trigger the correction process according to a preset correction cycle. Specifically, the main SoC of the transmitting device can send a delayed transmission command to the main UWB at system time T_soc1 to instruct the main UWB to transmit the target data at the target time. The target data carries the sum of system time T_soc1 and the preset delay, i.e., the system transmission time. When the slave UWB of the receiving device receives the target data, it records the local reception time T_uwb2_rx_ts and generates an interrupt trigger signal. It obtains the local interrupt time T_uwb2_sys_ts and the system interrupt time T_soc2_int corresponding to the completion of the interrupt trigger signal response. Based on the local interrupt time T_uwb2_sys_ts and the local reception time T_uwb2_rx_ts, the target response duration is calculated. Based on the system interrupt time T_soc2_int and the target response time, the system time from the SoC when the target data is received is determined, i.e., the system reception time T_soc2_rx.
[0210] Figure 6 A flowchart illustrating the specific steps of a time synchronization method provided in this embodiment of the disclosure is shown below. Figure 6As shown, the transmitting device includes a main SoC and a main UWB, and the receiving device includes a slave SoC and a slave UWB. In the first stage, the transmitting device performs time mapping and delayed transmission configuration: The main SoC obtains the current system time and reads the local time of the main UWB via SPI. The main UWB returns the current local time to the main SoC. The main SoC calculates the system transmission time based on the current system time and a preset delay, and calculates the target time based on the current local time and the preset delay. The system transmission time is written to TX_BUFFER, the target time is written to DX_TIME, and a delayed transmission command is sent to the main UWB. In the second stage, zero-jitter physical layer RF transmission: When the local time reaches the target time, the main UWB automatically triggers the transmission of target data. When the RF signal of the target data hits the slave UWB antenna, the hardware automatically locks the local reception time corresponding to the arrival of the target data. The third stage involves receiving data and extracting timestamps from the receiving device: After UWB verification is completed, the IRQ pin is pulled high to trigger an interrupt. The SoC responds to the interrupt trigger signal and enters the interrupt handling process. It reads the receive buffer from the UWB via the SPI bus, obtains the target data from the UWB, and gets the system transmission time. After the SoC's response to this interrupt ends, it records the current system time to obtain the system interrupt time and reads the local time from the UWB via the SPI bus as the local interrupt time. The SoC then reads the local receive time from the UWB for subsequent time deviation calculations. The fourth stage involves time deviation calculation and system clock correction: The SoC calculates the target response duration based on the local receive time and the local interrupt time; it calculates the system receive time based on the target response duration and the system interrupt time; it determines the time deviation based on the system receive time and the system transmission time; and it corrects its own system clock based on the time deviation.
[0211] This application provides a system composed of multiple devices, including a first electronic device and a second electronic device. The first electronic device and the second electronic device are wearable devices. Both the first electronic device and the second electronic device are wired to one or more camera units. The system clocks of the first electronic device and the second electronic device are synchronized. The trigger time of the frame corresponding to the first electronic device and the trigger time of the frame corresponding to the second electronic device satisfy a preset correspondence. The first electronic device and the second electronic device can be used to collect multi-frame data, which can be point cloud data, image data, etc. The first electronic device is configured to perform: A detection signal is generated at a preset time; In response to the detection signal, frame synchronization correction is performed based on the trigger time corresponding to the detection signal and the trigger time of a specified frame, wherein the specified frame is a frame in the frame sequence corresponding to the first electronic device; The second electronic device is configured to perform: A detection signal is generated at the preset time; In response to the detection signal, frame synchronization correction is performed based on the trigger time corresponding to the detection signal and the trigger time of a specified frame, wherein the specified frame is a frame in the frame sequence corresponding to the second electronic device.
[0212] In this way, by synchronizing the system clocks of the first electronic device and the second electronic device, the two devices generate detection signals at preset times and respond to the detection signals. Based on the trigger time corresponding to the detection signal and the trigger time of the specified frame, frame synchronization judgment and correction are performed to enable the corresponding frames of multiple camera units connected to multiple devices to be exposed at the same time, ensuring that the data collected by multiple devices are aligned in time and improving data quality.
[0213] Optionally, the first electronic device and the second electronic device are also configured to perform the various processes of the above-described multi-device frame synchronization method embodiments, and can achieve the same technical effect. To avoid repetition, they will not be described again here.
[0214] Optional, such as Figure 7 As shown, this application embodiment also provides an electronic device, including a processor 2410, a memory 2409, and a program or instructions stored in the memory 2409 and executable on the processor 2410. When the program or instructions are executed by the processor 2410, they implement the various processes of the above-described multi-device frame synchronization method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0215] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.
[0216] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.
[0217] The electronic device 2400 includes, but is not limited to, components such as: radio frequency unit 2401, network module 2402, audio output unit 2403, input unit 2404, sensor 2405, display unit 2406, user input unit 2407, interface unit 2408, memory 2409, and processor 2410.
[0218] Those skilled in the art will understand that the electronic device 2400 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 2410 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 8The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.
[0219] It should be understood that, in this embodiment, the input unit 2404 may include a graphics processing unit (GPU) 24041 and a microphone 24042. The GPU 24041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 2406 may include a display panel 24061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 2407 includes at least one of a touch panel 24071 and other input devices 24072. The touch panel 24071 is also called a touch screen. The touch panel 24071 may include a touch detection device and a touch controller. Other input devices 24072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.
[0220] The memory 2409 can be used to store software programs and various data. The memory 2409 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 2409 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 2409 in this embodiment includes, but is not limited to, these and any other suitable types of memory.
[0221] Processor 2410 may include one or more processing units; optionally, processor 2410 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 2410.
[0222] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described multi-device frame synchronization method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0223] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0224] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described multi-device frame synchronization method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0225] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0226] This application provides a computer program product stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the multi-device frame synchronization method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0227] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0228] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0229] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A multi-device frame synchronization method, characterized in that, The method is applied to a first electronic device, which is used to acquire multiple frames of data; the method includes: A detection signal is generated at a preset time, wherein the second electronic device also generates a detection signal at the preset time, the system clocks of the first electronic device and the second electronic device are synchronized, and the second electronic device is used to collect multiple frames of data; In response to a detection signal generated by the first electronic device, a target difference corresponding to the trigger time of a specified frame is determined with reference to the trigger time of the detection signal. Based on the target difference, frame synchronization correction is performed on the first electronic device. The specified frame is a frame in the frame sequence corresponding to the first electronic device before or after the trigger time, so that the trigger time of the frame corresponding to the first electronic device after frame synchronization correction and the trigger time of the frame corresponding to the second electronic device satisfy a preset correspondence. The preset correspondence is a frame index mapping rule determined based on the ratio of the preset frame rates of the first electronic device and the second electronic device.
2. The method according to claim 1, characterized in that, The method further includes: The first electronic device sends frame trigger signals to one or more camera units connected to the first electronic device at a preset frame rate via a wired connection to synchronously trigger the one or more camera units; wherein the first electronic device and the second electronic device send frame trigger signals at different preset frequencies; or the first electronic device and the second electronic device send frame trigger signals at the same preset frequency.
3. The method according to claim 2, characterized in that, The method further includes: Obtain the frame trigger timestamp corresponding to the frame trigger signal; The frame trigger timestamps are stored in the target data structure to obtain a frame trigger timestamp sequence; The trigger time of the specified frame is obtained from the frame trigger timestamp sequence.
4. The method according to claim 3, characterized in that, The method further includes: For any frame trigger timestamp stored in the target data structure, if the difference between the frame trigger timestamp and the current time is greater than a preset duration threshold, the frame trigger timestamp is deleted from the target data structure.
5. The method according to claim 1, characterized in that, The detection signal is also used to trigger an interrupt, and the detection signal is generated based on an external wired pulse signal or an internal interrupt event of the operating system.
6. The method according to claim 5, characterized in that, The method further includes: Upon receiving the detection signal, an interrupt event is triggered in response to the detection signal; The step of performing frame synchronization correction on the electronic device based on the trigger time corresponding to the detection signal and the trigger time of a specified frame includes: In the interruption event, the first electronic device is subjected to frame synchronization correction based on the trigger time corresponding to the detection signal and the trigger time of the specified frame.
7. The method according to claim 1, characterized in that, The method further includes: Configure signal control parameters; the signal control parameters include at least one of the following: frame period, duty cycle; Based on the signal control parameters, a frame trigger signal is sent via a hardwire to one or more camera units connected to the first electronic device at a preset frame rate to synchronously trigger each camera unit.
8. The method according to claim 1, characterized in that, The step of responding to a detection signal generated by the first electronic device and performing frame synchronization correction on the first electronic device based on the trigger time corresponding to the detection signal and the trigger time of a specified frame includes: In response to the detection signal, if the first electronic device has enabled the output of a frame trigger signal, frame synchronization correction is performed on the first electronic device based on the trigger time corresponding to the detection signal and the trigger time of a specified frame.
9. The method according to claim 3, characterized in that, Obtaining the trigger time of the specified frame from the frame trigger timestamp sequence includes: The trigger time of the specified frame is determined based on the frame trigger timestamp corresponding to the position with the smallest time difference from the trigger time in the frame trigger timestamp sequence corresponding to the first electronic device.
10. The method according to claim 7, characterized in that, The step of determining the target difference corresponding to the trigger time of a specified frame using the trigger time corresponding to the detection signal as a reference, and performing frame synchronization correction on the first electronic device based on the target difference, includes: Based on the frame period and the trigger time corresponding to the detection signal, a standard trigger time corresponding to the specified frame is determined; based on the standard trigger time and a first preset phase threshold, a standard trigger time range is determined; in response to the trigger time of the specified frame not being within the standard trigger time range, a target difference between the standard trigger time and the trigger time of the specified frame is determined; based on the target difference, the phase parameter in the signal control parameters is adjusted; or... Based on the alignment mode parameter in the signal control parameters and the frame period, a second preset phase threshold is determined; the absolute value of the difference between the trigger time of the specified frame and the trigger time corresponding to the detection signal is calculated as the target difference; if the target difference is greater than the second preset phase threshold, the phase parameter in the signal control parameters is adjusted based on the target difference.
11. The method according to claim 1, characterized in that, The frame synchronization correction of the first electronic device includes: Resend the frame trigger signal.
12. The method according to claim 1, characterized in that, The method further includes: When the first electronic device is a transmitting device, a target time is determined based on a preset delay, and in response to reaching the target time, target data is sent to the receiving device; or, If the first electronic device is a receiving device, the target data is received; Determine the target response time of the receiving device for the target data; the target response time is related to the current task status of the receiving device; The system clock of the receiving device is corrected based on the target response duration.
13. A frame synchronization system composed of multiple devices, characterized in that, The device includes a first electronic device and a second electronic device, both of which are wearable devices. Both the first and second electronic devices are wired to one or more camera units. The system clocks of the first and second electronic devices are synchronized. The trigger time of the frame corresponding to the first electronic device and the trigger time of the frame corresponding to the second electronic device satisfy a preset correspondence relationship. The preset correspondence relationship is a frame index mapping rule determined based on the ratio of the preset frame rates of the first and second electronic devices. The first and second electronic devices are used to collect multiple frames of data. The first electronic device is configured to perform: A detection signal is generated at a preset time; In response to the detection signal generated by the first electronic device, a target difference corresponding to the trigger time of the first specified frame is determined with reference to the trigger time of the detection signal, and frame synchronization correction is performed based on the target difference, wherein the first specified frame is a frame in the frame sequence corresponding to the first electronic device before or after the trigger time. The second electronic device is configured to perform: A detection signal is generated at the preset time; In response to the detection signal, a target difference corresponding to the trigger time of the second specified frame is determined with reference to the trigger time of the detection signal, and frame synchronization correction is performed based on the target difference, wherein the second specified frame is a frame in the frame sequence corresponding to the second electronic device before or after the trigger time.
14. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the multi-device frame synchronization method as described in any one of claims 1-12.
15. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the multi-device frame synchronization method as described in any one of claims 1-12.
Citation Information
Patent Citations
Image acquisition method and device, equipment, storage medium and program product
CN120416670A
Multi-sensor time synchronization method, device, equipment, medium and product
CN120583196A
