A method and system for cross-physical layer wireless synchronous acquisition for multiple heterogeneous glasses terminals
By acquiring the capability information of heterogeneous glasses terminals, a synchronization group is dynamically established and a synchronization code is generated. The synchronization code is then distributed using the wireless physical layer, solving the problem of synchronization acquisition between heterogeneous glasses terminals and achieving high-precision frame-level and exposure-level synchronization.
Patent Information
- Application Number
- CN202611361784.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-03
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to achieve high-precision synchronous data acquisition across multiple heterogeneous glasses terminals, especially in wireless environments. Traditional solutions suffer from complex wiring, high costs, insufficient synchronization accuracy, and inadequate cross-physical layer reuse capabilities. Furthermore, post-processing methods cannot guarantee the actual synchronization during the acquisition phase.
By acquiring the capability information of heterogeneous glasses terminals, a synchronization group is dynamically established and a synchronization code is generated. The synchronization code is distributed using the wireless physical layer and the global acquisition time is converted into the local control time based on the local timing model, thereby realizing wireless synchronous acquisition across physical layers.
It significantly improves compatibility and scalability in heterogeneous environments, eliminates redundant instruction interference, ensures trigger reliability, and achieves accurate frame-level and exposure-level synchronization of glasses terminals with different hardware characteristics under a unified global time base.
Smart Images

Figure CN123124079A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of wireless communication and wearable device data processing technology, specifically to a cross-physical-layer wireless synchronous acquisition method and system for multiple heterogeneous glasses terminals. Background Technology
[0002] With the development of smart wearable devices such as smart glasses, AR glasses, and audio / video capture glasses, the demand for multiple different glasses terminals to collaboratively collect video, images, inertial data, and audio data is constantly increasing. In scenarios such as multi-person interactive recording, spatial content production, 3D scene reconstruction, immersive media production, remote collaboration, and embodied intelligent data collection, it is often necessary for multiple glasses terminals to collect data synchronously at the most consistent time possible.
[0003] However, existing multi-terminal synchronous acquisition solutions typically suffer from the following problems. First, traditional industrial camera synchronization often relies on a unified trigger line, a unified Genlock reference signal, or a unified PTP network. While this provides high synchronization accuracy, it results in complex wiring, high costs, and difficulty in directly applying it to glasses terminals from different manufacturers with different hardware interfaces. Second, common software control methods for consumer terminals mostly employ wireless LAN or Bluetooth to send "start acquisition" commands. For example, as described in patent document CN105589753B, "Intelligent Terminal Array Synchronous Working System and its Working Method," the host computer control module issues synchronous shooting time and shooting interval instructions to multiple client control modules via LAN or WAN. This method is easily affected by wireless propagation delay, operating system scheduling, camera pipeline activation delay, and device differences, making it difficult for different devices to complete acquisition at the same frame boundary or even the same exposure time, even if they receive commands simultaneously. Third, different glasses terminals support different wireless protocols. Some glasses terminals support Wi-Fi, some only support Bluetooth Low Energy control, and some use short-range wireless solutions with lower latency and higher reliability. Existing solutions are often designed for a fixed protocol and lack cross-physical layer multiplexing and bridging capabilities.
[0004] Furthermore, the publicly available capabilities of current terminal platforms vary. Some glasses terminals can provide deeper timestamps related to exposure or readout times, while others can only provide session clock or sample timestamp information. Simply requiring all glasses terminals to "start recording simultaneously" does not truly solve the problems of frame synchronization, exposure synchronization, and even rolling shutter reference time synchronization among multiple devices.
[0005] Existing technologies also include solutions for aligning materials through unified timecodes or post-processing, but these solutions are more suitable for post-processing and are insufficient to directly guarantee the actual synchronization during the acquisition stage. Especially in multi-visual terminals, multi-person scenarios, heterogeneous wireless bearers, and dynamic environments, if the "immediate trigger" control logic is still used, it is often difficult to balance synchronization accuracy, scalability, and deployment costs.
[0006] Therefore, how to build a synchronous acquisition mechanism that can work across different wireless physical layers such as Wi-Fi, Bluetooth, and Starflash for multiple different glasses terminals without relying on a unified hard trigger line and hardware Genlock has become an urgent problem to be solved. Summary of the Invention
[0007] To address one of the shortcomings of existing technologies, the purpose of this application is to provide a cross-physical-layer wireless synchronous acquisition method and system for multiple heterogeneous glasses terminals.
[0008] A first aspect of this application provides a cross-physical-layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals, comprising: Obtain terminal capability information reported by multiple candidate heterogeneous glasses terminals; Based on the data collection task and the terminal capability information, determine the glasses terminal for this round of synchronous data collection, establish a synchronization group and determine the synchronization group information, and generate a synchronization group snapshot. Based on the snapshot of the synchronization group, a synchronization code is generated; The synchronization code is distributed via at least one wireless physical layer carrier according to the available distribution path of each glasses terminal in the synchronization group. The synchronization codes received by each glasses terminal in the synchronization group are verified and deduplicated using a quadruple consisting of the synchronization group identifier, group version, collection round identifier, and command version, and the deduplicated synchronization codes are determined. Based on the deduplicated synchronization code and the currently valid local timing model, the target global acquisition time is converted into the local control time of the glasses terminal in the synchronization group, and the corresponding glasses terminal is triggered to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information.
[0009] Optionally, the method further includes: Based on the feedback information, the acquisition deviation information is determined, and the currently effective local timing model is updated. The local timing model includes at least one of a signed control timing offset parameter, an anchor point phase parameter, and a historical residual parameter. The signed control timing offset parameter represents the signed predicted timing offset between the control action time and the basic acquisition anchor point. The anchor point phase parameter represents the offset from the basic acquisition anchor point to the target acquisition anchor point required by the synchronization configuration. The historical residual parameter represents the residual compensation determined based on historical feedback information.
[0010] Optionally, the terminal capability information includes wireless bearer capability status, time capability status, sensor acquisition capability status, and feedback capability status, wherein the time capability status includes time base acquisition capability status or local clock mapping capability status, and timestamp acquisition capability status. The process of determining the glasses terminal for this round of synchronous data collection based on the data collection task and the terminal capability information, establishing a synchronization group and determining the synchronization group information, and generating a synchronization group snapshot includes: Candidate glasses terminals are determined based on the data collection task identifier, the candidate terminal list corresponding to the data collection task, the terminal selection criteria corresponding to the data collection task, and the registration deadline. Based on the terminal capability requirements corresponding to the data collection task and the terminal capability information of the candidate glasses terminals, the candidate glasses terminals are screened to determine the glasses terminals to be included in the group. Based on the participation confirmation and readiness status of the glasses terminals included in the group, the glasses terminals to be synchronously collected in this round are determined; A synchronization group is established based on the glasses terminals that have been synchronously collected in this round, and the synchronization group information is determined. The synchronization group information includes the synchronization group identifier, group version, global reference time within the group, the mapping relationship between the local clock of each glasses terminal and the global reference time within the group, and the synchronization configuration corresponding to the collection task. After the registration deadline is reached, the current round member set consisting of the glasses terminals that were synchronously collected in this round is frozen, and the current round member set and the synchronization group information are recorded as a synchronization group snapshot associated with the group version and remaining unchanged during the current round of synchronous collection.
[0011] Optionally, the core fields of the synchronization code include: synchronization group identifier group_id, group version group_version, collection round identifier round_id, command version command_version, reference time identifier reference_time_id, target global collection time target_global_time, valid time window valid_window, synchronization configuration sync_profile, and collection object information sensor_assignment; The synchronization code also includes at least one of the following: bearer strategy, feedback deadline, rollback strategy, and integrity verification information.
[0012] Optionally, distributing the synchronization code via at least one wireless physical layer bearer according to the available distribution paths of each glasses terminal in the synchronization group includes: The first candidate distribution path is determined based on the available distribution paths from the coordinating node to each glasses terminal in the synchronization group and the commonly supported wireless physical bearers and / or the available distribution paths from the coordinating node to each glasses terminal in the synchronization group via the bridging node and the commonly supported wireless physical bearers. Based on the path being reachable and the expected delivery time not later than the synchronization code reception deadline, the first candidate distribution path is filtered to determine the second candidate distribution path. For each glasses terminal, the second candidate distribution path is comprehensively evaluated based on at least one of the following factors: latency, jitter, packet loss rate, delivery rate, reliability, energy consumption, congestion level, number of bridging hops, and number of terminals supporting wireless physical layer transmission, according to the weights set in the task, and the comprehensive evaluation result is determined. For each glasses terminal, based on the comprehensive evaluation results, the optimal second candidate distribution path is determined as the main distribution path, and the wireless physical layer bearer used in the main distribution path is determined as the main bearer corresponding to the main distribution path. The wireless physical layer bearer includes Wi-Fi, Bluetooth, or Starflash. When there are multiple second candidate distribution paths for each glasses terminal, the second best second candidate distribution path is determined as the backup distribution path according to the comprehensive evaluation result, and the wireless physical layer bearer used in the backup distribution path is determined as the backup bearer corresponding to the backup distribution path. For each glasses terminal, a distribution method is determined based on the main distribution path and its corresponding main bearer, the backup distribution path and its corresponding backup bearer, and the distribution method includes at least one of single bearer distribution, multi-bearer redundant distribution and bridging distribution. Generate a logical synchronization code that includes the synchronization group identifier, group version, collection round identifier, and command version; Based on the primary distribution path and the backup distribution path, the logical synchronization code is encapsulated into one or more transmission copies, and the one or more transmission copies are distributed through the corresponding wireless physical layer bearer according to the distribution method.
[0013] Optionally, the step of verifying and deduplicating the synchronization codes received by each glasses terminal in the synchronization group using a quadruple consisting of a synchronization group identifier, group version, collection round identifier, and command version, to determine the deduplicated synchronization code, includes: Synchronization codes with the same synchronization group identifier, group version, collection round identifier, and command version are identified as transmission copies of the same logical synchronization code; One of the transmitted copies of the same logical synchronization code may be selected as the deduplicated synchronization code.
[0014] Optionally, the feedback information includes execution feedback information and link feedback information; The process involves converting the target global acquisition time into the local control time of the glasses terminal in the synchronization group based on the deduplicated synchronization code and the currently valid local timing model, and triggering the corresponding glasses terminal to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information, including: Based on the mapping relationship between the target global acquisition time in the deduplicated synchronization code and the local clock of each glasses terminal in the synchronization group information and the global reference time within the group, the target global acquisition time is converted into the local reference time of the glasses terminal in the synchronization group. The local reference time is compensated using the currently effective local timing model to determine the local control time of the glasses terminal in the synchronization group; The glasses terminal in the synchronization group is controlled to configure the object information to be collected before the local control time. The object information configuration includes at least one of camera configuration, inertial sensor configuration and audio configuration. When the local control time arrives, the corresponding glasses terminal is controlled to perform synchronous data acquisition, and the acquired data, the execution feedback information, and the link feedback information are obtained.
[0015] Optionally, the method further includes: When a glasses terminal in the synchronization group experiences one or more of the following situations: refusing the acquisition task, failing to receive the synchronization code before the synchronization code reception deadline, failing to form a usable target acquisition anchor point, the target acquisition anchor point exceeding the acquisition allowed window, or abnormal execution feedback information in multiple consecutive acquisition rounds participated in by the same glasses terminal, a rollback strategy is adopted to rollback the glasses terminal. Abnormal execution feedback information in multiple consecutive acquisition rounds participated in by the same glasses terminal includes at least one of the following: failing to receive execution feedback information before the feedback deadline, the execution feedback information indicating an invalid local clock mapping relationship, the acquisition anchor point semantics in the execution feedback information being inconsistent with the synchronization configuration, or the execution feedback information indicating an execution status of FAILED.
[0016] A second aspect of this application provides a cross-physical-layer wireless synchronous acquisition system for multiple heterogeneous glasses terminals, comprising: The terminal capability information acquisition module is used to acquire the terminal capability information reported by multiple candidate heterogeneous glasses terminals respectively. The synchronization group construction module is used to determine the glasses terminal for this round of synchronous data collection based on the data collection task and the terminal capability information, establish a synchronization group and determine the synchronization group information, and generate a synchronization group snapshot. A synchronization code generation module is used to generate a synchronization code based on the synchronization group snapshot; The synchronization code distribution module is used to distribute the synchronization code through at least one wireless physical layer carrier according to the available distribution path of each glasses terminal in the synchronization group. The synchronization code verification and deduplication module is used to verify and deduplicate the synchronization codes received by each glasses terminal in the synchronization group using a quadruple consisting of the synchronization group identifier, group version, acquisition round identifier, and command version, and to determine the deduplicated synchronization code. The synchronous acquisition module is used to convert the target global acquisition time into the local control time of the glasses terminal in the synchronization group according to the deduplicated synchronization code and the currently valid local timing model, and to trigger the corresponding glasses terminal to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information.
[0017] A third aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method provided in the first aspect of this application.
[0018] A fourth aspect of this application provides an electronic device comprising: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method provided in the first aspect of this application.
[0019] This application presents a cross-physical layer wireless synchronization acquisition method for multiple heterogeneous glasses terminals. It dynamically establishes synchronization groups and generates synchronization group snapshots and synchronization codes by acquiring the capability information of the heterogeneous glasses terminals. This extracts synchronization semantics from specific wireless physical layers, enabling unified scheduling and distribution based on available distribution paths under at least one wireless physical layer, significantly improving compatibility and scalability in heterogeneous environments. A four-tuple consisting of a synchronization group identifier, group version, acquisition round identifier, and command version is used to verify and deduplicate the synchronization codes distributed across multiple paths, effectively eliminating redundant command interference and ensuring trigger reliability. Abandoning the traditional immediate trigger mode, it maps the target global acquisition time to the local control time of each glasses terminal based on the currently effective local timing model for scheduled execution. This allows glasses terminals with different hardware characteristics to accurately align their acquisition windows under a unified global time reference, significantly improving frame-level and exposure-level synchronization consistency.
[0020] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description
[0021] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a cross-physical-layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals according to an exemplary embodiment.
[0022] Figure 2 This is a schematic diagram illustrating a process for cross-physical layer synchronization code distribution and bridging according to an exemplary embodiment.
[0023] Figure 3 This is a schematic diagram illustrating a local clock mapping and scheduled execution process according to an exemplary embodiment.
[0024] Figure 4 This is a schematic diagram illustrating a feedback correction process based on execution feedback information and link feedback information according to an exemplary embodiment.
[0025] Figure 5 This is a schematic diagram illustrating the structure of a cross-physical-layer wireless synchronous acquisition system for multiple heterogeneous glasses terminals according to an exemplary embodiment.
[0026] Figure 6 This is a schematic diagram illustrating the structure of another cross-physical-layer wireless synchronous acquisition system for multiple heterogeneous glasses terminals, according to an exemplary embodiment.
[0027] Figure 7 This is a schematic diagram illustrating the structure of another cross-physical-layer wireless synchronous acquisition system for multiple heterogeneous glasses terminals, divided by method function, according to an exemplary embodiment. Detailed Implementation
[0028] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.
[0029] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.
[0030] In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0031] The terms "comprising" and "having," and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or devices.
[0032] It should be noted that, in this application, the term "synchronization code" can refer to one or more logical synchronization messages, which can be encapsulated as data packets, control frames, broadcast messages, periodic beacon payloads, or other message formats suitable for wireless distribution. The logical fields of the synchronization code can remain consistent across different wireless physical layers, and its specific encapsulation format can vary depending on the wireless physical layer it carries.
[0033] It should also be noted that, in this application, "glasses terminal" can include smart glasses, AR glasses, MR glasses, video recording glasses, first-person capture glasses, and other wearable terminals with image capture capabilities. "Heterogeneous" includes at least one or more of the following: different manufacturers, different operating systems, different wireless modules, different timestamp capabilities, different sensor types, and different camera pipelines.
[0034] In existing multi-terminal synchronous acquisition methods, traditional industrial camera synchronization relies on a unified trigger line, Genlock signal, or PTP network. Its drawbacks include complex wiring, high cost, and difficulty in adapting to heterogeneous glasses terminals. Consumer-grade software control solutions send "start acquisition" commands via wireless networks, but these are susceptible to wireless latency, system scheduling, and pipeline differences, leading to insufficient frame-level or exposure-level synchronization accuracy. Existing wireless synchronization solutions are mostly designed for a single fixed protocol, lacking the ability to multiplex and bridge across physical layers such as Wi-Fi, Bluetooth, and stroboscopic sensors, making them incompatible with heterogeneous wireless environments. Post-processing timecode alignment solutions are mainly used for post-processing material organization, but they cannot guarantee actual synchronization during the acquisition phase, and the "immediate trigger" logic in dynamic heterogeneous environments struggles to balance synchronization accuracy, scalability, and deployment cost. Based on these problems, this application provides a cross-physical-layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals to address these issues.
[0035] Figure 1 This is a flowchart illustrating a cross-physical-layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals according to an exemplary embodiment.
[0036] Reference Figure 1 As shown in one embodiment of this application, a cross-physical-layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals is provided, including S11 to S16.
[0037] S11, obtain terminal capability information reported by multiple candidate glasses terminals respectively.
[0038] Specifically, the terminal capability information includes wireless bearer capability status, time capability status, sensor acquisition capability status, and feedback capability status. Among them, the time capability status includes time base acquisition capability status or local clock mapping capability status, and timestamp acquisition capability status.
[0039] S12, determine the glasses terminal to be synchronously collected in this round based on the collection task and terminal capability information, establish a synchronization group and determine the synchronization group information, and generate a synchronization group snapshot.
[0040] Specifically, the synchronization group information includes the synchronization group identifier, group version, global reference time within the group, the mapping relationship between the local clock of each glasses terminal and the global reference time within the group, and the synchronization configuration corresponding to the acquisition task.
[0041] Synchronization group snapshots include the synchronization group and synchronization group information.
[0042] S13, generate a synchronization code based on the synchronization group snapshot.
[0043] Specifically, the synchronization code includes the synchronization group identifier, group version, collection round identifier, command version, reference time identifier, target global collection time, effective time window, synchronization configuration, and collection object information.
[0044] The effective time window includes the synchronization code reception deadline and the allowed acquisition time range based on the target global acquisition time.
[0045] The target global acquisition time refers to the planned time set for the target acquisition anchor point in the global reference time domain within the group.
[0046] The synchronization configuration includes synchronization time semantics and multi-sensor synchronization. Synchronization time semantics refers to at least one of frame synchronization, exposure synchronization, or rolling shutter reference line synchronization. Multi-sensor synchronization refers to the combination and alignment synchronization of at least two acquisition objects among the camera, inertial sensor, or audio acquisition.
[0047] S14, based on the available distribution paths of each glasses terminal in the synchronization group, distribute the synchronization code through at least one wireless physical layer.
[0048] Specifically, the wireless physical layer carries Wi-Fi, Bluetooth, or Starlink.
[0049] S15 uses a quadruple consisting of the synchronization group identifier, group version, collection round identifier, and command version to verify and deduplicate the synchronization codes received by each glasses terminal in the synchronization group, and determines the deduplicated synchronization code.
[0050] S16. Based on the deduplicated synchronization code and the currently valid local timing model, the target global acquisition time is converted into the local control time of the glasses terminal in the synchronization group, and the corresponding glasses terminal is triggered to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information.
[0051] Specifically, the currently valid local time series model refers to the local time series model updated in the previous synchronous acquisition round, wherein the local time series model includes at least one of the signed control time series offset parameter, anchor point phase parameter and historical residual parameter.
[0052] The signed control timing offset parameter represents the signed predicted timing offset between the control action moment and the basic acquisition anchor point; the anchor point phase parameter represents the offset from the basic acquisition anchor point to the target acquisition anchor point required by the synchronization configuration; and the historical residual parameter represents the residual compensation determined based on historical feedback information.
[0053] The embodiments described above in this application dynamically establish synchronization groups and generate synchronization group snapshots and synchronization codes by acquiring capability information of heterogeneous glasses terminals. This extracts synchronization semantics from specific wireless physical layers, enabling unified scheduling and distribution based on available distribution paths under at least one wireless physical layer, significantly improving compatibility and scalability in heterogeneous environments. A four-tuple consisting of synchronization group identifier, group version, acquisition round identifier, and command version is used to verify and deduplicate the synchronization codes distributed across multiple paths, effectively eliminating redundant command interference and ensuring trigger reliability. The traditional immediate trigger mode is abandoned; instead, based on the currently effective local timing model, the target global acquisition time is mapped to the local control time of each glasses terminal for scheduled execution. This allows glasses terminals with different hardware characteristics to accurately align acquisition windows under a unified global time reference, significantly improving frame-level and exposure-level synchronization consistency.
[0054] To obtain terminal capability information of heterogeneous terminals, in some specific embodiments of this application, for S11, obtaining the terminal capability information reported by multiple candidate heterogeneous glasses terminals respectively can be achieved by: S111, the coordinating node sends capability detection messages to candidate heterogeneous glasses terminals or receives active registration messages from candidate heterogeneous glasses terminals.
[0055] Specifically, the capability detection message includes the protocol version, data collection task identifier, the capability category to be queried, the registration deadline, a random number, the coordination node identifier, the expected feedback format, and integrity verification information.
[0056] The capability registration message returned by the candidate heterogeneous glasses terminal adopts the following logical structure: terminal_id: GB capability_snapshot_version: 7 wireless: WiFi: Supported Bluetooth: Supported sparklink: UNKNOWN time: local_monotonic_clock: SUPPORTED time_exchange: SUPPORTED capture_timestamp.frame_boundary: SUPPORTED capture_timestamp.exposure_start: NOT_SUPPORTED sensor: camera.resolution: [1920x1080] camera.frame_duration_us:
[33333] camera.manual_exposure: NOT_SUPPORTED imu.gyroscope: SUPPORTED audio.pcm: Supported feedback: link_metrics: SUPPORTED capture_anchor: frame_boundary Here, SUPPORTED, NOT_SUPPORTED, and UNKNOWN represent supported, unsupported, and unknown, respectively. The coordinating node stores the message reception time, capability snapshot version, and validity period. Candidate heterogeneous glasses terminals provide a status for each capability category, and an unknown status does not indicate support or unsupport.
[0057] S112, after receiving the capability probe message sent by the coordinating node, the candidate heterogeneous glasses terminal sends an active registration message to the coordinating node before the registration deadline.
[0058] S113, Based on the active registration message, determine the terminal capability information of multiple candidate heterogeneous glasses terminals.
[0059] Specifically, the terminal capability information includes wireless bearer capability status, time capability status, sensor acquisition capability status, and feedback capability status. Among them, the time capability status includes time base acquisition capability status or local clock mapping capability status, and timestamp acquisition capability status.
[0060] The wireless bearer capability status includes access method, broadcast or unicast support capability, maximum message length, receiver monitoring configuration, energy consumption level, and security capability.
[0061] The timing capability status includes the local monotonic clock, time exchange mechanism, hardware or software timestamp generation capability, mapping update cycle, clock resolution, and supported synchronization reference types, including frame boundaries, exposure start time, or reference line timestamp.
[0062] The sensor acquisition capability status includes the configurable parameter ranges for cameras, inertial sensors, and audio sensors.
[0063] Feedback capability status includes reportable execution status, synchronization anchor point identifier, and link quality indicators.
[0064] The time base acquisition capability status is used to characterize the terminal's ability to acquire a global time base from an external source.
[0065] The local clock mapping capability status is used to characterize the terminal's ability to align and convert the local clock with the global time base.
[0066] The data acquisition timestamp capability status is used to characterize the terminal's ability to mark the time of sensor data acquisition.
[0067] The coordination node in this implementation can be implemented using at least one of the following: mobile phones, tablets, edge computing devices, access points, wearable terminals, independent controllers, or control modules in cloud-edge collaborative systems. The coordination node represents a logical role, and its functions can be distributed across multiple devices.
[0068] The coordinating node can generate a capability profile table for each candidate heterogeneous glasses terminal based on the returned terminal capability information. This table is used to structurally record the differentiated capability parameters of each candidate heterogeneous glasses terminal and serves as the basis for subsequent matching and synchronization configuration, execution timing conversion, and planning of distribution paths.
[0069] The embodiments described above in this application, by sending capability detection messages to heterogeneous glasses terminals and obtaining their returned wireless bearer capability status, time capability status, sensor acquisition capability status, and feedback capability status, can structurally characterize the differences between each glasses terminal in terms of wireless communication, time mapping, acquisition objects, and feedback data. This provides capability basis for subsequent group building, synchronization code generation, timing conversion, and distribution path planning, and helps reduce the risk of acquisition task rejection, configuration failure, or increased synchronization deviation caused by unknown capabilities or capability mismatch.
[0070] In order to construct the synchronization group configuration for this round of synchronous data acquisition based on the capability differences of heterogeneous glasses terminals, in some specific embodiments of this application, for S12, the glasses terminal for this round of synchronous data acquisition is determined according to the acquisition task and terminal capability information, a synchronization group is established and the synchronization group information is determined, and a synchronization group snapshot is generated, which can be implemented as S121 to S125.
[0071] S121. Based on the data collection task identifier, the list of candidate terminals corresponding to the data collection task, the terminal selection criteria corresponding to the data collection task, and the registration deadline, determine the candidate glasses terminals.
[0072] Specifically, eyeglass terminals that correspond to the data collection task and receive an active registration message before the registration deadline are selected as candidate eyeglass terminals.
[0073] The data collection task identifier, the list of candidate terminals corresponding to the data collection task, and the terminal selection criteria for the data collection task vary depending on the data collection task.
[0074] Specifically, after the registration deadline arrives, the coordination node checks the multiple candidate heterogeneous glasses terminals in sequence: whether the data collection task identifier carried in the registration message of the glasses terminal is consistent with the current data collection task identifier; whether the glasses terminal is in the candidate terminal list corresponding to the data collection task; whether it meets the terminal selection conditions corresponding to the data collection task, such as location, identity, role, and reachability; and whether the glasses terminal confirmed its participation before the registration deadline.
[0075] For example, suppose the current data collection task is identified as task_001, and the candidate terminal list consists of terminal A, terminal B, and terminal C. The terminal selection criteria are that the terminal is located in a preset data collection area and is reachable from the coordination node, and the registration deadline is 10:00:05 local time of the coordination node. Terminals A and B carry task_001, register before the registration deadline, and meet the above conditions; terminal C carries other data collection task identifiers. Therefore, terminals A and B are identified as candidate glasses terminals, while terminal C is not identified as a candidate glasses terminal.
[0076] S122, Based on the terminal capability requirements corresponding to the data collection task and the terminal capability information of the candidate glasses terminals, screen the candidate glasses terminals and determine the glasses terminals to be included in the group.
[0077] Specifically, candidate glasses terminals that meet the mandatory terminal capability requirements corresponding to the data acquisition task and whose capability snapshots are valid are selected. The candidate glasses terminals are then checked to see if they have completed time mapping, sensor configuration resource reservation, and link readiness checks. Glasses terminals that pass the checks are included in the group.
[0078] For example, continuing from the previous example, suppose the terminal capability requirement corresponding to the acquisition task_001 is that the glasses terminal supports camera acquisition, frame boundary timestamps, and local clock mapping. In the capability snapshot of terminal A, all of the above capabilities are supported, while the frame boundary timestamp capability status of terminal B is unknown. Therefore, terminal A is identified as the glasses terminal to be included in the group, while terminal B is not.
[0079] S123, based on the participation confirmation and readiness status of the glasses terminals that have joined the group, determine the glasses terminals to be used for synchronous data collection in this round.
[0080] Specifically, based on the above steps S121 to S123, candidate glasses terminals that do not meet the above screening conditions are rejected from the group, removed from the current round of synchronous collection, or downgraded in a manner permitted by the task. A new group version is generated each time the members change.
[0081] Among these, downgrading according to the methods allowed by the task may include, but is not limited to, omitting optional data collection objects that are not supported by the glasses terminal.
[0082] For example, if the data acquisition task sets the camera as a mandatory acquisition target and the inertial sensor as an optional acquisition target, and the glasses terminal supports the camera but does not support the inertial sensor, the inertial sensor acquisition can be omitted and the data will be included in the group if the task allows; if the glasses terminal does not support mandatory camera acquisition, the data will not be included in the group.
[0083] S124, Establish a synchronization group based on the glasses terminal collected in this round of synchronous acquisition, and determine the synchronization group information.
[0084] Specifically, the coordinating node generates a synchronization group identifier (group_id) and a group version (group_version) based on the glasses terminals collected in this round of synchronization, and freezes the group members, the glasses terminal capability snapshot version, and the task version.
[0085] Subsequent additions, departures, or removals of members will take effect from the next set of versions, without affecting the already frozen versions.
[0086] Synchronization group information includes synchronization group identifier, group version, global reference time within the group, mapping relationship between the local clock of each glasses terminal and the global reference time within the group, and synchronization configuration corresponding to the acquisition task.
[0087] The intra-group global reference time is a logical time domain shared by the synchronization group, and the mapping relationship is jointly represented by the mapping model and mapping records. The mapping model is expressed by the formula described later. The mapping records are associated with the glasses terminal identifier and the reference time identifier reference_time_id, and record the mapping parameters ai and bi, the estimated time, the validity period, the uncertainty, and the sample range. The estimated time represents the intra-group global reference time when the mapping parameter estimation is completed, and the sample range represents the range of time-exchange samples or common beacon observation samples used in this estimation. The above record items are not substituted into the mapping formula. When the mapping expires or the uncertainty exceeds the task threshold, the glasses terminal re-establishes the mapping, requests a downgrade, or rejects the current round of tasks.
[0088] The time is provided by the coordinating node's monotonic clock, an external time source, or a glasses terminal clock elected based on clock stability, reachability, and continuous online capability, and is distinguished by a reference time identifier. For example, the coordinating node assigns a reference time identifier (reference_time_id) to the selected time domain and publishes the time source type, epoch definition, time unit, and validity period to group members.
[0089] Each eyeglass terminal within the synchronization group establishes a mapping between its local clock and the group's global reference time through time exchange, common beacon observation, or other means.
[0090] The mapping relationship between the local clock of each glasses terminal and the global reference time within the group is as follows:
[0091] in, Indicates the global reference time within the group. Indicates the relative frequency ratio. This represents the local clock of the i-th glasses terminal. This represents the offset parameter.
[0092] Two sets of time-corresponding samples were used. and At that time, it can be done according to Determine the mapping parameters.
[0093] When using multiple sets of samples, linear fitting can be used to determine... and .
[0094] The synchronization configuration corresponding to the above acquisition task can also be determined according to the terminal capability information and the acquisition task, and may include synchronization time semantics and multi-sensor synchronization. Synchronization time semantics means at least one of frame synchronization, exposure synchronization or rolling shutter reference line synchronization, and multi-sensor synchronization means the combination and alignment synchronization of at least two acquisition objects among camera, inertial sensor or audio acquisition.
[0095] Among them, frame synchronization uses the specified frame boundary as the target acquisition anchor point; exposure synchronization uses the exposure start, exposure center, or task-specified exposure anchor point as the target; and rolling shutter reference line synchronization uses the specified reference line or reference area as the target time.
[0096] Multi-sensor synchronization defines a combination of acquisition objects, such as the association between a camera frame and the nearest inertial sample or a specified audio sample interval, rather than treating "multi-sensor" as a single time level higher than exposure synchronization.
[0097] The synchronization configuration for the data acquisition task can be either a group-wide shared configuration or a hierarchical configuration. In the group-wide shared configuration, time semantics and mandatory objects that can be satisfied by all members are selected. In the hierarchical configuration, the entire group adopts a common synchronization time semantics that can be satisfied by all members, and additional synchronization time semantics or additional multi-sensor synchronization are applied to the more capable subset of terminals, such as configuring exposure or reference line anchor points. Feedback from different layers is calculated using their respective anchor points, and timestamps with different semantics are not directly compared.
[0098] Each timestamp, target time, and feedback information is associated with a reference time identifier and its corresponding collection anchor semantics.
[0099] S125, after the registration deadline arrives, freeze the current round member set consisting of the glasses terminals collected in this round of synchronous acquisition, and record the current round member set and synchronization group information as a synchronization group snapshot associated with the group version and unchanged during this round of synchronous acquisition.
[0100] Specifically, a synchronization group snapshot includes the synchronization group and synchronization group information, where the synchronization group is the set of members in this round.
[0101] The embodiments described above in this application, by comprehensively evaluating the terminal capability information of the glasses terminals, accurately select and form synchronization groups that meet the requirements of the data collection task, ensuring a high degree of coordination between the glasses terminals within the group in terms of time base and clock mapping; at the same time, by freezing the member set after the registration deadline and generating an immutable snapshot of the associated group version, the consistency and stability of the configuration during this round of synchronous data collection are effectively guaranteed, avoiding timing chaos caused by dynamic changes, thereby significantly improving the data synchronization accuracy, system reliability and task execution efficiency of multi-terminal collaborative data collection.
[0102] Figure 2 This is a schematic diagram illustrating a process for cross-physical layer synchronization code distribution and bridging according to an exemplary embodiment.
[0103] Reference Figure 2 As shown in some specific embodiments of this application, before generating the synchronization code, the following can be used: The coordinating node reads the frozen synchronization group snapshot, acquisition task version, time mapping status, synchronization configuration, routing plan, and network status of candidate distribution paths to determine the future target global acquisition time with sufficient lead time. Then generate the synchronization code.
[0104] The network status includes at least one of the following: one-way or round-trip delay of candidate paths, jitter, queue length, packet loss rate or delivery rate, number of retries, signal quality, available bandwidth, congestion level, sleep-wake cycle, and bridging processing time.
[0105] The allowance should at least cover the time required for message generation and transmission, estimated path delivery, bridging processing, terminal wake-up, configuration, and entering the ready state, plus a margin appropriate to the measurement uncertainty. When link fluctuations increase, a more distant target should be selected for global data acquisition. Add backup bearers or reject paths that cannot arrive before the reception deadline.
[0106] In some specific implementations of this application, the core fields of the synchronization code include: synchronization group identifier group_id, group version group_version, collection round identifier round_id, command version command_version, reference time identifier reference_time_id, target global collection time target_global_time, valid time window valid_window, synchronization configuration sync_profile, and collection object information sensor_assignment.
[0107] Specifically, the core fields of the synchronization code, the meaning of each field, and the basis for assigning values are shown in Table 1: Table 1: Core fields of the synchronization code, the meaning of each field, and the basis for assigning values.
[0108]
[0109] The coordinating node selects a future target global acquisition time with sufficient lead time for each round of data acquisition based on the group's global reference time, the one-way or round-trip delay of candidate paths, jitter, queue length, packet loss or delivery rate, number of retries, signal quality, available bandwidth, congestion level, sleep / wake-up cycle, and bridging processing time, as well as frozen terminal capability information. This lead time covers the time required for message generation and sending, estimated path delivery, bridging processing, terminal wake-up, configuration, and entering the ready state, and is supplemented with a margin appropriate to the measurement uncertainty. The target global acquisition time is a future moment, not a moment that "starts immediately."
[0110] The acquisition allowable window is determined by the acquisition task's tolerance for timing errors, the observable resolution of the acquisition anchor point, the mapping uncertainty, and the terminal's capabilities. The synchronization code reception deadline is determined by subtracting the time required for terminal wake-up, configuration, preparation, and control conversion processing, along with corresponding margins, from the target global acquisition time. The synchronization code reception deadline and the acquisition allowable window are assigned values separately.
[0111] The information on the objects to be collected is determined by the collection requirements of the task initiator and the frozen capability snapshot, and must not exceed the capability range of the corresponding glasses terminal. If a required object is not supported, the coordinating node will refuse to include the glasses terminal in the group, remove the glasses terminal, or reduce its configuration if the task allows; optional objects will be omitted if they are not supported.
[0112] The aforementioned core fields are used together to determine the synchronous acquisition task, freeze the synchronous group, acquisition round, target acquisition time under the global reference time within the group, acquisition allowed conditions, and acquisition objects and parameters for each terminal.
[0113] For synchronization codes with the same synchronization group identifier, group version, collection round identifier, and command version, the synchronization semantics of this core field remain unchanged.
[0114] The synchronization code also includes at least one of the following: bearer strategy, feedback deadline, rollback strategy, and integrity verification information.
[0115] Specifically, the bearer strategy refers to the distribution method, primary bearer, and backup bearer when they exist for each glasses terminal.
[0116] The feedback deadline is the latest time to obtain feedback information from the glasses terminals in the synchronization group when they perform synchronous data collection.
[0117] The rollback strategy indicates the abnormal conditions that trigger rollback, the number of consecutive abnormal collection rounds, and the allowed rollback processing methods. Rollback processing methods include retransmitting synchronization codes that have not yet expired, switching wireless physical layer bearers, reducing the synchronization time semantics of subsequent collection rounds, reducing the number of optional collection objects, generating a new set of versions, or removing abnormal glasses terminals from the next set of versions.
[0118] Integrity verification information represents verification data used by the receiving end to determine whether the agreed fields in the synchronization code are consistent before and after transmission.
[0119] Optional fields for the synchronization code may also include path identifier, message sequence number, and authentication information.
[0120] Reference Figure 2 As shown, in order to implement multiple wireless physical layer bearer distribution synchronization codes, in some specific embodiments of this application, for S14, according to the available distribution paths of each glasses terminal in the synchronization group, the synchronization code is distributed through at least one wireless physical layer bearer, which can be adopted as: S141 to S148.
[0121] S141, determine the first candidate distribution path based on the available distribution paths from the coordinating node to each glasses terminal in the synchronization group and the commonly supported wireless physical bearers and / or the available distribution paths from the coordinating node to each glasses terminal in the synchronization group via the bridging node and the commonly supported wireless physical bearers.
[0122] Specifically, each link segment of the first candidate distribution path is carried by a wireless physical layer supported by the nodes at both ends of the link segment.
[0123] For example, there is a first candidate distribution path for the bridging node. The bridging node selects a bearer from the wireless physical layer bearers jointly supported by the coordinating node and the bridging node, and selects a bearer from the wireless physical layer bearers jointly supported by the bridging node and the glasses terminal. The bridging node keeps the synchronization code core field unchanged and appends or updates the path metadata.
[0124] S142, based on the path being reachable and the expected delivery time not later than the synchronization code reception deadline, the first candidate distribution path is filtered to determine the second candidate distribution path.
[0125] Specifically, the second candidate distribution paths are all reachable paths whose expected delivery time is no later than the synchronization code reception deadline.
[0126] Based on the above steps S141 to S142, a reachability matrix of "node-path-bearer" is formed based on the coordinating node, bridging node, and wireless physical layer bearer. Each matrix item records the reachability status, estimated delivery time, latency, jitter, packet loss rate, delivery rate, reliability, energy consumption, congestion, bridging hop count, and the number of terminals supporting wireless physical layer bearer.
[0127] Path selection is based on the hard constraint of covering all members in the current round of data collection and delivering the synchronization code before the deadline. For paths that meet the hard constraint, they are then scored according to the weights set by the task to select the primary and backup distribution paths. The number of terminals supported by a certain wireless physical layer is only considered as a coverage or cost factor and does not replace the judgment of terminal-by-terminal reachability and deadline. Primary and backup distribution path selection is based on the granularity of terminals or path segments.
[0128] S143, for each glasses terminal, based on at least one of the following factors—latency, jitter, packet loss rate, delivery rate, reliability, power consumption, congestion level, number of bridging hops, and the number of terminals supporting wireless physical layer transmission—the second candidate distribution path is comprehensively evaluated according to the weights set in the task, and the comprehensive evaluation result is determined.
[0129] S144, for each glasses terminal, based on the comprehensive evaluation results, the optimal second candidate distribution path is determined as the main distribution path, and the wireless physical layer bearer used in the main distribution path is determined as the main bearer corresponding to the main distribution path.
[0130] Specifically, the wireless physical layer carries Wi-Fi, Bluetooth, or Starlink.
[0131] S145, for each glasses terminal, when there are multiple second candidate distribution paths, the second best second candidate distribution path is determined as the backup distribution path based on the comprehensive evaluation results, and the wireless physical layer bearer used in the backup distribution path is determined as the backup bearer corresponding to the backup distribution path.
[0132] Based on the above steps S143 to S145, for example, assuming that the second candidate distribution paths P1, P2, and P3 for a certain glasses terminal all meet the reachability and synchronization code reception deadline requirements, only the delay in milliseconds is used. and shaking The tasks were evaluated with weights of 0.7 and 0.3 respectively. If the (L, J) values for P1, P2, and P3 are (18, 4), (22, 2), and (30, 3) respectively, then according to... The calculated evaluation values were 13.8, 16.0 and 21.9 respectively; the smaller the evaluation value, the better the result. Therefore, P1 was determined as the primary distribution path and P2 was determined as the backup distribution path.
[0133] In some specific embodiments of this application, if there is only one second candidate distribution path, only the primary distribution path is determined.
[0134] S146, For each glasses terminal, determine the distribution method based on the main distribution path and its corresponding main bearer, backup distribution path and its corresponding backup bearer.
[0135] Specifically, the distribution methods include at least one of single-bearer distribution, multi-bearer redundant distribution, and bridging distribution.
[0136] Single-bearer distribution means that the coordinating node distributes the synchronization code to the glasses terminals in the synchronization group through a single wireless bearer. For example, the synchronization code is distributed to the glasses terminals in the synchronization group via Wi-Fi.
[0137] Multi-bearer redundancy distribution means that the coordinating node transmits logically equivalent synchronization code encapsulations through two or more wireless physical layer bearers.
[0138] For example, the master synchronization code is distributed via Wi-Fi, while a backup synchronization code containing the same synchronization group identifier, group version, collection round identifier, and command version is sent via Bluetooth to improve reliability.
[0139] Bridging distribution means that the coordinating node sends the synchronization code to the bridging node via a wireless physical layer bearer, and the bridging node then converts it to another wireless physical layer bearer and sends the synchronization code to the target glasses terminal. For example, the coordinating node sends the synchronization code to the bridging node via Wi-Fi, and the bridging node then sends the synchronization code to the corresponding glasses terminal via Bluetooth or a satellite link.
[0140] The bridging node only changes the wireless physical layer bearer encapsulation and records the ingress path, egress path, reception time, forwarding time, and bridging processing time, without changing the synchronization semantics of group_id, group_version, round_id, command_version, reference_time_id, target_global_time, valid_window, sync_profile, and sensor_assignment.
[0141] In the bridging distribution scenario, the bridging node does not need to change the core synchronization semantics of the synchronization code. Its main function is to keep the synchronization semantics of core fields such as synchronization group identifier, group version, collection round identifier, command version, reference time identifier, target global collection time, effective time window, synchronization configuration and collection object information unchanged, and adjust the encapsulation form and transmission rhythm according to the differences in wireless bearer.
[0142] In this embodiment, if there is only one wireless physical layer bearer, single bearer distribution can be used; if there are two or more wireless physical layer bearers, redundant distribution of the same logical synchronization code can be used, and multi-bearer redundant distribution can be used; if there is a distribution path with bridging nodes, bridging distribution can be used; the distribution methods determined for different glasses terminals can be combined and applied.
[0143] Through the aforementioned cross-physical layer distribution method, this application can cover complex scenarios where multiple different glasses terminals have different wireless modules, different power consumption strategies, and different bearer access methods.
[0144] S147 generates a logical synchronization code containing the synchronization group identifier, group version, collection round identifier, and command version.
[0145] S148, based on the primary distribution path and the backup distribution path, encapsulates the logical synchronization code into one or more transmission copies, and distributes the one or more transmission copies through the corresponding wireless physical layer bearer according to the distribution method.
[0146] Specifically, according to the wireless physical layer bearers used in the primary distribution path and the backup distribution path, the logical synchronization code is encapsulated into the corresponding transmission copy of the wireless physical layer bearer.
[0147] Different wireless physical layers carry additional encapsulated metadata such as fragmentation, reassembly, routing, and transmission time, but do not change the synchronization semantics of the core fields of the logical synchronization code.
[0148] The embodiments described above in this application evaluate at least one of the following indicators of available distribution paths: jitter, packet loss rate, delivery rate, reliability, energy consumption, congestion level, bridging hop count, and the number of terminals supporting wireless physical layer bearers. This intelligently selects primary and backup distribution paths and their corresponding wireless physical layer bearers, and flexibly determines distribution methods such as single bearer, multi-bearer redundancy, or bridging accordingly. This achieves dynamic adaptive optimization of the synchronization code transmission path. Combined with the multi-copy encapsulation and differentiated distribution mechanism of the logical synchronization code, it effectively improves the delivery rate, timeliness, and anti-interference capability of the synchronization code in complex wireless environments, and significantly enhances the robustness, compatibility, and overall synchronization accuracy of multi-terminal synchronous acquisition.
[0149] In some specific embodiments of this application, when a glasses terminal that simultaneously supports the primary bearer of the primary distribution path and the backup bearer of the backup distribution path receives multiple synchronization code encapsulations, it can first verify the integrity, group version, command applicable object, reference time identifier and validity period, and then perform deduplication.
[0150] Reference Figure 2 As shown, based on the above steps S13 to S14, the coordinating node generates a unified synchronization code (including synchronization group identifier and group version, collection round identifier and command version, reference time identifier and target time, reception deadline and collection allowed window, synchronization configuration sync_profile, collection object sensor_assignment, bearer strategy and feedback deadline). On the one hand, it triggers redundant transmission (primary bearer and backup bearer in parallel), and on the other hand, it encapsulates and transmits in multiple paths. It is encapsulated via Wi-Fi (primary bearer or high-throughput bearer) to connect to glasses terminal A (supporting Wi-Fi and timestamp readback), encapsulated via Bluetooth (low-power control bearer) to connect to glasses terminal B (supporting Bluetooth and frame-level timestamp), and encapsulated via Star Flash (task adaptation bearer) to connect to glasses terminal C (supporting Star Flash and multi-sensor acquisition). Another distribution path is connected to glasses terminal C (supporting Star Flash and multi-sensor acquisition) through cross-bearer equivalent forwarding via bridging node. It drives glasses terminal A, glasses terminal B, and glasses terminal C respectively, forming a communication distribution link that adapts to multiple types of glasses terminals according to different bearer methods.
[0151] To prevent duplicate or invalid synchronization commands from interfering with the acquisition timing, in some specific embodiments of this application, for S15, the synchronization codes received by each glasses terminal in the synchronization group are verified and deduplicated using a quadruple consisting of the synchronization group identifier, group version, acquisition round identifier, and command version, and the deduplicated synchronization code is determined. This can be done using S151 to S152.
[0152] S151 identifies synchronization codes with the same synchronization group identifier, group version, acquisition round identifier, and command version as transmission copies of the same logical synchronization code.
[0153] S152, select one of the transmitted copies of the same logical synchronization code as the deduplicated synchronization code.
[0154] Specifically, when the glasses terminal receives a synchronization code that arrives repeatedly via the main distribution path and its corresponding main bearer, backup distribution path and its corresponding backup bearer, it executes the synchronization code corresponding to the same synchronization group identifier, group version, collection round identifier and command version only once. That is, the deduplicated synchronization code is used as the only verified transmission copy to enter the scheduled execution process, and the synchronization collection command corresponding to the logical synchronization code is executed only once in the subsequent process. The duplicate logical synchronization code information is only used for link statistics or to return a received confirmation, and is not configured and executed repeatedly.
[0155] The embodiments described above in this application accurately identify multiple transmission copies of the same logical synchronization code using a quadruple and retain only one as a valid synchronization instruction. This effectively avoids redundant processing and resource waste caused by multi-path redundant distribution, while ensuring the uniqueness and consistency of the synchronization instruction, thus improving the response efficiency and operational stability of the glasses terminal.
[0156] In some specific embodiments of this application, the feedback information includes execution feedback information and link feedback information.
[0157] Figure 3 This is a schematic diagram illustrating a local clock mapping and scheduled execution process according to an exemplary embodiment.
[0158] Reference Figure 3 As shown, in some specific embodiments of this application, for S16, the target global acquisition time is converted into the local control time of the glasses terminal in the synchronization group according to the deduplicated synchronization code and the currently valid local timing model, and the corresponding glasses terminal is triggered to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information. This can be achieved using S161 to S164.
[0159] S161, based on the mapping relationship between the target global acquisition time in the deduplicated synchronization code and the local clock of each glasses terminal in the synchronization group information and the global reference time within the group, the target global acquisition time is converted into the local reference time of the glasses terminal in the synchronization group.
[0160] S162, using the currently effective local timing model to compensate for the local reference time, and determine the local control time of the glasses terminal in the synchronization group.
[0161] Based on the above steps S161 to S162, for example, for the first... Each glasses terminal, with This indicates an effective mapping between the local clock of the glasses terminal and the global reference time within the group, in order to Indicates the global acquisition time of the target, and the local reference time is... .
[0162] The local timing model includes signed control timing offset parameters. Anchor point phase parameters and historical residual parameters ,in, This represents the signed prediction timing offset between the control action moment and the basic acquisition anchor point. This indicates the offset from the base acquisition anchor point to the target acquisition anchor point required by the synchronization configuration. This indicates residual compensation determined based on historical feedback information; Local control time according to Sure.
[0163] The signed prediction time offset can be positive, zero, or negative to accommodate observable pipelines where the base acquisition anchor point is located after, coincident with, or before the control action time; its sign must be consistent with the actual value. The calculation conventions remain consistent.
[0164] If the target acquisition anchor point is used for estimation The basic data collection anchor point, then Zero. Configuration or preparation time is earlier than... The lead time is determined separately based on the sensor configuration effective time, cache establishment time, and resource reservation time.
[0165] In the initial compensation process, manufacturer- or terminal-preset values, pre-calibrated values before deployment, or calibration rounds not included in the official data can be used. When a compensable local timing model is lacking, the glasses terminal may reduce synchronization time semantics or refuse tasks.
[0166] S163, the glasses terminal in the control synchronization group configures the information of the object to be collected before the local control time.
[0167] Specifically, the configuration of the acquired object information includes at least one of the following: camera configuration, inertial sensor configuration, and audio configuration.
[0168] The camera configuration includes at least one of the following: acquisition mode, resolution, frame duration or frame rate, exposure parameters, ISO, automatic or manual exposure mode, shutter type, and rolling shutter reference line or reference area. Inertial sensor configuration includes at least one of the following: sensor type, sampling rate, duration or number of samples, batch processing, and timestamp method; Audio configuration includes at least one of the following: sampling rate, number of channels, target start time, duration, sample format, and buffer prefilling method.
[0169] The glasses terminal checks each parameter to see if it is within the capability snapshot range; if a required object or parameter is not supported, it returns REJECTED; if an optional object is omitted, it returns the actual configuration used; if the configuration is successful and the mapping is still valid, it returns ARMED.
[0170] S164, when the local control time arrives, control the corresponding glasses terminal to perform synchronous data acquisition, and obtain the acquired data, execution feedback information and link feedback information.
[0171] Specifically, the glasses terminal completes the configuration of the target information and enters the ready state before the local control time. At the local control time, it issues a control or trigger action and forms a window judgment result that is recorded separately from the execution state based on whether the target acquisition anchor point falls into the acquisition permission window.
[0172] When the local control time arrives, a control or trigger action is issued, and then the acquisition pipeline reaches the acquisition state at the predicted basic acquisition anchor point, and the target acquisition anchor point is brought as close as possible. The glasses terminal records the local control time, the available timestamps of the basic acquisition anchor point and the target acquisition anchor point, converts or associates them with reference_time_id, and records the execution status of whether the acquisition is completed and the window determination result of whether the target acquisition anchor point falls within the acquisition allowed window.
[0173] Scheduled execution does not imply that the glasses terminal has hardware triggering or hardware Genlock; the glasses terminal is still limited by operating system scheduling, sensor driving, and timestamp observability.
[0174] Referring to Table 2, the execution feedback information includes at least one execution status among ARMED, SUCCESS, PARTIAL, MISSED_WINDOW, REJECTED, and FAILED, and includes at least one of the following: actual acquisition timestamp, reference time identifier, acquisition anchor semantics, mapping validity and mapping uncertainty, actual configuration, and error reason.
[0175] Table 2: Execution Status and its Judgment Meaning.
[0176]
[0177] The execution status described above indicates whether the data acquisition process is complete. For samples with available target acquisition anchors, the coordinating node further records whether the target acquisition anchor falls within the allowed acquisition window and determines whether it falls within or exceeds the window. SUCCESS samples that exceed the window can still be used to estimate signed acquisition bias and correct subsequent rounds if the mapping is valid, the anchor semantics are consistent, and they are not identified as outliers. The execution status and window determination are used together for feedback filtering, link diagnosis, timing correction, and rollback processing.
[0178] Link feedback information includes at least one of the following: distribution path, wireless physical layer bearer, synchronization code reception time, one-way delay or round-trip delay, jitter, packet loss rate or delivery rate, number of retries, signal quality, ingress path, egress path, and bridging processing time.
[0179] The embodiments described above in this application achieve high-precision synchronous acquisition of multiple glasses terminals under a unified time reference by accurately converting the target global acquisition time into the local control time and combining it with the currently effective local timing model compensation. At the same time, the acquisition parameters such as camera, inertial sensor and audio are pre-configured before triggering acquisition to ensure that the device readiness state is strictly aligned with the acquisition time, which effectively improves the time consistency, acquisition reliability and system response efficiency of multimodal data, and provides a complete execution and feedback basis for subsequent timing calibration and link quality assessment.
[0180] Figure 4 This is a schematic diagram illustrating a feedback correction process based on execution feedback information and link feedback information according to an exemplary embodiment.
[0181] Reference Figure 4 As shown in some specific embodiments of this application, a cross-physical layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals can also employ S17.
[0182] S17. Based on the feedback information, determine the acquisition deviation information and update the currently valid local time series model.
[0183] Specifically, the local timing model includes at least one of the following: a signed control timing offset parameter, an anchor point phase parameter, and a historical residual parameter. The signed control timing offset parameter represents the signed predicted timing offset between the control action moment and the basic acquisition anchor point. The anchor point phase parameter represents the offset from the basic acquisition anchor point to the target acquisition anchor point required by the synchronization configuration. The historical residual parameter represents the residual compensation determined based on historical feedback information.
[0184] In this embodiment, only feedback samples with an execution status of SUCCESS, valid mapping relationship, mapping uncertainty that meets the threshold of this round, and whose collection anchor semantics are consistent with the synchronization configuration are used for time series parameter updates.
[0185] The window determination result serves as the basis for deviation magnitude, rollback, and anomaly filtering, but does not replace the execution status. For example, after converting the actual target acquisition anchor point of the glasses terminal to the same reference time identifier, according to Calculate the acquisition bias, where, This represents the signed acquisition deviation of the actual target acquisition anchor point of the i-th glasses terminal relative to the global acquisition time of the target. This indicates the time after the actual target acquisition anchor point has been converted to the time field corresponding to reference_time_id. The value represents the global acquisition time of the target, and i represents the serial number of the glasses terminal.
[0186] A value greater than zero indicates that the target acquisition anchor point was acquired too late. A value less than zero indicates that the target acquisition anchor point is too early. Timestamps of different acquisition anchor point semantics are not directly subtracted. PARTIAL samples are not included in the formal time-series update candidate set configured for synchronization across the entire group; among them, those that meet the mandatory acquisition anchor point conditions specified separately by the task can only enter the independent update of the corresponding acquisition object, and the remaining samples do not update the time-series parameters.
[0187] Specifically, clock mapping parameters are updated using time exchange or common beacon samples respectively; signed control timing offset parameters are updated using independent time difference samples between the control action moment and the basic acquisition anchor point; anchor point phase parameters are updated using independent time difference samples between the basic acquisition anchor point and the target acquisition anchor point within the same acquisition event; and historical residual parameters are updated using stable historical residuals after the first three types of models are effective.
[0188] Among them, the signed control timing offset parameters are determined as follows: ; , .
[0189] in, Indicates signed acquisition deviation The equivalent deviation after conversion to the local clock domain of the i-th glasses terminal This represents the inverse mapping from the group's global reference time to the local clock of the i-th glasses terminal. Indicates the global acquisition time of the target. This represents the signed acquisition bias of the i-th glasses terminal. This represents the updated signed control timing offset parameters. This represents the signed control timing offset parameters before the update. Represents the smoothing coefficient. In linear mapping Down, ,in, This represents the relative frequency ratio of the local clock of the i-th glasses terminal to the global reference time within the group.
[0190] When the above-mentioned marked acquisition deviation When the value is positive and the source of error is determined to be signed control timing offset, increase the corresponding value. When the above-mentioned signed acquisition deviation When the error is negative and the source of error is determined to be signed control timing offset, the corresponding value is reduced. .
[0191] Phase estimation of independently obtained new anchor points ,calculate: , .
[0192] in, This indicates the updated anchor point phase parameters. Represents the smoothing coefficient. This indicates the anchor point phase parameters before the update. This represents the independently obtained phase estimate of the new anchor point.
[0193] The corresponding parameters are updated only when the source of the error is identifiable; otherwise, the original parameters are retained and the uncertainty is increased or calibration is triggered.
[0194] The above formula is used to account for the acquisition deviation. Equivalent offset converted to local clock domain and utilize Update the signed control timing offset parameters used in subsequent acquisition rounds. This process does not modify the local clock itself.
[0195] The embodiments described above in this application, by rigorously screening feedback information and determining effective feedback samples, and by decoupling and updating four types of parameters—clock mapping parameters, signed control timing offset parameters, anchor point phase parameters, and historical residuals—in a layered manner, achieve accurate modeling and dynamic compensation of multi-dimensional timing deviations during cross-physical layer synchronization of heterogeneous glasses terminals. This significantly improves the time alignment accuracy, robustness, and adaptability to heterogeneous hardware differences in multi-terminal acquisition.
[0196] In some specific embodiments of this application, when a glasses terminal in the synchronization group experiences one or more of the following situations: refusing to collect data, failing to receive the synchronization code before the synchronization code reception deadline, failing to form a usable target data collection anchor point, the target data collection anchor point exceeding the allowed data collection window, or abnormal execution feedback information in multiple consecutive data collection rounds participated in by the same glasses terminal, a rollback strategy is adopted to roll back the glasses terminal.
[0197] Specifically, abnormal execution feedback information in multiple consecutive collection rounds participated in by the same glasses terminal includes at least one of the following: not receiving execution feedback information before the feedback deadline, execution feedback information indicating invalid local clock mapping relationship, collection anchor point semantics in execution feedback information inconsistent with synchronization configuration, and execution feedback information indicating execution status as FAILED.
[0198] In this case, multiple anomalies occurring in the same data collection round are counted as only one abnormal data collection round.
[0199] When the same glasses terminal experiences execution feedback anomalies in multiple consecutive collection rounds, the number of collection rounds is set by the backoff strategy in the synchronization code, and this number of collection rounds is an integer not less than 2.
[0200] If a data collection cycle without any abnormalities occurs, the continuous count is reset to zero.
[0201] Rollback processing may include at least one of the following: retransmitting a synchronization code that has not yet expired; switching the wireless physical layer bearer; reducing the synchronization time semantics and updating the command version for subsequent collection rounds; reducing the number of optional collection targets and updating the command version for subsequent collection rounds; generating a new set of versions; or removing abnormal glasses terminals from the next set of versions. The reduction of optional collection targets for subsequent collection rounds is determined based on the collection task and terminal capability information.
[0202] The currently frozen group version and the synchronization semantics of the current collection round remain unchanged. The rolled-back glasses terminal will take effect from the next collection round, only affecting subsequent collection rounds, and will take effect through the update command version.
[0203] The embodiments described above in this application, through a well-defined anomaly detection and continuous counting mechanism, combined with multiple hierarchical rollback strategies, enable precise and non-destructive dynamic recovery or isolation of abnormal glasses terminals while maintaining the semantic stability of the current acquisition round synchronization. This effectively improves the fault tolerance, operational continuity, and adaptive robustness to heterogeneous terminal anomalies in the multi-terminal synchronous acquisition process.
[0204] The method provided in this application enables multiple different glasses terminals to complete unified synchronous data acquisition in a heterogeneous wireless environment by breaking down the synchronization logic into several stages: "capability detection, time mapping, synchronization code distribution, local reservation execution, and feedback correction".
[0205] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.
[0206] The following examples and comparative examples will be used to further illustrate this application in order to better understand the above-mentioned technical solutions. It should be understood that the following are only some examples and are not intended to limit this application.
[0207] The following are illustrative calculation examples; the numerical values are only used to demonstrate the symbols, calculation direction, and parameter classification.
[0208] In a frozen group consisting of terminals A, B, and C, the monotonic clock of the coordinating node is used as the global reference time source within the group and is assigned reference_time_id=R1. The current global reference time within the group is 100.000 s. The coordinating node selects a future target global acquisition time T*=105.000 s, with an acquisition allowable window of ±0.010 s, and the synchronization code reception deadline is set to 104.900 s.
[0209] In this embodiment, terminal A supports Wi-Fi, frame boundaries, and exposure timestamps; terminal B supports Bluetooth and frame boundary timestamps; and terminal C supports star flash, frame boundary timestamps, rolling shutter reference line parameters, and an inertial sensor.
[0210] The data acquisition task requires a camera as a mandatory component and allows the use of the inertial sensor on terminal C as an optional component. The common temporal semantics for the entire group are frame synchronization, and an additional reference line synchronization layer is configured for terminal C. Camera acquisition parameters are set to 1920×1080 resolution and frame duration of 33.333 ms. The sampling rate of the inertial sensor on terminal C is set to 200 Hz. All these values are based on the premise that the corresponding terminal capabilities are supported.
[0211] The routing plan selects Wi-Fi as the primary bearer from the coordinating node to terminal A and the bridging node, and Bluetooth as the distribution bearer for terminal B and the redundant backup bearer for terminal A. The bridging node then forwards the data to terminal C via StarScan. The core synchronization semantics of each bearer encapsulation are identical; even if terminal A receives two encapsulations via Wi-Fi and Bluetooth, it will only execute them once.
[0212] To simplify calculations, the mapping between the three local clocks and the group's global reference time is as follows: , , .
[0213] in, This represents the local clock reading of terminal A. This indicates the local clock reading of terminal B. This represents the local clock reading of terminal C. This indicates the global reference time within the group.
[0214] Therefore, the local reference times corresponding to the target global acquisition time are respectively , , Where RA, RB, and RC are the values of tA, tB, and tC when Tg=T*, respectively; tA, tB, and tC represent general local clock variables, and RA, RB, and RC represent the specific local reference time corresponding to the target global acquisition time.
[0215] Let the signed control timing offsets be respectively , , The target acquisition anchor points for A and B are the same as the basic acquisition anchor points, therefore... The target of C is the rolling shutter reference line, and let the phase of this reference line relative to the base acquisition anchor point be . This round has exacerbated historical remnants. .
[0216] Local control time is , , .
[0217] The terminal completes camera parameter settings and returns ARMED at a configuration or preparation time earlier than the aforementioned control time, and issues an action at the control time. Let the actual target acquisition anchor points, after feedback conversion to the R1 time domain and consistent with the respective synchronization configurations of each terminal, be as follows: , , Compared to The errors are +2 ms, -3 ms, and +1 ms, all within the allowable acquisition window in this example. Positive values indicate that the data is too late, and negative values indicate that it is too early.
[0218] When all three mapping slopes are set to 1, the intra-group time error remains unchanged after conversion to local equivalent error. If the diagnosis confirms that the three residuals originate from the signed timing offset between the control action time and the basic acquisition anchor point, then... Update the local time series model: ; ; ; in, This represents the signed control timing offset parameter of the updated terminal A. This represents the signed control timing offset parameter of the updated terminal B. This represents the signed control timing offset parameter of the updated terminal C.
[0219] For terminal C, a new phase of 4.2 ms is independently estimated from a pair of timestamps of the base acquisition anchor point and the reference row anchor point in the same event. hour, ,in, This represents the anchor point phase parameter of the updated terminal C.
[0220] The phase update uses independent anchor interval observations and does not treat the aforementioned +1 ms acquisition bias as a phase error again.
[0221] In some specific embodiments of this application, in a set of dual-terminal method verification experiments, the same terminal client software is deployed on two heterogeneous Android / Camera2 terminal prototypes, and the monotonic clock of the PC is used as the global reference time within the group. The two terminals are connected through the same Wi-Fi local area network. Both terminals select the rear camera Camera 0, and the sensor timestamp source reported by the platform is REALTIME.
[0222] The coordinating node uses four-timestamp exchange to estimate the affine mapping from the terminal's local monotonic clock to the group's global reference time. Each mapping batch is configured with 60 time exchanges, and low round-trip latency samples are retained from the actual received responses; the mapping validity period is set to 60 s. The target global acquisition time is set 2 s after the synchronization code generation time, the round interval is set to 2 s, and the acquisition allowable window before and after the target time is set to 10 ms, for a total of 60 rounds of scheduled feedback acquisition. The initial value of the signed control timing offset used for formal acquisition is given by the calibration snapshot formed by 20 qualified samples from each terminal.
[0223] The camera parameters requested by both terminals were an exposure time of 8 ms, an ISO sensitivity of 400, and a frame duration of 33.333333 ms. Camera2's metadata indicates that the first terminal actually reported an exposure time of 30 ms, an ISO of 17989, and a frame duration of 33.333333 ms, while the second terminal actually reported an exposure time of 8 ms, an ISO of 400, and a frame duration of 33.333333 ms. Therefore, this embodiment verifies the timing method under conditions where the actual sensor parameters are not entirely identical, and does not equate identical requested parameters with identical actual exposure parameters.
[0224] During the formal data acquisition process, the clock mapping was updated every 12 rounds, using a total of 5 mapping versions. The mapping uncertainties for each version were: 3.153029–3.917682 ms for the first terminal and 5.643969–6.352180 ms for the second terminal, both lower than the 15 ms validity threshold set in this embodiment. Feedback control adopted a robust window per terminal; of the 120 feedback decisions, 58 were updates and 62 were hold. The absolute value of the timing offset adjustment in a single round of signed control did not exceed 2 ms, and no adjustments exceeding this limit occurred.
[0225] For the Each terminal converts the SENSOR_TIMESTAMP returned by Camera2 to the group's global reference time according to the freeze mapping of this round, and denotes it as... ,by This is used as the signed target time error. The time difference for each round of cross-terminal data acquisition is taken from two terminals. The difference between the maximum and minimum values. In this embodiment, the timestamp semantics are the exposure start time of the first line reported by the Camera2 platform.
[0226] The full-process statistical results and the final-stage statistical results of this embodiment are shown in Table 3.
[0227] Table 3. Statistical results of the entire process and the final stage of the dual-terminal reservation feedback collection.
[0228]
[0229] As shown in Table 3, which lists the statistical results for all 60 rounds and the last 10 rounds, this table is used to distinguish between the performance throughout the experiment and the performance in the limited number of rounds at the end of the experiment.
[0230] A total of 120 logical acquisition records were generated across 60 rounds of experiments. All records obtained the Camera2 sensor timestamp from the REALTIME source and had valid clock mapping. The initial execution state was SUCCESS, with no dropped rounds, repeated executions, rejections, or MISSED_WINDOW states indicating failure to form a valid acquisition anchor point. Among these, 36 records had target time errors exceeding the ±10 ms acquisition allowable window. This window determination was statistically analyzed separately from the SUCCESS execution state. Based on the full 60 rounds, the absolute target time error did not exceed 10 ms for 48 / 60 rounds on the first terminal and 36 / 60 rounds on the second terminal, totaling 36 / 120 terminal records exceeding the ±10 ms window. The median, P95, and maximum value of the acquisition time difference between the two terminals were 12.920 ms, 22.709 ms, and 26.102 ms, respectively. During the observation period of the last 10 rounds, the target time errors of both terminals were within ±10 ms. The median, P95, and maximum absolute target time error of the first terminal were 5.645 ms, 5.864 ms, and 5.869 ms, respectively, while those of the second terminal were 7.106 ms, 9.544 ms, and 9.889 ms. The median time difference between the two terminals in the corresponding rounds was 1.608 ms, the P95 was 3.893 ms, and the maximum was 4.183 ms. Furthermore, in the 13 consecutive rounds from round 48 to round 60, the target time errors of both terminals were within ±10 ms.
[0231] The above results indicate that, under the device, network, and Camera2 timestamp caliber in this example, the reservation feedback operation entered and maintained a target time error range of ±10 ms for a limited number of rounds in the final stage.
[0232] In some specific embodiments of this application, the glasses terminal adopts the Thunderbird X2 smart glasses.
[0233] The terminal client of the method provided in this application has completed the functional operation verification of installation, startup, Wi-Fi communication and terminal-side data acquisition process on the glasses.
[0234] In multi-terminal deployment, the same terminal client can be deployed in multiple Thunderbird X2 glasses, with each glasses connecting to the PC coordination node via Wi-Fi. The terminal client completes capability registration, clock exchange, synchronization code reception, target time inverse mapping, local scheduled data acquisition, and result feedback with acquisition anchor semantics. The PC coordination node is responsible for establishing a global reference time within the group, generating future target times, distributing synchronization codes, and updating the signed control timing offset of each terminal based on qualified feedback.
[0235] In this implementation, the Thunderbird X2 glasses first report their actual available wireless bearer, local clock, camera control, and timestamp acquisition capabilities. The coordinating node only includes the glasses in the frozen version group if the corresponding capability status is supported and the mapping uncertainty meets the threshold for this round. Camera identifier, available resolution, exposure time range, ISO range, frame duration range, timestamp source, and shutter anchor semantics are based on the capability snapshot reported by the glasses during runtime; no single preset parameter is considered a capability that all Thunderbird X2 hardware and software versions necessarily support.
[0236] For Thunderbird X2 glasses with available acquisition timestamps, the terminal returns the timestamp, acquisition anchor semantics, mapping version, execution status, and actual camera parameters used. The coordinating node only updates the local timing model using feedback with an execution status of SUCCESS, valid local clock mapping, and consistent acquisition anchor semantics; if the glasses do not support the pre-defined anchors or parameters, it returns REJECTED according to the task configuration, reduces the synchronization time semantics, or removes the glasses from the next set of versions.
[0237] This embodiment illustrates the actual functional deployment of the Thunderbird X2 glasses and its multi-terminal expansion capabilities.
[0238] Figure 5 This is a schematic diagram of a cross-physical layer wireless synchronous acquisition system divided by method function according to an exemplary embodiment.
[0239] Reference Figure 5 As shown, one embodiment of this application provides a synchronous acquisition system 100, including a terminal capability information acquisition module 110, a synchronization group construction module 120, a synchronization code generation module 130, a synchronization code distribution module 140, a synchronization code verification and deduplication module 150, and a synchronous acquisition module 160.
[0240] The terminal capability information acquisition module 110 is used to acquire the terminal capability information reported by multiple candidate heterogeneous glasses terminals respectively. The synchronization group construction module 120 is used to determine the glasses terminal for this round of synchronous acquisition based on the acquisition task and terminal capability information, establish a synchronization group and determine the synchronization group information, and generate a synchronization group snapshot. Synchronization code generation module 130 is used to generate synchronization codes based on synchronization group snapshots; The synchronization code distribution module 140 is used to distribute synchronization codes through at least one wireless physical layer according to the available distribution paths of each glasses terminal in the synchronization group. The synchronization code verification and deduplication module 150 is used to verify and deduplicate the synchronization codes received by each glasses terminal in the synchronization group using a quadruple consisting of the synchronization group identifier, group version, acquisition round identifier and command version, and to determine the deduplicated synchronization code. The synchronization acquisition module 160 is used to convert the target global acquisition time into the local control time of the glasses terminal in the synchronization group based on the deduplicated synchronization code and the currently valid local timing model, and to trigger the corresponding glasses terminal to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information.
[0241] The embodiments described above in this application dynamically establish synchronization groups and generate synchronization group snapshots and synchronization codes by acquiring capability information of heterogeneous glasses terminals. This extracts synchronization semantics from specific wireless physical layers, enabling unified scheduling and distribution based on available distribution paths under at least one wireless physical layer, significantly improving compatibility and scalability in heterogeneous environments. A four-tuple consisting of synchronization group identifier, group version, acquisition round identifier, and command version is used to verify and deduplicate the synchronization codes distributed across multiple paths, effectively eliminating redundant command interference and ensuring trigger reliability. The traditional immediate trigger mode is abandoned; instead, based on the currently effective local timing model, the target global acquisition time is mapped to the local control time of each glasses terminal for scheduled execution. This allows glasses terminals with different hardware characteristics to accurately align acquisition windows under a unified global time reference, significantly improving frame-level and exposure-level synchronization consistency.
[0242] The aforementioned system helps reduce observable acquisition bias caused by wireless delivery fluctuations, system scheduling, and sensor pipeline differences through scheduled execution at future target times and cross-wheel feedback correction; it does not require each glasses terminal to support all wireless physical layer bearers simultaneously, nor does it imply hardware triggering or hardware Genlock capabilities.
[0243] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments of the cross-physical layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals, and will not be elaborated here.
[0244] Figure 6 This is a schematic diagram illustrating the structure of another cross-physical-layer wireless synchronous acquisition system for multiple heterogeneous glasses terminals, according to an exemplary embodiment.
[0245] Reference Figure 6 As shown, this application also provides another cross-physical layer wireless synchronous acquisition system for multiple heterogeneous glasses terminals, including a server and a client. The client includes multiple glasses terminals, and the server includes a coordination node or a bridging node. The server and client are connected via at least one wireless physical layer.
[0246] Specifically, the server can have at least one of the following capabilities: sending synchronization codes within a wireless LAN, sending synchronization codes on a Bluetooth broadcast or connection link, sending synchronization codes on a short-range Starlink link, bridging and forwarding synchronization codes on different bearers, and aggregating the collected feedback from each client.
[0247] The client can have at least one of the following capabilities: continuously listen to the synchronization code on the corresponding wireless physical layer bearer; verify and deduplicate the synchronization code; complete the configuration and enter the ready state before the local control time; issue control or trigger actions to perform data acquisition at the local control time; and after the data acquisition is completed, send back execution feedback including execution status, window determination, actual data acquisition timestamp and data acquisition anchor semantics, as well as link feedback including reception time, bearer and path indicators.
[0248] In some specific embodiments of this application, a bridging node may also be introduced between the server and the client. The bridging node may be located between the Wi-Fi domain and the Bluetooth domain, or between the Wi-Fi domain and the Starlink domain. Its function is to keep the upper-layer synchronization semantics unchanged and only adjust the synchronization code encapsulation form according to the differences in wireless bearers.
[0249] Based on the same technical concept, in some specific embodiments of this application, an eyeglass terminal includes a wireless communication module, a clock mapping module, a synchronization code parsing module, a reservation execution module, a sensor control module, and a timestamp feedback module.
[0250] The wireless communication module is used to receive synchronization codes carried by at least one wireless physical layer; the clock mapping module is used to map the target global acquisition time in the synchronization code to a local reference time; the scheduled execution module is used to determine the local control time according to the local timing model and complete the configuration and preparation before that time; the sensor control module is used to issue control or trigger actions at the local control time to enable one or more of the camera, inertial sensor or audio sensor to perform acquisition; the timestamp feedback module is used to transmit back the execution status, window determination, actual acquisition timestamp, acquisition anchor semantics and link information.
[0251] Figure 7 This is a schematic diagram illustrating the structure of another cross-physical-layer wireless synchronous acquisition system for multiple heterogeneous glasses terminals, divided by method function, according to an exemplary embodiment.
[0252] Figure 7 The system 200 shown is an implementation method divided according to its functional division. Figure 6 The system 100 shown is an implementation method based on system roles. The two division methods can be implemented individually or in combination, and functions with the same name are supported by the aforementioned method steps.
[0253] Reference Figure 7As shown, one embodiment of this application provides a cross-physical layer wireless synchronization acquisition system 200 for multiple heterogeneous glasses terminals, divided by method function, including a terminal capability information acquisition module 210, a synchronization group construction and time mapping module 220, a synchronization code generation module 230, a wireless physical layer bearer adaptation and distribution module 240, a local control time conversion module 250, a scheduled acquisition and feedback module 260, and a feedback correction module 270.
[0254] The terminal capability information acquisition module 210 is used to acquire the wireless bearer capability status, time capability status, sensor acquisition capability status and feedback capability status reported by the candidate glasses terminal, and record the capability snapshot version and validity period.
[0255] The synchronization group construction and time mapping module 220 is used to determine the members of this round based on the collection task and terminal capability information, freeze the synchronization group identifier, group version, member set and capability snapshot, and determine the global reference time, reference time identifier, local clock mapping and synchronization configuration of each glasses terminal within the group.
[0256] The synchronization code generation module 230 is used to generate a synchronization code based on the frozen synchronization group snapshot. The synchronization code is bound to at least the synchronization group identifier, group version, collection round identifier, command version, reference time identifier, future target global collection time, effective time window, synchronization configuration, and collection object information.
[0257] The wireless physical layer bearer adaptation and distribution module 240 is used to select the wireless physical layer bearer according to the reachable path and the synchronization code reception deadline of each glasses terminal, and to perform bearer adaptation, redundant distribution or bridging forwarding on the synchronization code with unchanged core field synchronization semantics.
[0258] The local control time conversion module 250 is used for verification and deduplication of the synchronization code. It converts the target global acquisition time into the local reference time based on the effective local clock mapping, and determines the local control time based on the local timing model.
[0259] The scheduled acquisition and feedback module 260 is used to complete the configuration and preparation of the acquisition object before the local control time, issue control or trigger actions to execute acquisition at the local control time, and return execution feedback including execution status, window determination, actual acquisition timestamp and acquisition anchor semantics, as well as link feedback including synchronization code reception time, bearer and path indicators.
[0260] The feedback correction module 270 is used to filter feedback samples with an execution status of SUCCESS, valid local clock mapping, and consistent acquisition anchor semantics, forming a signed acquisition bias, and updating the clock mapping, signed control timing offset, anchor phase, or historical residual parameters according to the source of the error. The above functional division does not mean that the system must be implemented by independent physical devices; each module can be implemented by a combination of software and hardware in a coordination node, bridging node, or glasses terminal.
[0261] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.
[0262] Based on the same technical concept, in some specific embodiments of this application, a terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a cross-physical layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals.
[0263] Based on the same technical concept, in some specific embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a cross-physical layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals are implemented.
[0264] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.
[0265] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.
[0266] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.
[0267] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0268] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0269] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0270] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0271] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.
Claims
1. A cross-physical-layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals, characterized in that, include: Obtain terminal capability information reported by multiple candidate heterogeneous glasses terminals; Based on the data collection task and the terminal capability information, determine the glasses terminal for this round of synchronous data collection, establish a synchronization group and determine the synchronization group information, and generate a synchronization group snapshot. Based on the snapshot of the synchronization group, a synchronization code is generated; The synchronization code is distributed via at least one wireless physical layer carrier according to the available distribution path of each glasses terminal in the synchronization group. The synchronization codes received by each glasses terminal in the synchronization group are verified and deduplicated using a quadruple consisting of the synchronization group identifier, group version, collection round identifier, and command version, and the deduplicated synchronization codes are determined. Based on the deduplicated synchronization code and the currently valid local timing model, the target global acquisition time is converted into the local control time of the glasses terminal in the synchronization group, and the corresponding glasses terminal is triggered to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information.
2. The method for cross-physical layer wireless synchronous acquisition for multiple heterogeneous eyeglass terminals according to claim 1, wherein, The method further includes: Based on the feedback information, the acquisition deviation information is determined, and the currently effective local timing model is updated. The local timing model includes at least one of a signed control timing offset parameter, an anchor point phase parameter, and a historical residual parameter. The signed control timing offset parameter represents the signed predicted timing offset between the control action time and the basic acquisition anchor point. The anchor point phase parameter represents the offset from the basic acquisition anchor point to the target acquisition anchor point required by the synchronization configuration. The historical residual parameter represents the residual compensation determined based on historical feedback information.
3. The method of claim 1, wherein the method is characterized by, The terminal capability information includes wireless bearer capability status, time capability status, sensor acquisition capability status, and feedback capability status. The time capability status includes time base acquisition capability status or local clock mapping capability status, and timestamp acquisition capability status. The process of determining the glasses terminal for this round of synchronous data collection based on the data collection task and the terminal capability information, establishing a synchronization group and determining the synchronization group information, and generating a synchronization group snapshot includes: Candidate glasses terminals are determined based on the data collection task identifier, the candidate terminal list corresponding to the data collection task, the terminal selection criteria corresponding to the data collection task, and the registration deadline. Based on the terminal capability requirements corresponding to the data collection task and the terminal capability information of the candidate glasses terminals, the candidate glasses terminals are screened to determine the glasses terminals to be included in the group. Based on the participation confirmation and readiness status of the glasses terminals included in the group, the glasses terminals to be synchronously collected in this round are determined; A synchronization group is established based on the glasses terminals that have been synchronously collected in this round, and the synchronization group information is determined. The synchronization group information includes the synchronization group identifier, group version, global reference time within the group, the mapping relationship between the local clock of each glasses terminal and the global reference time within the group, and the synchronization configuration corresponding to the collection task. After the registration deadline is reached, the current round member set consisting of the glasses terminals that were synchronously collected in this round is frozen, and the current round member set and the synchronization group information are recorded as a synchronization group snapshot associated with the group version and remaining unchanged during the current round of synchronous collection.
4. The method of claim 1, wherein the method is a method of cross-physical layer wireless synchronization acquisition for a plurality of heterogeneous eyeglass terminals, the method comprising: The core fields of the synchronization code include: synchronization group identifier group_id, group version group_version, collection round identifier round_id, command version command_version, reference time identifier reference_time_id, target global collection time target_global_time, valid time window valid_window, synchronization configuration sync_profile, and collection object information sensor_assignment. The synchronization code also includes at least one of the following: bearer strategy, feedback deadline, rollback strategy, and integrity verification information.
5. The method of claim 1, wherein, The step of distributing the synchronization code via at least one wireless physical layer bearer according to the available distribution paths of each glasses terminal in the synchronization group includes: The first candidate distribution path is determined based on the available distribution paths from the coordinating node to each glasses terminal in the synchronization group and the commonly supported wireless physical bearers and / or the available distribution paths from the coordinating node to each glasses terminal in the synchronization group via the bridging node and the commonly supported wireless physical bearers. Based on the path being reachable and the expected delivery time not later than the synchronization code reception deadline, the first candidate distribution path is filtered to determine the second candidate distribution path. For each glasses terminal, the second candidate distribution path is comprehensively evaluated based on at least one of the following factors: latency, jitter, packet loss rate, delivery rate, reliability, energy consumption, congestion level, number of bridging hops, and number of terminals supporting wireless physical layer transmission, according to the weights set in the task, and the comprehensive evaluation result is determined. For each glasses terminal, based on the comprehensive evaluation results, the optimal second candidate distribution path is determined as the main distribution path, and the wireless physical layer bearer used in the main distribution path is determined as the main bearer corresponding to the main distribution path. The wireless physical layer bearer includes Wi-Fi, Bluetooth, or Starflash. When there are multiple second candidate distribution paths for each glasses terminal, the second best second candidate distribution path is determined as the backup distribution path according to the comprehensive evaluation result, and the wireless physical layer bearer used in the backup distribution path is determined as the backup bearer corresponding to the backup distribution path. For each glasses terminal, a distribution method is determined based on the main distribution path and its corresponding main bearer, the backup distribution path and its corresponding backup bearer, and the distribution method includes at least one of single bearer distribution, multi-bearer redundant distribution and bridging distribution. Generate a logical synchronization code that includes the synchronization group identifier, group version, collection round identifier, and command version; Based on the primary distribution path and the backup distribution path, the logical synchronization code is encapsulated into one or more transmission copies, and the one or more transmission copies are distributed through the corresponding wireless physical layer bearer according to the distribution method.
6. The cross-physical layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals according to claim 1, characterized in that, The method of using a quadruple consisting of a synchronization group identifier, group version, acquisition round identifier, and command version to verify and deduplicate the synchronization codes received by each glasses terminal in the synchronization group, and determining the deduplicated synchronization codes, includes: Synchronization codes with the same synchronization group identifier, group version, collection round identifier, and command version are identified as transmission copies of the same logical synchronization code; One of the transmitted copies of the same logical synchronization code may be selected as the deduplicated synchronization code.
7. The cross-physical layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals according to claim 1, characterized in that, The feedback information includes execution feedback information and link feedback information; The process involves converting the target global acquisition time into the local control time of the glasses terminal in the synchronization group based on the deduplicated synchronization code and the currently valid local timing model, and triggering the corresponding glasses terminal to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information, including: Based on the mapping relationship between the target global acquisition time in the deduplicated synchronization code and the local clock of each glasses terminal in the synchronization group information and the global reference time within the group, the target global acquisition time is converted into the local reference time of the glasses terminal in the synchronization group. The local reference time is compensated using the currently effective local timing model to determine the local control time of the glasses terminal in the synchronization group; The glasses terminal in the synchronization group is controlled to configure the object information to be collected before the local control time. The object information configuration includes at least one of camera configuration, inertial sensor configuration and audio configuration. When the local control time arrives, the corresponding glasses terminal is controlled to perform synchronous data acquisition, and the acquired data, the execution feedback information, and the link feedback information are obtained.
8. The cross-physical layer wireless synchronous acquisition method for multiple heterogeneous glasses terminals according to claim 1, characterized in that, The method further includes: When a glasses terminal in the synchronization group experiences one or more of the following situations: refusing the acquisition task, failing to receive the synchronization code before the synchronization code reception deadline, failing to form a usable target acquisition anchor point, the target acquisition anchor point exceeding the acquisition allowed window, or abnormal execution feedback information in multiple consecutive acquisition rounds participated in by the same glasses terminal, a rollback strategy is adopted to rollback the glasses terminal. Abnormal execution feedback information in multiple consecutive acquisition rounds participated in by the same glasses terminal includes at least one of the following: failing to receive execution feedback information before the feedback deadline, the execution feedback information indicating an invalid local clock mapping relationship, the acquisition anchor point semantics in the execution feedback information being inconsistent with the synchronization configuration, or the execution feedback information indicating an execution status of FAILED.
9. A cross-physical-layer wireless synchronous acquisition system for multiple heterogeneous glasses terminals, characterized in that, include: The terminal capability information acquisition module is used to acquire the terminal capability information reported by multiple candidate heterogeneous glasses terminals respectively. The synchronization group construction module is used to determine the glasses terminal for this round of synchronous data collection based on the data collection task and the terminal capability information, establish a synchronization group and determine the synchronization group information, and generate a synchronization group snapshot. A synchronization code generation module is used to generate a synchronization code based on the synchronization group snapshot; The synchronization code distribution module is used to distribute the synchronization code through at least one wireless physical layer carrier according to the available distribution path of each glasses terminal in the synchronization group. The synchronization code verification and deduplication module is used to verify and deduplicate the synchronization codes received by each glasses terminal in the synchronization group using a quadruple consisting of the synchronization group identifier, group version, acquisition round identifier, and command version, and to determine the deduplicated synchronization code. The synchronous acquisition module is used to convert the target global acquisition time into the local control time of the glasses terminal in the synchronization group according to the deduplicated synchronization code and the currently valid local timing model, and to trigger the corresponding glasses terminal to perform synchronous data acquisition at the local control time to obtain the acquired data and feedback information.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 8.
11. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Intelligent terminal array synchronous working system and working method thereof
CN105589753B
