A Multi-View Data Synchronization Method and System Based on Simulation Trainer

Abstract

This invention relates to the field of simulation trainer data processing technology, specifically providing a multi-view data synchronization method and system based on a simulation trainer. The method acquires a raw data stream containing time and viewpoint identifiers; performs time alignment based on predefined spatiotemporal mapping rules, extracting feature anchors and time sequence relationships; establishes spatial correspondences based on feature anchors; generates synchronization control commands; adjusts the output timing of source data from each viewpoint; and adaptively updates the spatiotemporal mapping rules using feedback information. This invention achieves accurate synchronization of multi-view data, improving the realism and immersive experience of the training system while reducing system latency.

Description

Technical Field

[0001] This invention relates to the field of simulation trainer data processing technology, and in particular to a multi-view data synchronization method and system based on simulation trainers. Background Technology

[0002] In modern simulation training systems, multi-view data acquisition and synchronous processing technology has been widely applied in fields such as military training, aerospace, medical surgery simulation, and driver training. Simulators typically contain multiple data sources from different perspectives, such as visual images, motion parameters, operating commands, and system status. These data need to be highly synchronized in both time and space to provide users with an immersive training experience and ensure the accuracy and effectiveness of training results.

[0003] Traditional multi-view data synchronization methods primarily rely on hardware clock synchronization or simple software timestamp alignment techniques, which have significant limitations when processing complex multi-view data. Data acquisition devices from different viewpoints typically have different sampling frequencies and processing delays, leading to temporal discrepancies in the original data. Simple timestamp alignment cannot effectively address these temporal differences, especially in highly dynamic training scenarios. Existing technologies lack in-depth modeling capabilities for spatial relationships between viewpoints, failing to accurately identify and match corresponding spatial elements from different viewpoints, thus affecting the accuracy and consistency of multi-view data fusion. Most existing synchronization methods employ static configuration parameters, lacking adaptive adjustment mechanisms, making it difficult to cope with dynamic factors such as system load changes and network latency fluctuations during training, and unable to guarantee stable synchronization performance over long periods.

[0004] As simulation training systems develop towards higher precision, higher simulation, and higher interactivity, higher requirements are placed on multi-view data synchronization technology. There is an urgent need for a technical solution that can adaptively handle spatiotemporal alignment, establish mapping relationships between viewpoints, and dynamically adjust synchronization strategies to improve the realism and training effect of simulation training. Summary of the Invention

[0005] The embodiments of the present invention provide a multi-view data synchronization method and system based on a simulation trainer, which can solve the problems in the prior art.

[0006] A first aspect of the present invention provides a multi-view data synchronization method based on a simulation trainer, comprising:

[0007] Acquire raw data streams from multiple viewpoint sources of the simulation trainer, wherein the raw data streams include time markers and viewpoint markers;

[0008] The original data stream is time-aligned based on predefined spatiotemporal mapping rules, and feature anchor points and temporal relationships of each perspective source are extracted to obtain an intermediate dataset with a unified time reference.

[0009] Based on the feature anchor points, a spatial correspondence is established between the various viewpoints. Data elements from different viewpoints in the intermediate dataset are matched according to spatial consistency constraints to obtain a multi-view related data structure.

[0010] Synchronization control instructions are generated based on the multi-view associated data structure and synchronization strategy constraints, wherein the synchronization strategy constraints indicate the priority relationship and consistency check rules between the views.

[0011] Based on the synchronization control command, the data output timing of each view source in the simulation trainer is adjusted, and the adjusted synchronization status information is collected as feedback information.

[0012] The spatiotemporal mapping rules are adaptively updated using the feedback information to continuously synchronize multi-view data.

[0013] The original data stream is time-aligned based on predefined spatiotemporal mapping rules, and feature anchor points and temporal relationships of each perspective source are extracted to obtain an intermediate dataset with a unified time reference, including:

[0014] The time identifiers of each viewpoint source are extracted from the original data stream as a timestamp sequence. Based on the time distribution characteristics of each viewpoint source in the timestamp sequence, the time offset between each viewpoint source is calculated through the spatiotemporal mapping rule to determine a unified time reference.

[0015] Feature descriptors are obtained by extracting features from data elements of each viewpoint source in the original data stream, and the similarity between feature descriptors of data elements between different viewpoint sources is calculated to obtain the viewpoint similarity matrix.

[0016] Based on the inter-view similarity matrix, data element pairs that meet preset corresponding conditions are identified as feature anchors, and the feature anchors carry timestamps under the unified time base;

[0017] Extract the timestamp of the feature anchor point as a time reference point, and interpolate the data elements located between adjacent feature anchor points in each view source according to the time relationship between their time identifier and the time reference point to obtain the corrected timestamp of each data element under a unified time base.

[0018] Based on the corrected timestamp, the data elements of each view source are arranged in chronological order to form a time series, and the temporal relationship between each data element and the feature anchor point is marked in the time series to generate the intermediate dataset.

[0019] Extracting the time identifiers of each viewpoint source from the original data stream as a timestamp sequence, and based on the time distribution characteristics of each viewpoint source in the timestamp sequence, calculating the time offset between each viewpoint source using the spatiotemporal mapping rules to determine a unified time reference, including:

[0020] Extract the time identifiers of each viewpoint source from the original data stream, convert the time identifiers into a unified time format, and obtain the timestamp sequence of each viewpoint source;

[0021] Statistical analysis was performed on the timestamp sequence to extract the temporal distribution characteristics of each perspective source;

[0022] The time distribution characteristics are input into the spatiotemporal mapping rules. Based on the spatiotemporal mapping rules, the time identifier differences of the same data frames in the timestamp sequences of each view source and the reference view source are analyzed, and the time association patterns between each view source and the reference view source are extracted.

[0023] Based on the time correlation pattern and the time distribution characteristics, a time offset calculation strategy is determined. The time offset calculation strategy is then used to calculate the time offset of each view source relative to the reference view source. The time offset represents the time difference between the local clock of each view source and the local clock of the reference view source.

[0024] A unified time reference is determined based on the timestamp sequence of the reference viewpoint source and the time offset of each viewpoint source.

[0025] Based on the feature anchor points, a spatial correspondence is established between different viewpoints. Data elements from different viewpoints in the intermediate dataset are matched according to spatial consistency constraints to obtain a multi-view associated data structure, including:

[0026] Extract the spatial location information of the feature anchor points in each view source, and calculate the spatial transformation relationship between each view source based on the spatial location information;

[0027] Based on the spatial transformation relationship, the transformation matrix between each view source coordinate system is determined so as to map the spatial position under the reference view source coordinate system to the corresponding spatial position under other view source coordinate systems.

[0028] Based on the transformation matrix, a spatial position mapping function is constructed between viewpoint sources, and the spatial position mapping function is used to establish spatial correspondences between each viewpoint source;

[0029] Based on the spatial correspondence and the spatial attributes of each data element in the intermediate dataset, spatial consistency constraints are determined;

[0030] Data elements from different perspectives are extracted from the intermediate dataset, the spatial relationships between the data elements are calculated, the spatial relationships are compared with the spatial consistency constraints, and the combinations of data elements that satisfy the spatial consistency constraints are selected as the matching results.

[0031] Based on the matching results, an association index is established between data elements from different perspectives, and the association index is integrated with the intermediate dataset to generate a multi-perspective association data structure.

[0032] Based on the spatial correspondence and the spatial attributes of each data element in the intermediate dataset, spatial consistency constraints are determined, including:

[0033] Extract the spatial attributes of each data element from the intermediate dataset;

[0034] The spatial attributes are input into the spatial correspondence, and the spatial position mapping function in the spatial correspondence is used to calculate the theoretical mapping position of each data element in different viewpoint source coordinate systems.

[0035] Compare the actual spatial position of the same data element with the theoretical mapped position in different viewpoints, calculate the spatial deviation between the actual spatial position and the theoretical mapped position, and determine the spatial deviation tolerance based on the distribution characteristics of the spatial deviation;

[0036] Based on the spatial deviation tolerance and the spatial location mapping function, spatial consistency constraint rules are generated. These spatial consistency constraint rules are used to indicate the maximum permissible spatial deviation between the actual spatial location and the theoretical mapped location of matchable data elements in different viewpoint sources.

[0037] Based on the spatial consistency constraint rules and the spatial orientation consistency constraint rules of each data element, spatial consistency constraints are determined.

[0038] Synchronization control instructions are generated based on the multi-view associated data structure and synchronization strategy constraints, including:

[0039] The multi-view associated data structure is analyzed, and the association index and spatiotemporal correspondence between the data elements recorded in different viewpoints are extracted. The association index represents the data element to be synchronized in each viewpoint source and its corresponding position. The spatiotemporal correspondence includes the time offset and spatial correspondence between each viewpoint source.

[0040] By analyzing the synchronization strategy constraints, we obtain synchronization timing sub-constraints and synchronization precision sub-constraints. The synchronization timing sub-constraints are used to indicate the time order and time interval of the data output from each viewpoint source, and the synchronization precision sub-constraints are used to indicate the maximum allowable error range for time synchronization of each viewpoint source.

[0041] Based on the time offset and the synchronization timing sub-constraint, calculate the time compensation amount to be adjusted for each viewpoint source relative to the reference viewpoint source;

[0042] The time compensation amount is combined with the associated index to generate time adjustment instructions for each viewpoint source;

[0043] The time adjustment command and the synchronization accuracy sub-constraint are encapsulated into a synchronization control command, and the synchronization control command is transmitted to each view source.

[0044] Based on the synchronization control command, the data output timing of each viewpoint source in the simulation trainer is adjusted, and the adjusted synchronization status information is collected as feedback information, including:

[0045] The synchronization control command is transmitted to each viewpoint source in the simulation trainer. Each viewpoint source parses the synchronization control command and extracts the time compensation amount and data output timing adjustment operation contained in the synchronization control command.

[0046] Each view source calibrates its local clock based on the time compensation amount, adjusts the data acquisition trigger time according to the calibrated local clock and the data output time, and performs data acquisition and data output according to the adjusted data acquisition trigger time to adjust the data output timing of each view source.

[0047] Real-time monitoring of the clock status and data output time of each view source, recording the current local clock timestamp and actual data output timestamp of each view source;

[0048] The actual data output timestamps of each viewpoint source are compared with the data output timestamps of the reference viewpoint source to calculate the actual time synchronization deviation between each viewpoint source and the reference viewpoint source.

[0049] The actual time synchronization deviation is integrated with the local clock timestamps of each view source to generate synchronization status information, and the synchronization status information is used as feedback information.

[0050] A second aspect of the present invention provides a multi-view data synchronization system based on a simulation trainer, comprising:

[0051] The first unit is used to acquire raw data streams from multiple viewpoint sources of the simulation trainer, wherein the raw data streams include time identifiers and viewpoint identifiers;

[0052] The second unit is used to perform time alignment processing on the original data stream based on predefined spatiotemporal mapping rules, extract feature anchor points and temporal relationships of each perspective source, and obtain an intermediate dataset with a unified time reference.

[0053] The third unit is used to establish spatial correspondence between various viewpoints based on the feature anchor points, and to match data elements from different viewpoints in the intermediate dataset according to spatial consistency constraints to obtain a multi-view associated data structure;

[0054] The fourth unit is used to generate synchronization control instructions based on the multi-view associated data structure and synchronization strategy constraints, wherein the synchronization strategy constraints indicate the priority relationship and consistency verification rules between the views;

[0055] The fifth unit is used to adjust the data output timing of each view source in the simulation trainer based on the synchronization control command, and to collect the adjusted synchronization status information as feedback information.

[0056] The sixth unit is used to adaptively update the spatiotemporal mapping rules using the feedback information in order to continuously synchronize multi-view data.

[0057] A third aspect of the present invention provides an electronic device, comprising:

[0058] processor;

[0059] Memory used to store processor-executable instructions;

[0060] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0061] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0062] The multi-view data synchronization method based on the simulation trainer provided by this invention establishes time alignment and spatial correspondence of the original data streams from different viewpoint sources through precise spatiotemporal mapping rules, thereby achieving high-precision synchronization of multi-view data in the simulation trainer. This effectively solves the problem of data inconsistency between viewpoints in traditional methods and improves the realism and immersion of simulation training.

[0063] By introducing feature anchor point extraction and multi-view association data structures, a flexible and reliable inter-view mapping mechanism is established, which enables data from different perspectives to maintain consistency in the spatiotemporal dimension. At the same time, it supports dynamic perspective switching in complex scenarios, enhancing the adaptability and functional scalability of the simulation training system.

[0064] This invention employs a feedback mechanism to adaptively update the spatiotemporal mapping rules, enabling real-time adjustment of the synchronization strategy based on the system's operating status. This effectively addresses synchronization challenges caused by factors such as network latency and hardware performance differences, enhancing the robustness and reliability of multi-view data synchronization and meeting the stable operation requirements of demanding simulation training environments. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating the multi-view data synchronization method based on a simulation trainer according to an embodiment of the present invention.

[0066] Figure 2 This is a schematic diagram illustrating the process of generating intermediate datasets according to an embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0069] Figure 1 This is a flowchart illustrating the multi-view data synchronization method based on a simulation trainer according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0070] Acquire raw data streams from multiple viewpoint sources of the simulation trainer, wherein the raw data streams include time markers and viewpoint markers;

[0071] The original data stream is time-aligned based on predefined spatiotemporal mapping rules, and feature anchor points and temporal relationships of each perspective source are extracted to obtain an intermediate dataset with a unified time reference.

[0072] Based on the feature anchor points, a spatial correspondence is established between the various viewpoints. Data elements from different viewpoints in the intermediate dataset are matched according to spatial consistency constraints to obtain a multi-view related data structure.

[0073] Synchronization control instructions are generated based on the multi-view associated data structure and synchronization strategy constraints, wherein the synchronization strategy constraints indicate the priority relationship and consistency check rules between the views.

[0074] Based on the synchronization control command, the data output timing of each view source in the simulation trainer is adjusted, and the adjusted synchronization status information is collected as feedback information.

[0075] The spatiotemporal mapping rules are adaptively updated using the feedback information to continuously synchronize multi-view data.

[0076] The process of acquiring raw data streams involves receiving data streams from multiple viewpoints of the simulation trainer. Each raw data stream contains a timestamp and a viewpoint identifier. The timestamp can be a timestamp in milliseconds, while the viewpoint identifier can be a unique identifier, such as "VIEW_FRONT_001" or "VIEW_LEFT_002". These raw data streams are generated at different sampling frequencies, such as 30 frames per second for the front view, 25 frames per second for the left view, and 15 frames per second for the top view, leading to time misalignment issues between data streams. To resolve this issue, the arrival time of each data packet is recorded and compared with the timestamp inside the data packet to calculate the transmission delay. For example, if a data packet has an internal timestamp of "14:23:45.876" and a reception time of "14:23:45.912", the transmission delay is 36 milliseconds. This information is stored in a temporary buffer for subsequent processing.

[0077] Time alignment is achieved through predefined spatiotemporal mapping rules. First, a unified time baseline is established, typically using the system clock as a reference. For each viewpoint source, the time series in its data stream is analyzed to identify time drift patterns. For example, a viewpoint source may exhibit a fixed offset (e.g., always delayed by 15 milliseconds) or a variable offset (e.g., a delay that increases linearly with time). Each data stream is sampled and analyzed using a sliding window technique, with the window size typically set to 100-500 data points. For each window, key feature anchors are extracted, such as the edges of static objects in the visual scene, feature points, or areas with significant color gradient changes. These feature anchors are mapped using their temporal relationships. Specifically, it can be identified that the feature point appearing at time "14:23:45.876" in the "VIEW_FRONT_001" data stream (e.g., an edge point at coordinates X=350, Y=420) corresponds to the feature point appearing at time "14:23:45.902" in the "VIEW_LEFT_002" data stream. In this way, an intermediate dataset with a unified time base is generated, in which data from each perspective source are adjusted to a common time reference frame.

[0078] The establishment of spatial correspondences is based on the feature anchor points extracted in the previous step. The spatial distribution and geometric relationships of feature anchor points in each viewpoint source are analyzed to construct a transformation matrix between viewpoints. For example, in a flight simulator, a feature point with coordinates (350, 420) in the front viewpoint and a feature point with coordinates (120, 380) in the left viewpoint represent the same physical entity. By collecting multiple sets of such corresponding points, a spatial mapping between viewpoints is established. In practical applications, at least 8-12 sets of corresponding points can be used to ensure mapping accuracy. When feature points are insufficient, textured regions or the contours of dynamic targets are used as auxiliary references. After mapping, data elements from different viewpoints in the intermediate dataset are matched according to spatial consistency constraints. The matching process uses a distance threshold, typically set to 2-5 units at the pixel level, to ensure matching accuracy. The output of this step is a multi-view associative data structure containing groups of multi-view data elements that are consistent in both time and space.

[0079] Synchronization control instructions are generated based on a multi-view associated data structure and predefined synchronization strategy constraints. These constraints define the priority relationships and consistency verification rules between views. Priority relationships determine which view's data should be prioritized in the event of a conflict; for example, in some training scenarios, the front view is set to the highest priority (priority value 10), while the side view has a lower priority (priority value 7). Consistency verification rules define the criteria for judging whether data is synchronized, such as the maximum allowable time difference (typically 16.67 milliseconds, corresponding to a 60Hz display refresh rate) and spatial error thresholds (e.g., 3 pixel units). Synchronization control instructions are generated based on these rules, including time compensation values, data interpolation parameters, and frame drop handling strategies. For example, for a view source with a detected 25-millisecond delay, the generated instruction requires pre-reading and buffering its data for synchronized output with other views; for views with low data sampling rates, an interpolation instruction is generated, using a weighted average of adjacent frames to generate an intermediate frame.

[0080] Adjusting data output based on synchronization control commands is a crucial step in achieving actual synchronization of multi-view data. The timing of data output is adjusted by controlling the read / write rates of the data buffers for each viewpoint. For viewpoints with significant delays, their buffer sizes are increased to request data earlier; for viewpoints that are ahead, their data output is appropriately delayed. In practical applications, a buffer size of 120 frames can be set for the "VIEW_FRONT_001" viewpoint, while only a 60-frame buffer is set for the "VIEW_TOP_003" viewpoint. The adjusted synchronization status is monitored in real time, and key performance indicators are collected as feedback information, such as the actual time difference between viewpoints, spatial mapping accuracy, and data loss rate. These indicators are recorded every 100 milliseconds by the data acquisition module and stored in the system log for subsequent analysis and optimization.

[0081] The adaptive update mechanism continuously optimizes spatiotemporal mapping rules using feedback information, analyzing trends and patterns in the feedback data to identify parameters that need adjustment. For example, if it is found that under specific operating conditions (such as high-speed movement scenarios), the time difference between the front view and the left view gradually increases and exceeds a predetermined threshold (such as 20 milliseconds), the time mapping parameters of the corresponding view will be automatically adjusted to increase their lead. A sliding window technique (the window size is typically 5-10 minutes of data) is used to calculate the statistical characteristics of key indicators, such as mean, standard deviation, and trend slope, and the direction and magnitude of parameter adjustments are determined based on these characteristics. In cases of decreased mapping accuracy, a process of feature point re-extraction and spatial relationship reconstruction is triggered. This adaptive mechanism ensures high-precision synchronization of multi-view data even under environmental changes or equipment drift.

[0082] Through the above, this invention effectively solves the problem of multi-view data synchronization in simulation trainers, improving the realism and consistency of training results. In practical application testing, this method controls the time synchronization error of multi-view data to within 8 milliseconds, and the spatial correspondence accuracy reaches 96.5%, significantly outperforming the performance of traditional fixed-parameter synchronization methods.

[0083] In one optional implementation, the original data stream is time-aligned based on predefined spatiotemporal mapping rules to extract feature anchors and temporal relationships from each viewpoint source, resulting in an intermediate dataset with a unified time reference, including:

[0084] The time identifiers of each viewpoint source are extracted from the original data stream as a timestamp sequence. Based on the time distribution characteristics of each viewpoint source in the timestamp sequence, the time offset between each viewpoint source is calculated through the spatiotemporal mapping rule to determine a unified time reference.

[0085] Feature descriptors are obtained by extracting features from data elements of each viewpoint source in the original data stream, and the similarity between feature descriptors of data elements between different viewpoint sources is calculated to obtain the viewpoint similarity matrix.

[0086] Based on the inter-view similarity matrix, data element pairs that meet preset corresponding conditions are identified as feature anchors, and the feature anchors carry timestamps under the unified time base;

[0087] Extract the timestamp of the feature anchor point as a time reference point, and interpolate the data elements located between adjacent feature anchor points in each view source according to the time relationship between their time identifier and the time reference point to obtain the corrected timestamp of each data element under a unified time base.

[0088] Based on the corrected timestamp, the data elements of each view source are arranged in chronological order to form a time series, and the temporal relationship between each data element and the feature anchor point is marked in the time series to generate the intermediate dataset.

[0089] Figure 2 This is a schematic diagram illustrating the process of generating intermediate datasets according to an embodiment of the present invention. Figure 2 As shown, the time signatures of each viewpoint source are first extracted from the raw data stream as timestamp sequences. For example, for three different camera viewpoint sources A, B, and C, their original timestamp sequences are as follows: viewpoint A timestamp sequence [10ms, 30ms, 50ms, 70ms], viewpoint B timestamp sequence [15ms, 35ms, 55ms, 75ms], and viewpoint C timestamp sequence [20ms, 40ms, 60ms, 80ms]. By analyzing the distribution characteristics of these timestamp sequences, including the starting value of the timestamps, the interval pattern, and the length of the sequence, the time distribution pattern can be identified. According to the predefined spatiotemporal mapping rules, the time offset between viewpoints A and B is calculated to be 5ms, and the time offset between viewpoints A and C is 10ms. Based on these offsets, the time reference of viewpoint A is determined as the unified time reference, and the timestamps of other viewpoints are adjusted accordingly.

[0090] Next, feature extraction is performed on the data elements from each viewpoint source in the original data stream to obtain feature descriptors. Taking image data as an example, color histograms, texture features, edge features, etc., can be extracted as feature descriptors. Specifically, for image frame A1 in viewpoint A, the extracted feature descriptor is a vector containing information such as color distribution and edge density [0.3, 0.5, 0.2, 0.8]; for image frame B1 in viewpoint B, its feature descriptor is [0.28, 0.52, 0.19, 0.79]; for image frame C1 in viewpoint C, its feature descriptor is [0.25, 0.48, 0.22, 0.75]. The similarity between these feature descriptors can be calculated using methods such as cosine similarity or Euclidean distance to obtain the similarity matrix between viewpoints. For example, the similarity between A1 and B1 is 0.95, the similarity between A1 and C1 is 0.92, and the similarity between B1 and C1 is 0.90.

[0091] Based on the inter-view similarity matrix, data element pairs that meet preset correspondence conditions are identified as feature anchors. These preset conditions can be element pairs with a similarity exceeding a threshold of 0.9 and the highest similarity within a local time window. According to this condition, frames A1 of viewpoint A and B1 of viewpoint B, and frames A1 of viewpoint A and C1 of viewpoint C, are identified as forming two sets of feature anchors. These feature anchors are assigned timestamps under a unified time base; for example, the timestamp of A1 is 10ms, the corrected timestamp of B1 is also 10ms (originally 15ms minus an offset of 5ms), and the corrected timestamp of C1 is also 10ms (originally 20ms minus an offset of 10ms).

[0092] The timestamps of these feature anchors are extracted as time reference points for time correction of data elements between adjacent feature anchors. Assume the next set of feature anchors is A3-B3-C3, with a corresponding timestamp of 50ms under a unified time base. For frame A2 (original timestamp 30ms) in viewpoint A, located between A1 and A3, its corrected timestamp remains 30ms because it lies between two feature anchors. For frame B2 (original timestamp 35ms) in viewpoint B, located between B1 and B3, its corrected timestamp is calculated using linear interpolation: 10ms + (35ms-15ms) / (55ms-15ms) × (50ms-10ms) = 30ms. Similarly, frame C2 in viewpoint C is corrected to a corrected timestamp of 30ms.

[0093] Based on these corrected timestamps, the data elements from each perspective source are arranged in chronological order to form a time series under a unified time base. Taking a 10ms interval as an example, the arranged sequence is: [A1-B1-C1]@10ms, [A2-B2-C2]@30ms, [A3-B3-C3]@50ms, [A4-B4-C4]@70ms. In this time series, the temporal relationship between each data element and the feature anchor point is labeled. For example, A2 is located between feature anchor points A1 and A3, with a time interval of 20ms from A1 and 20ms from A3. This generates an intermediate dataset with a unified time base and temporal relationships.

[0094] This intermediate dataset has the following characteristics: data elements from all perspective sources are mapped to the same time base; feature anchors provide the correspondence between perspectives; and the temporal relationships between data elements are explicitly labeled. This processing method solves the problem of time asynchrony in multi-perspective source data, providing a foundation for subsequent tasks such as multi-perspective data fusion and event correlation analysis. In practical applications, this method can flexibly adjust feature extraction algorithms, similarity calculation methods, and interpolation strategies to adapt to different types of data streams and application scenarios.

[0095] In one optional implementation, the time identifiers of each viewpoint source are extracted from the original data stream as a timestamp sequence. Based on the time distribution characteristics of each viewpoint source in the timestamp sequence, the time offset between each viewpoint source is calculated using the spatiotemporal mapping rules to determine a unified time reference, including:

[0096] Extract the time identifiers of each viewpoint source from the original data stream, convert the time identifiers into a unified time format, and obtain the timestamp sequence of each viewpoint source;

[0097] Statistical analysis was performed on the timestamp sequence to extract the temporal distribution characteristics of each perspective source;

[0098] The time distribution characteristics are input into the spatiotemporal mapping rules. Based on the spatiotemporal mapping rules, the time identifier differences of the same data frames in the timestamp sequences of each view source and the reference view source are analyzed, and the time association patterns between each view source and the reference view source are extracted.

[0099] Based on the time correlation pattern and the time distribution characteristics, a time offset calculation strategy is determined. The time offset calculation strategy is then used to calculate the time offset of each view source relative to the reference view source. The time offset represents the time difference between the local clock of each view source and the local clock of the reference view source.

[0100] A unified time reference is determined based on the timestamp sequence of the reference viewpoint source and the time offset of each viewpoint source.

[0101] In achieving multi-view data time synchronization, the first step is to extract the timestamps from each viewpoint source from the raw data stream as a timestamp sequence. Taking a three-camera video shooting system as an example, this system contains three cameras labeled A, B, and C, each recording a different timestamp format. Camera A records the time in the format "2023-10-15 09:30:45.123", camera B records the time in a Unix timestamp format "1697345445123", and camera C records the time in a relative time format of "09:30:45:123". After extracting these timestamps, they are converted into a unified millisecond-level Unix timestamp format, resulting in the timestamp sequence for the three devices.

[0102] Statistical analysis was performed on the timestamp sequences to extract temporal distribution characteristics and calculate the sampling interval, drift rate, and jitter of the timestamp sequences from each viewpoint source. For example, the average sampling interval of camera device A was 33.33 milliseconds (corresponding to 30 frames / second), with a standard deviation of 0.15 milliseconds; the average sampling interval of camera device B was 16.67 milliseconds (corresponding to 60 frames / second), with a standard deviation of 0.08 milliseconds; and the average sampling interval of camera device C was 40 milliseconds (corresponding to 25 frames / second), with a standard deviation of 0.22 milliseconds. The long-term trend of each timestamp sequence was also analyzed, revealing that device B exhibited a clock drift of approximately 2 milliseconds per hour.

[0103] These temporal distribution characteristics are input into the spatiotemporal mapping rules to analyze the temporal correlation patterns between each viewpoint source and the reference viewpoint source. Camera device A is selected as the reference viewpoint source. By analyzing the timestamps of key events (such as the flash activating or a specific object appearing) in different viewpoint sources within a shared shooting scene, a temporal correspondence is established. For example, when the flash activates, device A records a timestamp of 1697345470123, device B records 1697345470223, and device C records 1697345470073. Through multiple such key event points, it is identified that device B consistently lags behind device A by approximately 100 milliseconds, while device C precedes device A by approximately 50 milliseconds.

[0104] Based on the time correlation pattern and time distribution characteristics, a time offset calculation strategy was determined. Considering the clock drift of device B, a linear regression model was used to calculate the time offset. Time-corresponding data for 20 key event points were collected, and the time offset of device B relative to the reference device A was found to be: Offset = 100 + 0.0005 × (current time - start time) milliseconds, where 0.0005 represents the drift increment per millisecond. For device C, since there is no significant drift, a fixed offset strategy was adopted, and the calculated time offset was -50 milliseconds.

[0105] Based on the timestamp sequence of reference source A and the calculated time offset, a unified time base is determined. The timestamp of device A is directly used as the unified time base. The unified time of device B is equal to its local time minus the corresponding time offset, and the unified time of device C is equal to its local time plus 50 milliseconds. For example, at a certain moment, device A records the time as 1697345500000, device B records the time as 1697345500150 (running for 100 seconds), and device C records the time as 1697345499950. Applying the time offset calculation, the unified time of device B is 1697345500150 - (100 + 0.0005 × 100000) = 1697345500000, and the unified time of device C is 1697345499950 + 50 = 1697345500000.

[0106] To further verify the synchronization effect, 10 key events recorded by three devices were compared under a unified time reference, and the time synchronization error was measured. The results showed that the average synchronization error of device B was 0.8 milliseconds, with a maximum error of 1.7 milliseconds; the average synchronization error of device C was 0.5 milliseconds, with a maximum error of 1.2 milliseconds. This proves that the proposed method can effectively solve the time synchronization problem of multiple viewpoint sources.

[0107] Special cases are also addressed. When intermittent data loss occurs at a certain viewpoint source, the offset of the missing time point is estimated by interpolation of adjacent valid timestamps. For example, if device B experiences data loss between 1697345600000 and 1697345605000, the trend of time offset change during this period is calculated using valid timestamps before and after the data loss to ensure the continuity of time synchronization.

[0108] This method effectively solves the problem of dynamic clock drift that traditional direct timestamp alignment methods cannot handle, thus improving the accuracy and reliability of multi-view data fusion. Experiments show that this method can control the time synchronization error within 2 milliseconds during long-term (over 4 hours) multi-view data acquisition, meeting the needs of most multi-view applications.

[0109] In one optional implementation, a spatial correspondence is established between different viewpoints based on the feature anchor points, and data elements from different viewpoints in the intermediate dataset are matched according to spatial consistency constraints to obtain a multi-view associated data structure, including:

[0110] Extract the spatial location information of the feature anchor points in each view source, and calculate the spatial transformation relationship between each view source based on the spatial location information;

[0111] Based on the spatial transformation relationship, the transformation matrix between each view source coordinate system is determined so as to map the spatial position under the reference view source coordinate system to the corresponding spatial position under other view source coordinate systems.

[0112] Based on the transformation matrix, a spatial position mapping function is constructed between viewpoint sources, and the spatial position mapping function is used to establish spatial correspondences between each viewpoint source;

[0113] Based on the spatial correspondence and the spatial attributes of each data element in the intermediate dataset, spatial consistency constraints are determined;

[0114] Data elements from different perspectives are extracted from the intermediate dataset, the spatial relationships between the data elements are calculated, the spatial relationships are compared with the spatial consistency constraints, and the combinations of data elements that satisfy the spatial consistency constraints are selected as the matching results.

[0115] Based on the matching results, an association index is established between data elements from different perspectives, and the association index is integrated with the intermediate dataset to generate a multi-perspective association data structure.

[0116] First, feature anchors are extracted from source data from different perspectives. These anchors can be obvious feature points in the scene, object edges or corners, or other elements that can be identified from multiple perspectives.

[0117] In practical applications, feature anchor point extraction can be accomplished using feature detection algorithms. For example, in an industrial inspection environment, fixed markers on the production line can be used as feature anchor points. Assume that four feature anchor points are detected from three different camera viewpoints, and record their pixel coordinates in each viewpoint. In the first viewpoint, the coordinates of these four anchor points are (120, 350), (450, 380), (280, 520), and (560, 540); in the second viewpoint, the corresponding coordinates are (180, 320), (510, 340), (340, 490), and (620, 500); and in the third viewpoint, the corresponding coordinates are (150, 380), (480, 410), (310, 550), and (590, 570).

[0118] After extracting the spatial location information of these feature anchor points in each viewpoint source, the spatial transformation relationship between different viewpoint sources is calculated. This process involves estimating parameters such as rotation, translation, and scaling between viewpoints. In the example above, the first viewpoint is set as the reference coordinate system. By comparing the positional differences of the feature anchor points in different viewpoints, the transformation from the first viewpoint to the second viewpoint is calculated to include a 5-degree rotation, a (60, -30) translation, and a scaling factor of 0.95; the transformation from the first viewpoint to the third viewpoint includes a -8-degree rotation, a (30, 30) translation, and a scaling factor of 1.05.

[0119] Based on the transformation parameters described above, transformation matrices between the source coordinate systems of each viewpoint are determined. These matrices describe how a point in one coordinate system is mapped to another. In practical implementations, homogeneous coordinate representation can be used to construct the transformation matrices, which contain rotation, translation, and scaling information. For the transformation from the first viewpoint to the second viewpoint in the example, the constructed transformation matrix can map a point (x, y) in the first viewpoint to a point (x', y') in the second viewpoint.

[0120] Spatial position mapping functions between viewpoint sources are constructed based on the transformation matrix. These functions are used to calculate the corresponding position of any point in one viewpoint in another viewpoint. In practical applications, a bidirectional mapping function is created for each pair of viewpoint sources. For example, for a point (300, 400) in the first viewpoint, by applying the mapping function from the first to the second viewpoint, its corresponding position in the second viewpoint is calculated to be approximately (360, 370).

[0121] These spatial location mapping functions establish spatial correspondences between different viewpoints, describing how the same physical point is associated under different viewpoints. During the intermediate dataset processing stage, each data element is assigned spatial attributes, including its position coordinates in each viewpoint. Taking an industrial inspection environment as an example, a defect point detected on the product surface at (250, 430) in the first viewpoint is calculated using mapping functions to be approximately (310, 400) in the second viewpoint and approximately (280, 460) in the third viewpoint.

[0122] Based on the established spatial correspondence and the spatial attributes of each data element in the intermediate dataset, spatial consistency constraints are determined. These constraints specify the standards for matching data elements from different perspectives, typically expressed as a permissible threshold for positional deviation. In the example, a deviation threshold of 10 pixels is set, meaning that two points from different perspectives are considered to be different perspective representations of the same physical point if the difference between their actual positions and the positions predicted by the mapping function does not exceed 10 pixels.

[0123] Data elements from different perspectives are extracted from the intermediate dataset, and their spatial relationships are calculated. In practical applications, each data element pair is validated to confirm whether it satisfies spatial consistency constraints. For example, if an element is detected at (315, 405) in the second perspective, and the element at (250, 430) in the first perspective is predicted to be at (310, 400) in the second perspective using a mapping function, since the deviation is approximately 7 pixels (less than the 10-pixel threshold), these two elements are considered to satisfy the spatial consistency constraint and are regarded as a matching pair.

[0124] Based on the matching results, an association index is established between data elements from different perspectives. This index records which data elements come from the same physical entity but behave differently from different perspectives. In the example above, an index table can be created to associate the defect element at (250, 430) in the first perspective, the defect element at (315, 405) in the second perspective, and the defect element at (282, 458) in the third perspective, indicating that they describe the same physical defect.

[0125] Finally, the correlation index is integrated with the intermediate dataset to generate a multi-view correlation data structure. This data structure retains the information from the original intermediate dataset while adding correlations between data elements from different perspectives. In industrial inspection applications, this multi-view correlation data structure can provide a complete description of the same defect from different angles, facilitating more accurate defect classification and assessment.

[0126] In one optional implementation, spatial consistency constraints are determined based on the spatial correspondence and the spatial attributes of each data element in the intermediate dataset, including:

[0127] Extract the spatial attributes of each data element from the intermediate dataset;

[0128] The spatial attributes are input into the spatial correspondence, and the spatial position mapping function in the spatial correspondence is used to calculate the theoretical mapping position of each data element in different viewpoint source coordinate systems.

[0129] Compare the actual spatial position of the same data element with the theoretical mapped position in different viewpoints, calculate the spatial deviation between the actual spatial position and the theoretical mapped position, and determine the spatial deviation tolerance based on the distribution characteristics of the spatial deviation;

[0130] Based on the spatial deviation tolerance and the spatial location mapping function, spatial consistency constraint rules are generated. These spatial consistency constraint rules are used to indicate the maximum permissible spatial deviation between the actual spatial location and the theoretical mapped location of matchable data elements in different viewpoint sources.

[0131] Based on the spatial consistency constraint rules and the spatial orientation consistency constraint rules of each data element, spatial consistency constraints are determined.

[0132] In the specific implementation process, the spatial attributes of data elements are extracted first. This involves extracting spatial attributes from the intermediate dataset for each data element, including but not limited to geographic coordinates, building height, and road width. For example, for a building data element, its spatial attributes might include longitude 115.2345°, latitude 39.4567°, height 45 meters, and southeast orientation. For road data elements, spatial attributes such as centerline coordinate sequence, width, and road type can be extracted. The extraction process employs attribute parsing technology to structure the spatial information in the data elements, forming a standardized set of spatial attributes. In practical applications, for building data of a specific urban area, the spatial attributes of thousands of buildings can be extracted, each containing key spatial parameters such as location, shape, and orientation.

[0133] Next, the theoretical mapping position is calculated using spatial correspondence. The extracted spatial attributes are input into a pre-established spatial correspondence model, and the spatial location mapping function is used to calculate the theoretical mapping position of each data element in different viewpoint source coordinate systems. Assuming that the coordinates of a building in the satellite image viewpoint are (x1, y1, z1), the theoretical coordinates of the building in the street view viewpoint (x2', y2', z2') can be calculated using the spatial location mapping function. The mapping function can include coordinate transformation operations such as translation, rotation, and scaling, for example, the conversion from a map coordinate system to a field measurement coordinate system. In a specific case, an office building with coordinates (200, 350, 0) in the satellite image, after calculation using the mapping function, should have a theoretical position of (1200, 2100, 0) in the street view coordinate system.

[0134] Further spatial deviation analysis and tolerance determination are conducted by comparing the actual spatial location and theoretical mapped location of the same data element in different viewpoints, and calculating the spatial deviation between the actual and theoretical locations. For example, if the actual coordinates of a building in a street view viewpoint are (1205, 2103, 0), while the theoretical mapped location is (1200, 2100, 0), then the spatial deviation is (5, 3, 0). The spatial deviations of all data elements are collected, and their distribution characteristics are analyzed, including statistical indicators such as the mean, standard deviation, and maximum value of the deviation. Based on these statistical characteristics, a reasonable spatial deviation tolerance is determined. For example, if the analysis shows that 95% of buildings have a spatial deviation within 10 meters, the spatial deviation tolerance can be set to 10 meters. Different tolerance values ​​are set for different types of data elements, such as a 5-meter tolerance for buildings and an 8-meter tolerance for roads, to accommodate the characteristics of different data elements.

[0135] Subsequently, spatial consistency constraint rules are generated based on the determined spatial deviation tolerance and spatial location mapping function. These rules explicitly indicate the maximum permissible spatial deviation between the actual and theoretical mapped positions of matchable data elements from different viewpoints. A rule can be stated as: "For any data element, the Euclidean distance between its actual position in viewpoint A and its theoretical position calculated from viewpoint B using the mapping function should not exceed a preset tolerance value." For example, for building data elements, the rule could be specified as: "The positional deviation of a building between satellite imagery and street view imagery must not exceed 5 meters, and the height deviation must not exceed 2 meters." These rules can be stored as configuration files or rule bases for subsequent data matching and verification.

[0136] Finally, based on the generated spatial consistency constraint rules and combined with the spatial orientation consistency constraint rules for each data element, a complete spatial consistency constraint is determined. The spatial orientation consistency constraint rules consider the directional characteristics of data elements, such as the orientation of buildings and the direction of roads. The complete constraint includes both positional and orientation constraints, ensuring that matched data elements meet consistency requirements in both spatial position and orientation. In practical applications, matching a building complex across different viewpoints must simultaneously satisfy the constraints of a positional deviation of less than 5 meters and an orientation angle difference of less than 15 degrees. Applying these constraints to tasks such as data fusion, anomaly detection, and spatial relationship verification effectively improves the accuracy and reliability of multi-source data processing.

[0137] The above-mentioned technical process can effectively address the spatial consistency problem among multi-perspective source data, and improve the accuracy of data matching and fusion.

[0138] In one optional implementation, a synchronization control instruction is generated based on the multi-view associated data structure and synchronization strategy constraints, including:

[0139] The multi-view associated data structure is analyzed, and the association index and spatiotemporal correspondence between the data elements recorded in different viewpoints are extracted. The association index represents the data element to be synchronized in each viewpoint source and its corresponding position. The spatiotemporal correspondence includes the time offset and spatial correspondence between each viewpoint source.

[0140] By analyzing the synchronization strategy constraints, we obtain synchronization timing sub-constraints and synchronization precision sub-constraints. The synchronization timing sub-constraints are used to indicate the time order and time interval of the data output from each viewpoint source, and the synchronization precision sub-constraints are used to indicate the maximum allowable error range for time synchronization of each viewpoint source.

[0141] Based on the time offset and the synchronization timing sub-constraint, calculate the time compensation amount to be adjusted for each viewpoint source relative to the reference viewpoint source;

[0142] The time compensation amount is combined with the associated index to generate time adjustment instructions for each viewpoint source;

[0143] The time adjustment command and the synchronization accuracy sub-constraint are encapsulated into a synchronization control command, and the synchronization control command is transmitted to each view source.

[0144] In this embodiment, the process of parsing the multi-view correlation data structure begins with receiving a data structure containing multi-view data correlation information. This structure can be in JSON format and contains metadata for different view sources (such as "view source A", "view source B", and "view source C") and their correlation relationships. Each view source item in the data structure contains a view source ID, an array of data element identifiers, and a corresponding timestamp array. The parser extracts this information to obtain a correlation index table, which records the identifiers of the data elements that need to be synchronized in each view source and their position indices in their respective data streams. For example, data element "element A1" of view source A is located at index position 5, and "element B3" of view source B is located at index position 8. Furthermore, the parser also extracts the spatiotemporal correspondence, including the time offset between view sources (e.g., view source B has a 15-millisecond delay relative to view source A) and a spatial transformation matrix (used to describe the spatial positional relationships captured by different view sources).

[0145] The process of resolving synchronization policy constraints involves processing a configuration file that defines the specific synchronization requirements. Synchronization timing sub-constraints are extracted from the configuration file to determine the order of view source data output (e.g., view source A must output before view source B) and time interval requirements (e.g., the output time interval between view source A and view source B should not exceed 20 milliseconds). Simultaneously, synchronization precision sub-constraints are extracted, specifying the maximum allowable error range for synchronization (e.g., ±5 milliseconds). These constraints are stored in a policy object in memory for subsequent calculations.

[0146] Based on the resolved time offset and synchronization timing sub-constraints, the time compensation amount required for each view source is calculated. Assuming view source A is selected as the reference view source (baseline time source), and view source B is detected to have a 15-millisecond delay relative to view source A, while the synchronization timing sub-constraints require the time interval between them to not exceed 20 milliseconds and view source B must output after view source A, then the calculated time compensation amount for view source B is -10 milliseconds (outputting 10 milliseconds earlier, making the actual delay 5 milliseconds, satisfying the interval requirement). For view source C, if a 25-millisecond delay relative to view source A is detected, and the synchronization requirement is an interval not exceeding 15 milliseconds, then the calculated compensation amount is -15 milliseconds. These compensation amounts ensure that the time difference between each view source is minimized while satisfying the synchronization order requirements.

[0147] During the generation of time adjustment instructions for each viewpoint source, the calculated time compensation amount is combined with the associated index. For viewpoint source B, the instruction is generated: "Adjust the timestamp of viewpoint source B, apply a time compensation of -10 milliseconds to data element 'element B3' at index position 8". Similarly, for the corresponding data element of viewpoint source C, an adjustment instruction applying a compensation of -15 milliseconds is generated. These instructions explicitly specify which data elements need their timestamps adjusted and the specific adjustment value.

[0148] In the final step, the time adjustment command and the synchronization accuracy sub-constraint are encapsulated together into a complete synchronization control command. A complete synchronization control command includes: a time adjustment command for viewpoint source B (adjusting the timestamp of "element B3" by 10 milliseconds); a synchronization accuracy requirement (maximum error not exceeding ±5 milliseconds); and a verification mechanism command (requiring viewpoint source B to verify the synchronization accuracy by sending back the actual adjustment result after performing the adjustment). These commands are serialized into a standard command format, such as a binary protocol message or an XML document, and then sent to the control unit of each viewpoint source via a network transport layer (such as the TCP / IP protocol).

[0149] Upon receiving a synchronization control command, the control unit adjusts the output time of the corresponding data elements accordingly and applies the specified time compensation during data processing. Simultaneously, the control unit monitors the actual time synchronization error after adjustment, ensuring it does not exceed the maximum error range specified in the synchronization accuracy sub-constraint. If the synchronization error exceeds the allowable range, the control unit triggers an alarm mechanism and sends feedback to the central control system, allowing for recalculation of the time compensation and the transmission of updated synchronization control commands.

[0150] Through the above-mentioned technologies, it is possible to accurately calculate and generate synchronization control commands based on the constraints of multi-view related data structures and synchronization strategies, ensuring that the data outputs from different viewpoints are highly synchronized in time, thus meeting the precise synchronization requirements of multi-view data analysis and processing.

[0151] In one optional implementation, the data output timing of each viewpoint source in the simulation trainer is adjusted based on the synchronization control command, and the adjusted synchronization status information is collected as feedback information, including:

[0152] The synchronization control command is transmitted to each viewpoint source in the simulation trainer. Each viewpoint source parses the synchronization control command and extracts the time compensation amount and data output timing adjustment operation contained in the synchronization control command.

[0153] Each view source calibrates its local clock based on the time compensation amount, adjusts the data acquisition trigger time according to the calibrated local clock and the data output time, and performs data acquisition and data output according to the adjusted data acquisition trigger time to adjust the data output timing of each view source.

[0154] Real-time monitoring of the clock status and data output time of each view source, recording the current local clock timestamp and actual data output timestamp of each view source;

[0155] The actual data output timestamps of each viewpoint source are compared with the data output timestamps of the reference viewpoint source to calculate the actual time synchronization deviation between each viewpoint source and the reference viewpoint source.

[0156] The actual time synchronization deviation is integrated with the local clock timestamps of each view source to generate synchronization status information, and the synchronization status information is used as feedback information.

[0157] In a simulation training system, the data outputs from each viewpoint source need to be strictly synchronized to ensure the consistency and realism of the training environment. This implementation achieves high-precision timing synchronization control by precisely adjusting the timing of the data outputs from each viewpoint source and collecting synchronization status information as feedback.

[0158] The central control unit generates a synchronization control command containing time compensation and data output timing adjustment operations, which is transmitted to each viewpoint source via the system bus. This synchronization control command uses a standard format, including a command header (0x55AA), command type (0x01 indicates synchronization control), time compensation amount (accuracy in microseconds, e.g., +1200 indicates a delay of 1.2 milliseconds), data output timing adjustment operations (e.g., 0x01 indicates immediate adjustment, 0x02 indicates adjustment starts in the next frame), and a checksum. Upon receiving the command, each viewpoint source extracts the time compensation amount and adjustment operation parameters through the command parsing module and transmits them to the local clock control module and data acquisition control module.

[0159] After receiving the time compensation amount, the local clock control module adds the value to the clock offset register. For example, if the time compensation amount is +1200 microseconds, the current value of the local clock (e.g., 10:25:30.120000) will be adjusted to 10:25:30.121200. The calibrated local clock will be used for the generation of timestamps in all subsequent data acquisition and output. The local clock uses a temperature-compensated crystal oscillator as its time base, with a stability better than 1 PPM, ensuring that clock drift is minimized after calibration.

[0160] The data acquisition control module adjusts its operation based on the calibrated local clock and data output time, modifying the trigger time value in the data acquisition trigger register. In practical applications, if a viewpoint source is originally scheduled to acquire data at time T (e.g., 13:45:20.500000), and the received time compensation is +1200 microseconds, the trigger time is adjusted to 13:45:20.501200. When the local clock reaches this time, the data acquisition hardware is triggered to perform the acquisition operation. After acquisition, the acquired data, along with the timestamp, is encapsulated into a data frame and sent through the data output interface.

[0161] The system monitors the clock status and data output time of each viewing source in real time. Each viewing source has a monitoring module that records the current local clock timestamp and the timestamp of the most recent data output at a frequency of 100Hz. The recorded information includes the viewing source ID (2 bytes), the local clock timestamp (8 bytes, accurate to microseconds), and the data output timestamp (8 bytes, accurate to microseconds). These records are transmitted to the central control unit in real time through a dedicated status feedback channel. For example, for the viewing source with ID 0x0001, the record is: 0x0001, 0x00E1723A5B4D2C68 (representing the local clock timestamp), 0x00E1723A5B4D2C70 (representing the actual data output timestamp).

[0162] After receiving status feedback from all viewpoint sources, the central control unit selects the viewpoint source with ID 0x0000 as the reference source. It compares the actual data output timestamps of the other viewpoint sources with the data output timestamp of the reference viewpoint source to calculate the time difference. For example, if the data output timestamp of the reference viewpoint source is 0x00E1723A5B4D2C70, and the data output timestamp of the viewpoint source with ID 0x0001 is 0x00E1723A5B4D2CA8, then the calculated time deviation is 56 microseconds (0x38).

[0163] The central control unit integrates the calculated actual time synchronization deviation of each view source with the corresponding local clock timestamp to generate synchronization status information. This information includes a timestamp (8 bytes), the total number of view sources (1 byte), and detailed information for each view source (including view source ID, local clock timestamp, and actual synchronization deviation). For example, for a system with 3 view sources, the synchronization status information would be: 0x00E1723A5B4D2D00 (current timestamp), 0x03 (3 view sources), followed by detailed information for each view source. This synchronization status information is sent to the monitoring terminal and storage system via the system status bus as important feedback on the system's operating status. Simultaneously, this information is also used to generate the next round of synchronization control commands, forming a closed-loop control.

[0164] Through the above implementation, this method achieves precise control over the timing of data output from each viewpoint source in the simulation trainer, ensuring that the time synchronization accuracy between viewpoint sources is better than 100 microseconds, thus meeting the stringent requirements of visual synchronization in high-fidelity training environments. Simultaneously, the complete synchronization status feedback mechanism provides strong support for system operation status monitoring and problem diagnosis.

[0165] Figure 2 This is a schematic diagram of the structure of a multi-view data synchronization system based on a simulation trainer according to an embodiment of the present invention, as shown below. Figure 2 As shown, the system includes:

[0166] The first unit is used to acquire raw data streams from multiple viewpoint sources of the simulation trainer, wherein the raw data streams include time identifiers and viewpoint identifiers;

[0167] The second unit is used to perform time alignment processing on the original data stream based on predefined spatiotemporal mapping rules, extract feature anchor points and temporal relationships of each perspective source, and obtain an intermediate dataset with a unified time reference.

[0168] The third unit is used to establish spatial correspondence between various viewpoints based on the feature anchor points, and to match data elements from different viewpoints in the intermediate dataset according to spatial consistency constraints to obtain a multi-view associated data structure;

[0169] The fourth unit is used to generate synchronization control instructions based on the multi-view associated data structure and synchronization strategy constraints, wherein the synchronization strategy constraints indicate the priority relationship and consistency verification rules between the views;

[0170] The fifth unit is used to adjust the data output timing of each view source in the simulation trainer based on the synchronization control command, and to collect the adjusted synchronization status information as feedback information.

[0171] The sixth unit is used to adaptively update the spatiotemporal mapping rules using the feedback information in order to continuously synchronize multi-view data.

[0172] A third aspect of the present invention provides an electronic device, comprising:

[0173] processor;

[0174] Memory used to store processor-executable instructions;

[0175] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0176] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0177] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for multi-view data synchronization based on a simulation trainer, characterized in that, The method comprises the following steps: acquiring original data streams from multiple perspective sources of a simulation trainer, the original data streams containing time identifiers and perspective identifiers; performing time alignment processing on the original data streams based on a predefined space-time mapping rule, extracting feature anchors and time sequence relationships of the perspective sources, and obtaining an intermediate data set with a unified time reference; establishing a spatial correspondence relationship between the feature anchors of the perspective sources, matching data elements of different perspectives in the intermediate data set according to spatial consistency constraints, and obtaining a multi-perspective correlation data structure; generating a synchronization control instruction according to the multi-perspective correlation data structure and synchronization strategy constraints, the synchronization strategy constraints indicating a priority relationship and consistency checking rules between perspectives; adjusting the data output time sequence of the perspective sources in the simulation trainer based on the synchronization control instruction, and collecting the adjusted synchronization state information as feedback information; using the feedback information to adaptively update the space-time mapping rule to continuously synchronize the multi-perspective data; the step of performing time alignment processing on the original data streams based on a predefined space-time mapping rule, extracting feature anchors and time sequence relationships of the perspective sources, and obtaining an intermediate data set with a unified time reference comprises the following steps: extracting time identifiers of the perspective sources from the original data streams as a timestamp sequence, calculating time offsets between the perspective sources based on the time distribution characteristics of the perspective sources in the timestamp sequence, and determining a unified time reference through the space-time mapping rule; extracting feature descriptors of data elements of the perspective sources to obtain a perspective inter-similarity matrix by calculating the similarity between the feature descriptors of the data elements of different perspective sources; identifying data element pairs that meet a preset correspondence condition as feature anchors based on the perspective inter-similarity matrix, the feature anchors carrying timestamps under the unified time reference; extracting the timestamps of the feature anchors as time reference points, and performing interpolation calculation on data elements between adjacent feature anchors in each perspective source according to the time relationship between the time identifiers of the data elements and the time reference points to obtain corrected timestamps of the data elements under the unified time reference; arranging the data elements of the perspective sources in time order to form a time sequence according to the corrected timestamps, and labeling the time sequence relationship between the data elements and the feature anchors in the time sequence to generate the intermediate data set; the step of generating a synchronization control instruction according to the multi-perspective correlation data structure and synchronization strategy constraints comprises the following steps: parsing the multi-perspective correlation data structure to extract the association index and space-time correspondence relationship between different perspective data elements recorded therein, the association index representing data elements to be synchronized and their corresponding positions in each perspective source, and the space-time correspondence relationship containing time offsets and spatial correspondence relationships between the perspective sources; parsing the synchronization strategy constraints to obtain synchronization time sequence sub-constraints and synchronization accuracy sub-constraints, the synchronization time sequence sub-constraints being used to indicate the time sequence and time interval of data output of the perspective sources, and the synchronization accuracy sub-constraints being used to indicate the maximum error range allowed by the time synchronization of the perspective sources. calculating a time offset between each view source and the reference view source based on the time offset and the synchronization timing sub-constraint; combining the time offset and the association index to generate a time adjustment instruction for each view source; packaging the time adjustment instruction and the synchronization precision sub-constraint into a synchronization control instruction, and transmitting the synchronization control instruction to each view source.

2. The method of claim 1, wherein, extracting the time identifiers of each view source from the original data stream as a timestamp sequence, and calculating the time offset between each view source based on the time distribution characteristics of each view source in the timestamp sequence, and determining a unified time reference, including: extracting the time identifiers of each view source from the original data stream, converting the time identifiers into a unified time format, and obtaining the timestamp sequence of each view source; statistically analyzing the timestamp sequence to extract the time distribution characteristics of each view source; inputting the time distribution characteristics into the space-time mapping rule, analyzing the time identifier difference of the same data frame in the timestamp sequence of each view source and the reference view source based on the space-time mapping rule, and extracting the time association mode between each view source and the reference view source; determining a time offset calculation strategy based on the time association mode and the time distribution characteristics, and calculating the time offset between each view source and the reference view source using the time offset calculation strategy, wherein the time offset represents the time difference between the local clock of each view source and the local clock of the reference view source; determining a unified time reference based on the timestamp sequence of the reference view source and the time offset of each view source.

3. The method of claim 1, wherein, establishing a spatial correspondence between each view source according to the feature anchor points, matching the data elements of different views in the intermediate data set according to the spatial consistency constraint, and obtaining a multi-view associated data structure, including: extracting the spatial position information of the feature anchor points in each view source, and calculating the spatial transformation relationship between each view source based on the spatial position information; determining the conversion matrix between the coordinate systems of each view source according to the spatial transformation relationship, so as to map the spatial position under the coordinate system of the reference view source to the corresponding spatial position under the coordinate system of other view sources; constructing a spatial position mapping function between view sources according to the conversion matrix, and establishing a spatial correspondence between each view source using the spatial position mapping function; determining a spatial consistency constraint according to the spatial correspondence and the spatial attributes of each data element in the intermediate data set; extracting data elements of different views from the intermediate data set, calculating the spatial position relationship between each data element, comparing the spatial position relationship with the spatial consistency constraint, and screening out a data element combination that satisfies the spatial consistency constraint as a matching result; establishing an association index between different view data elements based on the matching result, and integrating the association index and the intermediate data set to generate a multi-view associated data structure.

4. The method of claim 3, wherein, determining a spatial consistency constraint according to the spatial correspondence and the spatial attributes of each data element in the intermediate data set, including: extracting the spatial attributes of each data element from the intermediate data set; Input the spatial attribute into the spatial correspondence, and calculate the theoretical mapping position of each data element in the different view source coordinate system by using the spatial position mapping function in the spatial correspondence; Compare the actual spatial position of the same data element in different view sources with the theoretical mapping position, calculate the spatial deviation between the actual spatial position and the theoretical mapping position, and determine the spatial deviation tolerance based on the distribution characteristics of the spatial deviation; Generate a spatial consistency constraint rule according to the spatial deviation tolerance and the spatial position mapping function, which is used to indicate the maximum spatial deviation allowed between the actual spatial position and the theoretical mapping position of the matchable data element in different view sources; Determine the spatial consistency constraint based on the spatial consistency constraint rule and the spatial orientation consistency constraint rule of each data element.

5. The method of claim 1, wherein, Adjust the data output timing of each view source in the simulation trainer based on the synchronization control instruction, and collect the adjusted synchronization state information as feedback information, including: Transmit the synchronization control instruction to each view source in the simulation trainer, each view source parses the synchronization control instruction, and extracts the time compensation amount and data output time adjustment operation contained in the synchronization control instruction; Each view source calibrates the local clock based on the time compensation amount, adjusts the data collection trigger time according to the calibrated local clock and the data output time adjustment operation, and performs data collection and data output according to the adjusted data collection trigger time, so as to adjust the data output timing of each view source; Real-time monitor the clock state and data output time of each view source, record the current local clock timestamp and actual data output timestamp of each view source; Compare the actual data output timestamp of each view source with the data output timestamp of the reference view source, and calculate the actual time synchronization deviation between each view source and the reference view source; Integrate the actual time synchronization deviation and the local clock timestamp of each view source to generate synchronization state information, and take the synchronization state information as feedback information.

6. A multi-view data synchronization system based on a simulation trainer for implementing the method according to any one of claims 1 to 5, characterized in that It includes: The first unit is used for obtaining the original data stream from multiple view sources of the simulation trainer, and the original data stream contains time identification and view identification; The second unit is used for time alignment processing of the original data stream based on a predefined space-time mapping rule, extracting feature anchor points and time sequence relationship of each view source, and obtaining an intermediate data set with a unified time reference; The third unit is used for establishing a spatial correspondence between view sources according to the feature anchor points, matching data elements of different views in the intermediate data set according to spatial consistency constraints, and obtaining a multi-view associated data structure; The fourth unit is used for generating a synchronization control instruction according to the multi-view associated data structure and a synchronization strategy constraint, and the synchronization strategy constraint indicates the priority relationship and consistency checking rule between views; The fifth unit is used for adjusting the data output timing of each view source in the simulation trainer based on the synchronization control instruction, and collecting the adjusted synchronization state information as feedback information. A sixth unit is configured to update the space-time mapping rule adaptively using the feedback information to continuously synchronize multi-view data.

7. An electronic device, comprising: The method comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1 to 5.

Images