Artificial intelligence-based multi-source data association and integration method

Abstract

The present application belongs to the technical field of data processing and multi-source information fusion, and is used to solve the problem that the prior art cannot detect and effectively respond to the collective time drift caused by hidden node faults in the synchronization tree in real time. Specifically, it is a multi-source data correlation integration method based on artificial intelligence, which comprises: obtaining multiple data sources and their synchronization topology information; in response to a request for time sequence correlation of a first data source and a second data source, determining the data source with the smaller value between the first trust risk value and the second trust risk value as the reference source, and the other data source as the source to be calibrated; calculating the absolute value of the difference between the nominal time offset and the witness time offset as the consistency residual; and performing correlation determination based on the consistency residual. By introducing a physical reference benchmark independent of the synchronization tree and a periodic traceability prisoner test, the present application can accurately identify the collective time drift caused by hidden node faults in the synchronization tree, thereby effectively blocking the step-by-step transmission of errors along the synchronization tree.

Description

Technical Field

[0001] This invention belongs to the field of data processing and multi-source information fusion technology, specifically a multi-source data association and integration method based on artificial intelligence. Background Technology

[0002] Since the problem of multi-source data association and integration is very common in various applications such as autonomous driving vehicle-road cooperation, industrial IoT equipment monitoring, and power grid fault diagnosis, and the accurate alignment of data collected by different sensors and data acquisition devices on the time axis is the basis for subsequent fusion and decision-making, there is a very natural and mature solution to the timestamp alignment problem of various data sources in actual systems: that is, to construct a synchronization tree structure, with a high-precision time synchronization source (GPS receiver) as the root node, and each data source obtains the time reference from the parent node through hardware synchronization lines, network time protocols, or calculation methods to form a cascaded synchronization topology; more importantly, existing technologies assign a static trust level to each data source, which is generally determined by the number of synchronization hops to the root node: the smaller the number of hops, the higher the trust level; therefore, when it is necessary to associate the timestamps of two data sources, the system directly selects the source with the higher trust level as the time reference, and reasonably assumes that the timestamp of the other source can be reliably calibrated by this reference.

[0003] However, when the timing signal of the root node is affected by electromagnetic interference, or when the crystal oscillator of an intermediate node in the synchronization path experiences temperature drift or even failure, the time error will propagate down the synchronization tree and accumulate. Since all downstream nodes believe that they obtained the reference from the legitimate parent node, their timestamps still present a "relatively consistent" self-consistent state when compared with each other, but the absolute time of the entire subtree has drifted as a whole. At this time, the static trust level mechanism cannot detect this "collective self-consistent drift" because the trust level of each node has not changed, and the time difference within the subtree is still within the preset tolerance range. Only when an independent external absolute reference event intervenes will it be discovered that the recorded time of the entire subtree has deviated from the real physical time. At this time, the multi-source correlation integration, causal inference, and early warning decision-making based on these timestamps have produced irreversible systemic errors.

[0004] Therefore, how to detect and effectively respond to collective time drift caused by hidden node failures in the synchronization tree in real time, and avoid the resulting "self-consistent but inaccurate" time base contaminating the entire multi-source association and integration process, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-source data association and integration method based on artificial intelligence, which can solve the problem that existing technologies cannot detect and effectively deal with collective time drift caused by hidden node faults in the synchronization tree in real time;

[0006] The technical problem to be solved by this invention is: how to provide an artificial intelligence-based multi-source data association and integration method that can detect and effectively respond to collective time drift caused by hidden node faults in a synchronization tree in real time.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] AI-based multi-source data association and integration methods include:

[0009] Obtain multiple data sources and their synchronization topology information. The synchronization topology information includes at least the synchronization parent node identifier and synchronization method type of each data source. Based on the synchronization topology information, initialize the trust depth parameter for each data source.

[0010] In response to the request to perform a time-series correlation between the first data source and the second data source, a first trust risk value and a second trust risk value are generated for the first data source and the second data source respectively based on the trust depth parameter;

[0011] The data source with the smaller value between the first trust risk value and the second trust risk value is determined as the reference source, and the other data source is determined as the source to be calibrated.

[0012] Acquire at least one pair of physical event records that are jointly sensed by the reference source and the source to be calibrated, each pair of physical event records containing the first recording time of the same physical event on the reference source and the second recording time on the source to be calibrated;

[0013] The nominal time offset is calculated based on the timestamp of the reference source and the timestamp of the source to be calibrated. The witness time offset is calculated based on the first recording time and the second recording time. The absolute value of the difference between the nominal time offset and the witness time offset is calculated as the consistency residual.

[0014] Association determination is based on consistent residuals.

[0015] The present invention has the following beneficial effects:

[0016] 1. By introducing a physical reference benchmark independent of the synchronization tree and a periodic source-tracing prisoner's dilemma test, it is possible to accurately identify collective time drift caused by hidden faults in nodes of the synchronization tree (such as crystal oscillator temperature drift and time signal interference), and trace back up the synchronization tree to the root anchor point and down to all descendant nodes to isolate the entire tree, thereby effectively blocking the step-by-step transmission of errors along the synchronization tree and avoiding the concealment of the true time deviation by the self-consistency of the subtree.

[0017] 2. The trust risk value is calculated by coupling the trust hop count and the dynamic quality factor, and the benchmark source is dynamically selected based on the risk value. At the same time, the proportion of data sources currently in the penalty period is used as the feedback quantity to modulate the dynamic alignment threshold of the association judgment in real time. This allows the system to automatically relax the tolerance to maintain availability when the synchronization condition deteriorates, and tighten the threshold to ensure the association quality when the system is healthy, thus achieving an adaptive balance between sensitivity and robustness.

[0018] 3. By using a public beacon generator to broadcast a pulse signal carrying an incrementing sequence number as a positive interactive verification event, and by comparing the nominal time offset with the witness time offset, a consistency residual is obtained. Thus, without relying on the absolute time reference of any single data source, the real-time verification of the relative deviation between the two timestamps is completed, which solves the fundamental defect of the existing static trust level mechanism that cannot perceive the collective drift that is "self-consistent but inaccurate".

[0019] 4. A complete closed-loop maintenance mechanism is constructed through periodic source tracing prisoner test, penalty steps for drifting pollution sources, and recovery steps after the penalty period expires. Among them, the penalty steps cause the trust hop count of the pollution source and its descendant nodes to increase synchronously and the dynamic quality factor to decrease synchronously. The recovery steps gradually restore the expired penalty source to the initial trust level, thereby ensuring the time base stability and self-healing capability of the multi-source data association and integration system in long-term operation. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of the method according to Embodiment 1 of the present invention. Detailed Implementation

[0022] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] In existing multi-source data association and integration systems, such as vehicle-road collaboration for autonomous driving, equipment monitoring for the Industrial Internet of Things, and fault diagnosis of power grids, the timestamp alignment of different sensors or data acquisition devices usually relies on a synchronization tree structure: with a high-precision time synchronization source as the root node, other data sources obtain the time reference level by level from the parent node through hardware synchronization lines, network time protocols, or calculation methods. Under this architecture, existing technologies generally adopt a static trust level mechanism, that is, a fixed trust level is pre-assigned according to the number of synchronization hops of each node from the root node - the fewer the hops, the higher the trust level. When it is necessary to associate the timestamps of two data sources, the system directly selects the source with the higher trust level as the time reference and assumes that the timestamp of the other source can be calibrated from it.

[0024] However, this static trust level mechanism has a deep flaw: it cannot detect the "collective self-consistent drift" caused by hidden node faults in the synchronization tree (such as crystal oscillator temperature drift or time signal interference). Specifically, when a small but persistent clock deviation occurs in an upstream intermediate node, the error will be propagated and accumulated down the synchronization tree. Since all downstream nodes obtain their reference from their respective parent nodes, they still show a consistent time difference when comparing with each other (i.e., self-consistent within the subtree), but the absolute time of the entire subtree has deviated from the real physical time. At this time, the static trust level has not changed, and the system still uses the high trust level node as the reference for association, causing subsequent multi-source fusion, causal inference, and early warning decisions to be based on a timeline with systematic deviation. For example, in the autonomous driving scenario, if the GPS timing module is affected by electromagnetic interference and produces a fixed delay of 20 milliseconds, this error is propagated to the LiDAR and camera through the synchronization tree. Although the timestamps of all sensors are consistent with each other, the vehicle's position estimation will have a deviation of more than half a meter, while the confidence assessment inside the system still shows "high confidence".

[0025] If the above problems are not addressed, the multi-source data integration system will continuously lose its ability to objectively determine the authenticity of the time reference. In particular, the failure to identify collective drift will cause the timestamps of the entire subtree to be mistakenly taken as the absolutely accurate reference, resulting in subsequent time series correlation, trend analysis, and causal inference deviating from the actual physical process. At the same time, due to the self-consistency within the subtree, the system will not trigger any abnormal alarms, and the deviation will exist covertly for a long time and gradually amplify with the accumulation of data, eventually causing irreversible systemic errors in the integration results on the absolute timeline. As a result, the inaccuracy of correlation integration will systematically undermine the reliability of upper-level decision-making and affect the safety and effectiveness of application scenarios with stringent time synchronization requirements, such as autonomous driving and industrial control.

[0026] Example 1: As Figure 1 As shown, the multi-source data association and integration method based on artificial intelligence includes:

[0027] Step S1: Obtain multiple data sources and their synchronization topology information. The synchronization topology information includes at least the synchronization parent node identifier and synchronization method type of each data source. Based on the synchronization topology information, initialize the trust depth parameter for each data source.

[0028] This paper provides a clear and hierarchical explanation of the method for obtaining synchronization topology information from multiple data sources in step S1. First, it explains that the synchronization parent node identifier and synchronization mode type of each data source are read using the interface provided by the underlying synchronization protocol (i.e., Precision Time Protocol (PTP) or Network Time Protocol (NTP)). Then, it naturally and appropriately introduces the meaning of the synchronization parent node identifier, which indicates the previous level reference source in the synchronization tree. Next, it systematically and rigorously discusses the three classifications of synchronization mode types: hardware synchronization, software synchronization, and deduced synchronization, providing their definitions and typical implementations: hardware synchronization is implemented using a dedicated synchronization signal line or PTP hardware timestamp; software synchronization is implemented using NTP or SNTP protocols; and deduced synchronization calculates the timestamp from data values ​​at different times from the same data source or from external triggering logic. Finally, it points out that the parent node identifier of the source (e.g., a GPS timing module or atomic clock interface) at the root node of the synchronization tree is empty.

[0029] Since the trust depth parameter can be naturally and reasonably initialized for each data source after obtaining the synchronization topology information, the paper clearly defines the trust depth parameter: it has two distinct components, namely the trust hop count (denoted as...). ) and dynamic quality factor (denoted as The former indicates how many synchronization levels the source under examination is from the root anchor point, while the latter measures the reliability of the source timestamp under examination. The larger the value, the higher the reliability. Initialization is performed in the breadth-first traversal order of the synchronization tree, first the parent node, then the child node.

[0030] Since the root anchor point in the synchronization tree without a parent node has no parent node, it is natural to set its trust hop count to 0 and its dynamic quality factor to 1.0, that is: , ;

[0031] For any child node, first obtain the trust hop count of its parent node (denoted as ). ) and dynamic quality factor (denoted as The trust hop count of the child node is set to the trust hop count of the parent node plus 1. ;

[0032] Subsequently, the initial value of its dynamic quality factor is determined based on the synchronization mode type of the child node; let the attenuation factor corresponding to the synchronization mode type be... Then the dynamic quality factor of the child node is:

[0033] ;

[0034] Because of the different synchronization methods, it is natural and appropriate to discuss them separately according to hardware synchronization, software synchronization, and calculation synchronization: , , ;

[0035] The example clearly and naturally illustrates that: the dynamic quality factor of the root anchor point is 1.0, and the dynamic quality factor of the first-level child node connected by hardware synchronization is... The dynamic quality factor of the second-level child node connected to the first-level child node via software synchronization is... Therefore, the trust hop count for the first-level child node is 1, and for the second-level child node it is 2.

[0036] Since the initialization process performs loop detection on the synchronization topology at the beginning, it first checks for closed-loop dependencies. If a topology anomaly is found, the initialization is naturally stopped and an error report is given. At the same time, the number of trusted hops is reasonably limited to the range of 0 to 64. Nodes exceeding 64 are counted as 64, which effectively avoids the problem of risk value calculation overflow.

[0037] In addition to the trust hop count and dynamic quality factor, the system naturally and reasonably initializes several auxiliary parameters for each data source: a circular queue with a fixed length of 100, specifically used to store the consistency residual values ​​generated during future association processes; therefore, this queue is empty during initialization; and a variable at the end of the penalty period (denoted as...). The initial value is 0, so it is obviously not in the penalty period at this time, and there is an immutable backup of the initial trust hop count (denoted as ). This is used as a lower bound constraint during penalty recovery, and its initial value is the trust hop count at initialization.

[0038] Since the initialization process does not use any absolute time reference, it is natural and reasonable to use the topology provided by the synchronization protocol and the selected attenuation factor to independently and directly calculate the trust depth parameters of all data sources. This naturally solves the problem of subsequent changes in the synchronization tree (i.e., parent node switching): at this time, only the trust depth parameters need to be recalculated based on the new topology information, without resetting other accumulated states. This naturally leads to the conclusion that after initialization, each data source carries complete trust depth parameters and auxiliary states for direct use in subsequent steps.

[0039] Step S2: In response to the request to perform time-series correlation on the first data source and the second data source, a first trust risk value and a second trust risk value are generated for the first data source and the second data source respectively based on the trust depth parameter; the data source with the smaller value between the first trust risk value and the second trust risk value is determined as the reference source, and the other data source is determined as the source to be calibrated;

[0040] Since step S2 is the step taken by the system after receiving a request regarding the time-series association of the first and second data sources, the first trust risk value and the second trust risk value must first be calculated using the trust depth parameters initialized for each data source. The trust depth parameter is the trust hop count that has been calculated and stored in step S1. and dynamic quality factor Furthermore, the trust risk value itself is used to quantify the unreliability of the data source timestamp, so the larger the value, the more unreliable the corresponding timestamp.

[0041] To calculate the trust risk value, the system first needs to read the first trust hop count contained in the first data source. First dynamic quality factor and the second trust hop count contained in the second data source. Second dynamic quality factor Then, a preset zero constant is appropriately introduced. (A reasonable value can be taken) This naturally and appropriately leads to the first trust risk value. Second Trust Risk Value : , ;

[0042] The physical meaning of this formula is that the larger the trust hop count, the more transmission links the timestamp goes through, and the greater the possibility of error accumulation, thus the greater the risk value. On the other hand, the larger the dynamic quality factor, the higher the credibility of the timestamp, and the smaller the risk value. Therefore, the two have a clear inverse relationship. This naturally and reasonably leads to the practice of using division to couple the two into a unified risk indicator.

[0043] For example, suppose the first data source is the root anchor point, its , ,but

[0044] ;

[0045] The second data source is a child node that has undergone two levels of hardware synchronization. , ,but ;

[0046] Therefore, the risk value of the root anchor point is 0, while the risk value of the deep nodes increases significantly.

[0047] Obtain the first trust risk value Second Trust Risk Value The system then compares the two; the data source with the smaller value is identified as the baseline source, and the other data source is identified as the source to be calibrated. This comparison logic is based on the principle that "the smaller the risk value, the more reliable the timestamp," therefore, the source with the smaller risk value is more suitable as the reference benchmark for subsequent alignment. Continuing with the previous example, since... Less than Therefore, the first data source is selected as the baseline source, and the second data source is the source to be calibrated; if the risk values ​​of the two are equal (for example, both are 0), the system can determine the baseline according to the preset default rules (for example, selecting the data source with the smaller identifier as the baseline).

[0048] After determining the reference source and the source to be calibrated, the system temporarily and reliably stores the identity of the two data sources and their respective risk values ​​in the context of this association request for subsequent steps S3 and S4 to call. More importantly, the calculation of the trust risk value makes full use of the trust depth parameter initialized in step S1, without introducing any new measurements or external references, so the processing is lightweight and real-time.

[0049] Step S3: Obtain at least one pair of physical event records jointly sensed by the reference source and the source to be calibrated. Each pair of physical event records includes the first recording time of the same physical event on the reference source and the second recording time on the source to be calibrated.

[0050] Since step S3 requires acquiring at least one pair of physical event records sensed by both the reference source and the source to be calibrated, this embodiment naturally and reasonably introduces a common beacon generator, independent of the synchronization tree of each data source, to periodically... The broadcast wireless pulse signal has a basic period of one second, and each pulse signal carries a globally unique, strictly increasing sequence number (denoted as ). The sequence number starts from 0 and increments by 1 for each pulse sent. More importantly, the selected beacon generator can be a very mature and low-cost radio frequency transmission module with an independent crystal oscillator as its clock source, independent of the timing system of each data source. Therefore, the signal it generates can be reliably regarded as a real "physical event".

[0051] Since each data source is equipped with a corresponding receiving module, it automatically and reliably records the local clock time (denoted as ) upon receiving the beacon pulse. ) and the sequence number carried by the pulse Then put this pair of data After pairing, the records are stored in their respective event buffers. These event buffers are essentially circular queues with a capacity of M=100 records, storing records sequentially according to the order of their reception time. Therefore, when the buffer is full, the newest record naturally overwrites the oldest record, thus always retaining the beacon reception information from the most recent period.

[0052] Based on this, when step S2 has determined the reference source (denoted as...) ) and the source to be calibrated (denoted as ) After that, the system performs the following operations to obtain physical event record pairs; first, it accesses the reference source. The event buffer selects an unused pulse record in ascending order of sequence number; "unused" means that the pulse record has not been used in any witness calculation for an association request; to maintain this state, each pulse record from each data source can be accompanied by a Boolean usage flag, initially set to 0. When the record is retrieved by a related request, the flag is set to 0. For the sequence number increment order, the system starts searching backward from the unused record with the largest sequence number in the buffer (i.e., the latest and unused record), or it can start traversing from the smallest sequence number. In this embodiment, it is preferred to take the records in ascending order of sequence number to ensure that each pulse record is used at most once and to avoid time correlation deviation caused by reusing the same physical event.

[0053] Assume the record selected from the reference source buffer is ,in The reference source records the local time of the pulse; the system uses the source to be calibrated. The event buffer is used as the retrieval target to search for items with the same sequence number. The pulse record; if found, the corresponding local time is extracted. and will As a record of the same physical event on the reference source and the source to be calibrated; if no record with the same sequence number is found in the buffer of the source to be calibrated (possibly due to packet loss or non-receipt), the system continues to select the next unused pulse record (second largest or second smallest sequence number) in the reference source buffer, repeating the above search process until at least one matching pair is successfully obtained, or no match is found after traversing all unused records in the reference source buffer; in a typical embodiment, as long as the beacon is working properly and the communication quality is good, a matching pair can usually be easily obtained.

[0054] For example, suppose a common beacon generator sends pulses at 2-second intervals, with sequence numbers 0, 1, 2, ...; at a certain moment, the reference source... The event buffer stores three records with sequence numbers 100, 101, and 102, corresponding to local times respectively. Second, Second, Seconds; Source to be calibrated The buffer stores records with sequence numbers 100 and 102 (assuming sequence number 101 was lost due to interference), corresponding to the local time. Second, Seconds; the system retrieves the record with serial number 100 from the reference source in ascending order of serial number. If a record with the same serial number 100 is found in the source to be calibrated, then the recording time is obtained. If, for some reason, serial number 100 also does not exist in the source to be calibrated, then try serial number 101. If it is not found, then try serial number 102 to obtain the recorded time. .

[0055] Since step S3 will block if no matching pair is found for an extended period, the system employs a natural and reasonable timeout mechanism: if no matching pair is found within a preset maximum number of attempts (e.g., 10), the association request abandons the positive interaction and directly proceeds to the degradation processing branch corresponding to step S5. For details on the degradation processing behavior, please refer to the description of the corresponding step. Complementing this is the system setting the time value of at least one physical event record pair after obtaining at least one physical event record pair. The data is passed to step S4 to calculate the witness time offset. Meanwhile, in order to ensure the accuracy of the event buffer content, the system officially marks the reference source pulse record used this time as "used". Whether the record corresponding to the source to be calibrated is marked as used has no actual impact on the subsequent matching.

[0056] To prevent all records in the event buffer from being marked as "used" after long-term operation, resulting in no available witness events, the system resets the event buffer usage flag of all data sources every 24 hours. In addition, if there are no unused pulse records in the current baseline source event buffer, the degradation processing branch is triggered directly, and no further attempts are made to find matching pairs.

[0057] Step S4: Calculate the nominal time offset based on the timestamp of the reference source and the timestamp of the source to be calibrated, calculate the witness time offset based on the first recording time and the second recording time, and calculate the absolute value of the difference between the nominal time offset and the witness time offset as the consistency residual.

[0058] In step S4, the system calculates the nominal time offset, witness time offset, and consistency residual in sequence based on the timestamps of the reference source and the source to be calibrated determined in step S2 and the physical event record pair obtained in step S3.

[0059] Set reference source The timestamp carried by the record to be associated is Source to be calibrated The timestamp carried by the record to be associated is These two timestamps are the readings of the local clocks of their respective data sources at the time the data record was generated; nominal time offset. Defined as the timestamp of the source to be calibrated minus the timestamp of the reference source: ;

[0060] This offset reflects the direct difference between the recording times of the two data sources without considering any calibration; for example, assuming the timestamp of the reference source. Seconds, timestamp of the source to be calibrated Seconds, then the nominal time offset The seconds indicate that the timestamp of the source to be calibrated is approximately 250 milliseconds faster than that of the reference source.

[0061] Secondly, the system obtains the first recorded time of the same physical event on the reference source obtained in step S3. and the second recording time on the source to be calibrated Witness time offset Defined as the second recording time minus the first recording time: ;

[0062] Since the offset discussed is a direct reflection of the true relative difference between the local clocks of the two data sources when the same physical event occurs, the following can be naturally and clearly concluded: If the reference source receives a beacon pulse at the time when... The time when the calibration source receives the same pulse is seconds. Seconds, thus witnessing time shift The second is the time when the local clock of the source to be calibrated is 210 milliseconds ahead of the reference source at the time the physical event occurs.

[0063] Theoretically, it is natural and appropriate to assume that the nominal time offset With the offset of witness time They should be equal because both are direct measures of the relative clock deviation between the source to be calibrated and the reference source. However, it must be acknowledged that in actual systems, they are generally not equal: the time of the record to be correlated differs from the time of the physical event, clock drift has nonlinear characteristics, and data acquisition and processing have delays. Therefore, a reasonable and rigorous approach is to calculate the absolute value of the difference between the nominal time offset and the witness time offset, defining it as the consistency residual. : .

[0064] The consistency residuals can be derived quite naturally and clearly from the numerical examples above: , , The time interval is 40 milliseconds, so its physical meaning is naturally given: the residual is essentially a quantitative reflection of the degree of inconsistency between the timestamp used by the record to be associated and the clock deviation measured by the most recent physical event. Therefore, the smaller the residual, the more reliable the relative relationship between the timestamp of the record to be associated and the reference source timestamp. The larger the residual, the more likely there is to be abnormal timestamp drift or nonlinear change between the recording time and the physical event time.

[0065] Since step S3 can reliably and naturally acquire several pairs of physical event records (i.e., matching pairs of three consecutive beacon pulses), the witness time offset can be calculated for each record pair, and then the median or average value can be reasonably taken as the witness time offset. The final result used in the consistency residual calculation naturally suppresses the influence of single measurement noise: specifically, if obtained from three pairs of records... Second, Second, The median of 0.210 seconds is taken as the final witness time offset. For simplicity, this embodiment describes the case of a single pair of records, while the processing logic for multiple pairs of records is completely the same.

[0066] Because the consistency residual was calculated In the future, it will be compared with the dynamically obtained dynamic alignment threshold to determine the type of association judgment. Therefore, it is natural to store the consistency residual value in the historical residual queues of the reference source and the source to be calibrated, for use in the subsequent asymptotic drift mode identification. More importantly, the calculation of the consistency residual is strictly based on the measurable timestamp value and the physical event recording time, and does not require any external absolute time reference. Therefore, it has excellent engineering feasibility and real-time performance.

[0067] Step S5: Determine the association based on the consistency residuals;

[0068] Since step S5 uses the consistency residuals obtained in step S4 Since the association determination is based on the data, it is natural to introduce a dynamically determined alignment threshold, which is directly related to the proportion of data sources in the penalty period in the system at that time to the total number of data sources.

[0069] Specifically, the system first obtains the set of data sources that are currently in the penalty period. and the total number of all data sources in the system. The definition of the penalty period comes from the penalty and recovery mechanism: when a data source is penalized due to drift contamination, the penalty period ends at the specified time. The current time is set to add a preset penalty period (e.g., 86400 seconds). Before this time, the data source is considered to be within the penalty period; the system maintains a counter in real time. ,equal The number of elements in the data; the proportion p of the current penalty period data source to the total number of data sources is calculated by the following formula: ;

[0070] Subsequently, the dynamic alignment threshold Based on the basic threshold Determined together with the penalty ratio p: ;

[0071] Base threshold This is a preset fundamental constant used in this system, so it is calibrated naturally and reasonably in this embodiment: after the system starts up and runs for 1 hour, the consistency residual values ​​of all relevant requests are collected, and then the 99th percentile is taken as the basic threshold. If reliable calibration is not possible, the default value of 0.002 seconds is used. More importantly, the given formula allows the dynamic alignment threshold to adjust as the penalty ratio p increases from 0 to 0.5. from linearly increase to When the system has no penalty (p=0) When half of the data sources are in the penalty period ( )hour, Therefore, its design philosophy is very clear: when the overall system synchronization is poor, the tolerance is automatically relaxed to avoid a large number of associations being incorrectly judged as suspicious due to overly strict thresholds, while when the system is healthy, strict thresholds are maintained to strictly ensure the quality of associations.

[0072] Since the dynamic alignment threshold has already been obtained Therefore, this step naturally involves calculating the consistency residual obtained in step S4. The numerical comparison is performed with this threshold, as detailed below.

[0073] like If so, it is determined to be a trustworthy association;

[0074] like If so, it is determined to be a suspicious association.

[0075] The following describes the subsequent processing for the two judgment results:

[0076] Timestamp Correction under Trusted Correlation: When a trusted correlation is determined, it indicates that the timestamp of the source to be calibrated is in good agreement with the timestamp of the reference source, and highly matches the relative deviation revealed by the witnessed physical event; at this time, the timestamp of the source to be calibrated is corrected using the witnessed time offset; let the witnessed time offset calculated in step S4 be... The timestamp of the reference source is Then the corrected timestamp of the calibration source Calculate using the following formula: ;

[0077] The essence of this correction is to replace the deviation in the original timestamp of the source to be calibrated with the true relative deviation revealed by the physical event; after the correction, the system uses the timestamp of the reference source. and the corrected source timestamp to be calibrated The time stamp output after this association and alignment will be used for subsequent data fusion or time series analysis; at the same time, the system will output the consistency residuals. Push the data into the historical residual queues of the reference source and the source to be calibrated (overwrite the earliest record if the queue is full) so that subsequent steps can perform progressive drift mode identification.

[0078] Outputting time intervals under suspicious associations: Since the consistency residual must be greater than the current dynamic alignment threshold when a suspicious association is determined, the timestamp of the source to be calibrated cannot be directly adjusted to a single definite value. Therefore, the natural and reasonable approach is to output the time alignment result with an uncertain interval.

[0079] Because the system first obtains the timestamp of the reference source timestamp of the source to be calibrated Find its smaller and larger values, so it is natural to express it as follows:

[0080] , ;

[0081] The system obtains the first trust risk value calculated in step S2. Second Trust Risk Value For the current association, the trust risk values ​​corresponding to the reference source and the source to be calibrated are respectively... and The system presets a drift tolerance coefficient C, which is a constant greater than 0. In this embodiment, it is obtained through on-site calibration: after the system is initially deployed and no drift is confirmed, it runs continuously for 24 hours, and the absolute drift distribution of all deepest sources is measured, and the 99.9th percentile is taken. And calculate the average trust risk value of the deepest source at this time. ,but If calibration is not possible, a conservative default value of C=0.005 seconds can be used.

[0082] Due to drift compensation amount It can be naturally and reasonably defined as the product of the sum of trust risk values ​​and the drift tolerance coefficient. However, considering the drift compensation amount Excessive size would render the time interval unreliable, therefore the system sets its upper limit to [value missing]. Seconds; if exceeded, the upper limit value will be used instead.

[0083] Since the compensation amount sought is an estimate of the maximum uncertainty range of the timestamp under the current level of trust risk, it is natural to update the upper bound of the interval to the sum of the original larger value and the drift compensation amount: .

[0084] Since the system outputs the uncertain interval, it sets the consistency residual for this operation. (Regardless of whether the threshold is exceeded) it is naturally and appropriately pushed into the historical residual queue of the reference source and the source to be calibrated. Therefore, under the suspicious correlation branch, the system does not modify the timestamp of the source to be calibrated, but only retains its original timestamp and adds the obtained interval information, so that the upper layer application can handle it reasonably according to its specific needs (take the midpoint of the interval as the best estimate, or treat the interval itself as a constraint).

[0085] Continuous updates to the dynamic alignment threshold: It is worth noting that the dynamic alignment threshold... It is not a fixed value, but is updated in real time as the data source set changes during the penalty period; the system recalculates p and before each judgment. This ensures that the threshold always reflects the overall failure level of the current system; for example, when a data source is marked as a drift contamination source due to periodic testing and enters the penalty period, Increase, It will then grow larger; when the penalty period ends and the system recovers, The corresponding reduction; this adaptive mechanism ensures a dynamic balance between the sensitivity and stability of the association determination.

[0086] Since step S5 has completed the judgment and subsequent processing discussed, this step naturally ends. At this time, the system returns the obtained results (corrected timestamp pairs under trusted association, or uncertain intervals under suspicious association) to the caller, and updates the historical residual queue and global penalty status of the relevant data source.

[0087] Furthermore, since the system periodically and hierarchically executes the source-tracing prisoner test and punishment recovery mechanism, and this mechanism is unrelated to the real-time correlation determination process (steps S1-S5) mentioned above, it is natural to design it as a background task to run, specifically to detect collective time drift in the synchronization tree.

[0088] The triggering of periodic inspections and the selection of inspection sources can be described naturally and hierarchically: the system first sets a calibration cycle. In this example, we take it as Seconds (1 hour), then every such time interval To trigger a Prisoner's Check, first iterate through all data sources and collect the current trust hop count for each data source. This refers to the integer that is initialized in step S1 and then dynamically updated in each subsequent penalty step. All data sources are then sorted from largest to smallest based on their trust hop count. Finally, the K data sources with the largest trust hop counts are selected as the set of sources to be inspected, L. The formula for the selected K value is given below:

[0089] ;

[0090] That is, at least 10 should be selected; if 5% of the total number of data sources exceeds 10, then only 5% should be selected; and the maximum number of trust hops for all data sources should be considered. If the synchronization tree is too shallow and there are no deep subtrees worth checking, then this check will be skipped.

[0091] Deployment and characteristics of the physical reference benchmark: The system pre-deploys a physical reference benchmark R independent of all synchronization trees; this reference benchmark can be a high-precision real-time clock module (e.g., an RTC with a temperature-compensated crystal oscillator, with an annual drift of less than ±1 minute) or an external stable pulse source; this reference benchmark has the following fixed parameters: trust hop count. (A negative value indicates that it is independent of the synchronization tree), dynamic quality factor (Maximum confidence); This reference benchmark also receives pulse signals broadcast by the public beacon generator and records the sequence number and local reception time of each beacon pulse, just like a normal data source, and stores them in its event buffer; Since the reference benchmark does not participate in any synchronization tree, the time it records is completely independent of the time system of the source under test, and can be used as an approximate substitute for the absolute reference.

[0092] Calculation of absolute drift: Since each source S in the source set L needs to be processed uniformly and hierarchically, it can be naturally and appropriately described as follows: Extract the beacon pulse event matching pairs that are sensed by both the event buffer of S and the event buffer of the reference base R. The matching method used is exactly the same as in step S3, that is, to associate according to the sequence number of the beacon pulse, and then select the most recent time period (the most recent time period). Within a given time (seconds), select a maximum of 3 matching pairs (if there are fewer than 3 pairs, select all available pairs, but at least one pair must be present). , Otherwise, skip this S).

[0093] Absolute drift Defined as the median of the difference between the record time of the source under inspection and the record time of the reference standard: ;

[0094] Since the text uses M to represent the total number of matching pairs, it naturally and appropriately explains the purpose of the median: to eliminate the influence of a single abnormal matching pair. That is, the median of 0.108 seconds can be directly and reasonably obtained from the differences of 0.105 seconds, 0.108 seconds, and 0.112 seconds between the three matching pairs, while the difference between the three matching pairs is taken when there is only a single matching pair.

[0095] Calculation of maximum allowable drift: Due to the maximum allowable drift... It is determined by the trust risk value of the source S to be inspected. The drift tolerance coefficient C is determined by the selected drift tolerance coefficient, so naturally, we must first use the formula given in step S2 to apply the current S value. , Recalculate the trust risk value : .

[0096] The discussion clearly and systematically reveals the value of trust risk. In actual calculations, the value is strictly limited to the range of 0 to 100; values ​​exceeding 100 are treated as 100. Therefore, the corresponding maximum drift is... The upper limit is naturally set at 100×C, for example, 0.5 seconds, which naturally avoids the problem of excessively relaxing the judgment threshold when the trust risk value of deep nodes is too high.

[0097] Due to the prevention of zero constant Therefore, the drift tolerance coefficient C can be naturally and reasonably obtained from the field calibration: that is, when the system is initially deployed, it is first determined that there is no drift, and then it is run continuously for 24 hours, and the absolute values ​​of the absolute drift of all the deepest sources are measured, with the 99.9th percentile as the threshold. Then, the arithmetic mean of the trust risk values ​​of the deepest source measured is taken as the basis for the calculation. ;but If calibration is not possible, a conservative default value of C=0.005 seconds will be used, and the maximum allowable drift will be: .

[0098] The physical meaning of this formula is that the greater the trust risk value, the deeper the synchronization level, and the smaller the quality factor, the greater the upper limit of the absolute drift allowed by the system. Therefore, it is not advisable to make overly strict judgments on deep nodes.

[0099] Identification of drifting pollution sources and generation of pollution aggregates: Comparison of the absolute value of absolute drift with the maximum allowable drift:

[0100] like If so, it is determined that the source S under test has undergone collective drift and is marked as a drift pollution source;

[0101] Logically, it is natural and reasonable to conclude that if this is not the case, then the timestamp of S is considered to be within the normal range.

[0102] For each S marked as a drift contamination source, the system further performs source tracing: tracing upwards along the parent node chain of the synchronization tree (through the data stored in each data source). ), until the root anchor point is encountered ( ) or nodes that have already been added to the polluted set; at the same time, recursively traverse all descendant nodes of S downwards (using reverse indexes). (Obtain all child nodes and continue downwards); add all data sources along the tracing path (including S itself, all its ancestor nodes, and all its descendant nodes) to the drift contamination set. .

[0103] To illustrate with a concrete example: the root anchor point in the synchronization tree is a ( ), a has child node b ( b has child node c ( c has child node d ( Periodic testing revealed that the absolute drift of d exceeded the maximum allowable drift, thus d was naturally identified as the source of drift contamination. Tracing back to c, b, and a, and since a has no descendant nodes, the final contamination set was determined to be... This naturally and aptly leads to the logic of "ancestors being held responsible for their descendants' mistakes": collective drift must originate from a hidden fault somewhere upstream.

[0104] Penalty steps: Due to drift contamination of the collection Since it has already been generated, the system will immediately perform a penalty step, that is, update each data source T contained in the polluted set in sequence.

[0105] Dynamic quality factor update: (Current value halved);

[0106] Trust hop count increased: ;

[0107] Set the end time of the penalty period: Current system absolute time + ,in Seconds (24 hours);

[0108] Add T to the global penalty period set. ;

[0109] After updating T itself, the system traverses all its descendant nodes down the synchronization tree, starting from T; for each descendant node U, the following is executed:

[0110] Trust hop count increased: (The increase is the same as T, i.e., an increase of 1).

[0111] Dynamic quality factor recalculation: From this point, it can be seen very naturally and clearly that... The dynamic quality factor after the parent node update is determined by setting the synchronization method type of the selected U to 0.95 (hardware synchronization), 0.85 (software synchronization), or 0.70 (calculated synchronization). The recalculation method is completely consistent with the initial attenuation rule used in step S1.

[0112] Because the increase in trust hop count has a clear global effect: if the pollution set contains several nodes on the same link (i.e., ancestors and descendants have been marked), the system will perform an increase operation on each node individually. However, it is necessary to avoid duplicate increases. Therefore, this embodiment naturally and reasonably adopts the method of processing the nodes in the pollution set in ascending order of trust hop count. That is, each node only increases its trust hop count once when it is processed. Its descendant nodes will also increase their trust hop count when propagating downwards. However, since the ancestor node has already propagated once, the descendant node may be increased repeatedly.

[0113] To avoid this problem, in practice, the polluted set should first be sorted in descending order of trust hop count (starting from the deepest node). After penalizing the deepest node and propagating downwards, its ancestor nodes should be excluded from subsequent individual processing, or duplicate penalties can be skipped by detecting that the node is already in the penalty period. For the sake of simplicity, this embodiment assumes that the nodes in the polluted set do not form ancestor-descendant relationships with each other (i.e., only the deepest node is marked for propagation). In reality, since tracing back to the source will add the entire chain to the set, a strategy of "only propagating to the deepest node in each connected branch" can be adopted. Regardless of the specific implementation, the final effect is that the trust hop count of all polluted nodes and their descendants increases by 1, and the quality factor is downgraded according to the decay rule.

[0114] Recovery steps after the penalty period expires: The system simultaneously runs an independent scheduled task, with a preset recovery check cycle at each interval. Seconds, iterate through all data sources and check the end time of the penalty period for each data source. If the current system time If the penalty period for the data source has expired, proceed with the recovery steps:

[0115] Dynamic quality factor update: That is, multiply it by 1.5, but the upper limit is 1.0 (initial maximum confidence).

[0116] Trust hop count update: That is, reduce by 1, but not less than the initial trust hop count recorded by the source in step S1. ;

[0117] Clear penalty markers: and T from Removed from the middle.

[0118] Starting from node T, traverse down the synchronization tree to all its descendant nodes; for each descendant node U, execute:

[0119] Trust hop count reduced: That is, reduce by 1 but not less than its initial trust hop count;

[0120] Dynamic quality factor recalculation: Starting from the updated dynamic quality factor of the parent node, recalculate it using the attenuation factor corresponding to the selected synchronization mode type. The formula used is exactly the same as that used during initialization.

[0121] For example, a data source before the penalty , Initial trust hop count After punishment , When it recovers after 24 hours, Trust hop count The node is restored to its initial value; its descendant nodes are also adjusted accordingly.

[0122] Since the proposed periodic inspection, penalty, and recovery mechanisms can naturally and reasonably detect collective time drift and apply weight reduction penalties to drift pollution sources and their entire subtrees, the trust level can be restored in a timely and stable manner after the drift is eliminated, thus ensuring the long-term stability of the multi-source data association and integration system.

[0123] The above-described embodiment 1 constructs a complete collective time drift detection and isolation system by introducing a dynamic selection mechanism for the benchmark source based on trust risk value, a positive interaction verification method based on public beacon pulses, and a periodically executed source-tracing prisoner's dilemma test. This system does not rely on the absolute time reference of any single data source; it only uses the relative deviation between each data source and an independent physical reference to accurately identify the overall drift caused by the failure of hidden nodes in the synchronization tree. Through a penalty and recovery mechanism, the drift source and its entire subtree are dynamically downweighted. Thus, when the upstream node experiences hidden faults such as crystal oscillator temperature drift or time signal interference, the system can promptly block the step-by-step propagation of errors along the synchronization tree, preventing the self-consistency within the subtree from masking the true time deviation. This ensures that the time benchmark on which subsequent multi-source fusion, causal inference, and early warning decisions rely is always consistent with the real physical process. Compared with the shortcomings of existing technologies where static trust levels cannot detect collective self-consistent drift, this solution fundamentally solves the problem of irreversible inaccuracy of the association integration results on the absolute time axis caused by the systematic skewness of timestamps, significantly improving the stability and reliability of multi-source data association integration in complex engineering environments.

[0124] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

[0125] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0126] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A multi-source data association and integration method based on artificial intelligence, characterized in that, include: Obtain multiple data sources and their synchronization topology information, wherein the synchronization topology information includes at least the synchronization parent node identifier and synchronization method type of each data source; based on the synchronization topology information, initialize the trust depth parameter for each data source; In response to the request to perform a time-series correlation between the first data source and the second data source, a first trust risk value and a second trust risk value are generated for the first data source and the second data source respectively based on the trust depth parameter; The data source with the smaller value between the first trust risk value and the second trust risk value is determined as the reference source, and the other data source is determined as the source to be calibrated; Acquire at least one pair of physical event records jointly sensed by the reference source and the source to be calibrated, each pair of physical event records containing a first recording time of the same physical event on the reference source and a second recording time on the source to be calibrated; The nominal time offset is calculated based on the timestamp of the reference source and the timestamp of the source to be calibrated; the witness time offset is calculated based on the first recording time and the second recording time; and the absolute value of the difference between the nominal time offset and the witness time offset is calculated as the consistency residual. Association determination is based on consistent residuals.

2. The multi-source data association and integration method based on artificial intelligence according to claim 1, characterized in that, Trust depth parameters include trust hop count and dynamic quality factor; The specific steps for initializing the trust depth parameter for each data source include: Initialize the trust hop count of the root anchor point without a parent node in the synchronization tree to 0, and initialize the dynamic quality factor to 1.0; For any child node, obtain the trust hop count and dynamic quality factor of its parent node, and initialize the trust hop count of the child node to the trust hop count of the parent node plus 1; Get the synchronization mode type of the child node. If the synchronization mode type is hardware synchronization, initialize the dynamic quality factor to 0.95 times that of the parent node; if the synchronization mode type is software synchronization, initialize it to 0.85 times; if the synchronization mode type is calculated synchronization, initialize it to 0.70 times.

3. The multi-source data association and integration method based on artificial intelligence according to claim 2, characterized in that, The specific steps for generating the first trust risk value and the second trust risk value based on the trust depth parameter include: Obtain the first trust hop count and the first dynamic quality factor of the first data source, and obtain the second trust hop count and the second dynamic quality factor of the second data source; The first trust hop count is divided by the sum of the first dynamic quality factor and the preset zero-prevention constant to obtain the first trust risk value. Similarly, the second trust hop count is divided by the sum of the second dynamic quality factor and the preset zero-prevention constant to obtain the second trust risk value. The first trust risk value and the second trust risk value are used to characterize the unreliability of the data source timestamp.

4. The multi-source data association and integration method based on artificial intelligence according to claim 1, characterized in that, The specific steps for obtaining the physical event record pair jointly sensed by the reference source and the source to be calibrated include: Deploy a public beacon generator that broadcasts a pulse signal carrying an incrementing sequence number at a fixed period; Each data source receives the pulse signal and records the local reception time and corresponding sequence number, storing them in its respective event buffer; In response to the association request, unused pulse records are selected from the reference source event buffer in ascending order of sequence number, and their sequence number and first recording time are extracted; pulse records with the same sequence number are searched in the event buffer of the source to be calibrated, and their second recording time is extracted; the first recording time and the second recording time are regarded as the recording time pair of the same physical event on the reference source and the source to be calibrated.

5. The multi-source data association and integration method based on artificial intelligence according to claim 1, characterized in that, The specific steps for determining association based on consistent residuals include: Obtain the current dynamic alignment threshold, which is dynamically generated based on the proportion of the number of data sources currently in the penalty period to the total number of data sources, and is positively correlated with this proportion; Compare the consistency residual with the current dynamic alignment threshold; If the consistency residual is less than or equal to the dynamic alignment threshold, it is determined to be a reliable association, and subsequent timestamp correction steps are performed; if the consistency residual is greater than the dynamic alignment threshold, it is determined to be a suspicious association, and time alignment results with uncertain intervals are output.

6. The multi-source data association and integration method based on artificial intelligence according to claim 5, characterized in that, The timestamp correction steps under trusted association include: Obtain the timestamp of the reference source and the witness time offset; Add the timestamp of the reference source to the witness time offset to obtain the corrected timestamp of the source to be calibrated; The timestamp of the reference source and the corrected timestamp are used as the associated and aligned timestamp output.

7. The multi-source data association and integration method based on artificial intelligence according to claim 5, characterized in that, The specific steps for outputting time-aligned results with uncertain intervals under suspicious correlations include: Obtain the timestamp of the reference source and the timestamp of the source to be calibrated, take the smaller of the two as the lower bound of the interval, and take the larger of the two as the upper bound of the interval. Obtain the first trust risk value and the second trust risk value, add the two together and multiply by the preset drift tolerance coefficient to obtain the drift compensation amount; The upper bound of the interval is updated to the sum of the larger value and the drift compensation amount, and the updated lower bound and upper bound of the interval are used to form a time-aligned interval output.

8. The multi-source data association and integration method based on artificial intelligence according to claim 1, characterized in that, It also includes the periodically performed Prisoner's Dilemma step: Every preset calibration cycle, a preset number of data sources with the highest number of trusted hops are selected as the set of data sources to be inspected. Obtain a physical reference base independent of the synchronization tree, which has a fixed negative trust hop count and a fixed dynamic quality factor of 1.0; For each source in the set of sources to be inspected, obtain the matching pair of beacon pulse events jointly sensed by the source to be inspected and the physical reference reference, and calculate the absolute value of the difference between the recording time of the source to be inspected and the recording time of the physical reference reference as the absolute drift amount. The maximum allowable drift is calculated based on the trust hop count and dynamic quality factor of the source under test; If the absolute drift amount is greater than the maximum allowable drift amount, the source to be detected is marked as a drift contamination source, and the data is traced upwards along the synchronization tree to the root anchor point and downwards to all descendant nodes, and all data sources on the tracing path are added to the drift contamination set.

9. The multi-source data association and integration method based on artificial intelligence according to claim 8, characterized in that, It also includes penalty steps performed in response to the drift contamination set: For each data source in the drift contamination set, multiply its dynamic quality factor by 0.5, increase its trust hop count by 1, and set the penalty period end time to the current time plus the preset penalty duration; Starting from the data source, traverse down the synchronization tree to all its descendant nodes. For each descendant node, perform the following: increment its trust hop count by 1, and recalculate its dynamic quality factor based on the updated dynamic quality factor of its parent node. The calculation method is the same as the decay rule during initialization.

10. The multi-source data association and integration method based on artificial intelligence according to claim 9, characterized in that, It also includes recovery steps after the penalty period expires: A preset recovery check cycle is performed at each interval, and all data sources are traversed to obtain the end time of the penalty period for each data source. If the current time is greater than or equal to the end time of the penalty period of the data source, then update the dynamic quality factor of the data source to the smaller value between 1.0 and the current dynamic quality factor multiplied by 1.5, and update the trust hop count of the data source to the larger value between the initial trust hop count and the current trust hop count minus 1. Starting from the selected data source, traverse the synchronization tree from top to bottom, and process each descendant node as follows: first, decrement the trust hop count by 1, but not below its initial trust hop count, and then recalculate its dynamic quality factor in conjunction with the updated dynamic quality factor of its parent node. The method used is the decay rule adopted during initialization.

Images