A data fusion method and device based on two-stage temporal registration

Abstract

The present invention discloses a data fusion method and device based on two-stage time registration. A specific embodiment of the method includes a plurality of nodes, each of which is in communication connection with a first device; the method is applied to the first device; based on the first local state estimation value of each node for the target observation object within the same time window, a first state estimation queue is generated; for any first local state estimation value in the first state estimation queue: the first local state estimation value is time-registered and fused to generate a second registered state estimation value; based on the second registered state estimation value corresponding to each first local state estimation value in the first state estimation queue, a fusion process is performed to output a global state estimation value. Therefore, this embodiment uses the two-stage weighted asynchronous fusion technology of Kalman filtering to reduce the impact of communication delay on fusion accuracy, thereby improving the accuracy and real-time performance of the state estimation value of the target observation object in high-speed scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a data fusion method and device based on two-stage time registration. Background Art

[0002] Multi-sensor fusion integrates data from multiple sensors to obtain more comprehensive and accurate information. Its advantage lies in overcoming the limitations of individual sensors and improving system robustness and reliability. In target tracking, multi-sensor fusion offers significant advantages: high accuracy, robustness, wide-area perception, and information integrity. Multi-sensor fusion can be broadly categorized into multi-machine data fusion and multimodal data fusion. Multimodal data fusion emphasizes the integration of sensor data within a single machine, generally utilizing raw data from multiple sensors. If a single sensor fails or is interfered with, other sensors can provide supplementary information, ensuring the system can continue tracking the target. Furthermore, multimodal sensors have different measurement ranges and information types, effectively expanding the perception range and improving information integrity. Radars can measure the distance to the target and its surroundings, while cameras can perceive changes in the appearance and texture of the target and its surroundings. Spatiotemporal synchronization is a key foundation for multi-sensor data fusion. When multiple sensors are deployed on a single platform, data alignment can be achieved through hard synchronization methods such as pulse triggering.

[0003] Limited by the inherent properties of a single-machine platform, target tracking and search range is limited, resulting in insufficient robustness and efficiency. Multi-machine platforms, with their flexible distribution, can effectively offset these shortcomings. Unlike hard-time synchronization methods on single-machine platforms, multi-machine platforms achieve global time synchronization solely through communication synchronization triggering, relying on the ideal centralized assumption of reliable full connectivity and low communication latency. In real-world scenarios, centralized triggering restricts the movement of multiple machines, potentially leading to target tracking failures.

[0004] Analysis reveals three main causes of multi-sensor data asynchrony: 1) asynchronous time bases; 2) different sensor sampling frequencies and inconsistent sensor sampling start times; and 3) communication delays. Despite time base synchronization, communication delays can lead to time differences in the synchronization trigger, resulting in different sensor sampling start times. To address these differences in sensor sampling times, existing methods, such as curve fitting, spline interpolation, and interpolation and extrapolation, can be used to align sampled data. The least squares virtual method and interpolation and extrapolation methods achieve data alignment under different sampling frequencies in a synchronous and asynchronous manner, respectively. However, interpolation and extrapolation and the least squares virtual method are only applicable to ideal motion models such as uniform velocity and uniform acceleration. Curve fitting and spline interpolation methods only provide good approximations for data interpolation, but these methods lack real-time performance. Summary of the Invention

[0005] The present invention provides a data fusion method and device based on two-stage time registration; the method can solve the distributed asynchronous data fusion problem caused by communication delay and inherent problems of sensors, thereby improving the accuracy and real-time performance of the state estimation value of the target observation object in high-speed scenarios.

[0006] According to a first aspect of an embodiment of the present invention, a data fusion method based on two-stage time alignment is provided, the method comprising a plurality of nodes for state estimation of an observed object; each of the nodes is in communication connection with a first device; the method is applied to the first device; comprising: generating a first state estimation queue based on a first local state estimation value of each of the nodes for a target observed object within the same time window; wherein each of the nodes sends a local state estimation value of the target observed object to the first device once within the same time window; for any first local state estimation value in the first state estimation queue: obtaining a sampling moment of the first local state estimation value; if the first local state estimation value meets a preset condition, obtaining a second state estimation value of the target observed object from the paired node; Obtain a second local state estimate value closest to the sampling moment in the sequence; based on the sampling moment, perform time alignment on the second local state estimate value to obtain a first aligned state estimate value; fuse the first aligned state estimate value and the first local state estimate value to output a fused local state estimate value; based on the fusion moment, perform time alignment on the fused local state estimate value to generate a second aligned state estimate value; wherein, the pairing node is used to indicate a node that can match the trajectory of the target object with the target node corresponding to the first local state estimate value; fuse the second aligned state estimate value corresponding to each first local state estimate value in the first state estimation queue, and output the global state estimate value corresponding to the target observation object at the fusion moment.

[0007] Optionally, the method also includes: if the first local state estimation value does not meet the preset conditions, performing time alignment on the first local state estimation value based on the fusion moment to generate a first local state estimation value after alignment; and determining the first local state estimation value after alignment as the second alignment state estimation value.

[0008] Optionally, the method further includes: obtaining a number of tracks generated by each of the several nodes within a preset time; wherein the tracks are used to indicate the motion trajectory generated by the node for the observable observation object, and each of the observation objects has a corresponding track; each track is formed by a number of local state estimation values ​​arranged in chronological order; selecting any two nodes from the several nodes as a pairing group: pairing a number of first tracks corresponding to the target node in the pairing group and a number of second tracks corresponding to the pairing node to generate paired tracks; based on the paired tracks, determining the target observation object corresponding to the pairing group.

[0009] Optionally, the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue is fused, and the global state estimation value corresponding to the target observation object at the fusion moment is output; including: obtaining a fusion cache queue based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue; for any second registration state estimation value in the fusion cache queue: obtaining the second registration covariance matrix corresponding to the second registration state estimation value; determining the information matrix and the first weight corresponding to the information matrix based on the second registration covariance matrix; obtaining several information matrices based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue; for any information matrix among the several information matrices: applying the corresponding first weight to the information matrix, and summing up the information matrices after applying the first weight to generate an optimal information matrix; optimizing the second registration state estimation value based on the optimal information matrix, the information matrix and the first weight corresponding to each second registration state estimation value in the fusion cache queue, and outputting the global state estimation value corresponding to the target observation object at the fusion moment.

[0010] According to the second aspect of an embodiment of the present invention, a data fusion method based on two-stage time alignment is also provided, which is applied to the node: the node includes at least one sensor; each of the sensors is used to observe at least one observation object; the method includes: associating the current measurement result generated by the node with the target observation object to generate a target measurement result; determining a corresponding filtering method based on the sensor type corresponding to the current measurement result; filtering the target measurement result based on the filtering method, and outputting a filtered measurement result; optimizing the current state of the target observation object based on the current initialized state value of the target observation object and the filtered measurement result, and outputting a local state estimate corresponding to the target observation object at the current moment and a predicted state value at the next moment.

[0011] Optionally, associating the current measurement result generated by the node with the target object to generate the target measurement result includes: receiving the measurement result corresponding to each of all the observation objects that can be observed by the node to generate at least one current measurement result; for any current measurement result among the at least one current measurement result: taking the observation object corresponding to the current measurement result as the target observation object, and obtaining the predicted measurement result corresponding to the predicted state of the target observation object at the current moment; determining the Euclidean distance between the current measurement result and the predicted measurement result; if the Euclidean distance is not greater than a gated threshold, determining that the current measurement result is associated with the target observation object; counting the number of current measurement results associated with the target observation object to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, taking the current measurement result as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, determining the distance between the observation object of each current measurement result and the target observation object; and selecting the current measurement result with the smallest Euclidean distance from the at least two current measurement results as the target measurement result of the target observation object.

[0012] Optionally, the method also includes: receiving target observation information for the target observation object sent by the sensor; if there is prior information of the target observation object, performing attribute identification on the target observation object based on the target observation information and the prior information to generate a target matching result; generating a target trajectory based on the target matching result; if there is no prior information of the target observation object and the data type of the target observation information meets the observability requirement, performing Hough detection on the target observation information to generate a target trajectory; if there is no prior information of the target observation object and the data type of the target observation information does not meet the observability requirement, after receiving the local state estimation value for the target object sent by other nodes, performing trajectory similarity matching on the target observation information and the local state estimation value to generate a target trajectory; obtaining the predicted state value for the target observation object at the current moment output at the previous moment from the target trajectory; and using the predicted state value as the current initialization state value.

[0013] According to the third aspect of an embodiment of the present invention, there is also provided a data fusion device based on two-stage time alignment, the device comprising a plurality of nodes for state estimation of an observed object; each of the nodes is in communication connection with a first device; the method is applied to the first device; comprising: a first generating module for generating a first state estimation queue based on a first local state estimation value of each of the nodes for a target observed object within the same time window; wherein each of the nodes sends a local state estimation value of the target observed object to the first device once within the same time window; a second generating module for obtaining a sampling moment of the first local state estimation value for any first local state estimation value in the first state estimation queue; if the first local state estimation value meets a preset condition, the first local state estimation value for the target observed object from the paired node is sent to the first device once. A second local state estimation value closest to the sampling moment is obtained from the two state estimation sequences; based on the sampling moment, the second local state estimation value is time-aligned to obtain a first aligned state estimation value; the first aligned state estimation value and the first local state estimation value are fused to output a fused local state estimation value; based on the fusion moment, the fused local state estimation value is time-aligned to generate a second aligned state estimation value; wherein the pairing node is used to indicate a node that can perform trajectory matching with the target node corresponding to the first local state estimation value for the target object; a fusion processing module is used to perform fusion processing based on the second aligned state estimation value corresponding to each first local state estimation value in the first state estimation queue, and output a global state estimation value corresponding to the target observation object at the fusion moment.

[0014] According to the fourth aspect of an embodiment of the present invention, a data fusion state based on two-stage time alignment is also provided, which is applied to the node: the node includes at least one sensor; each sensor is used to observe at least one observation object; and includes: a generation module, which is used to associate the current measurement result generated by the node with the target observation object to generate a target measurement result; a determination module, which is used to determine the corresponding filtering method based on the sensor type corresponding to the current measurement result; a filtering module, which is used to filter the target measurement result based on the filtering method and output the filtered measurement result; an optimization module, which is used to optimize the current state of the target observation object according to the current initialization state value of the target observation object and the filtered measurement result, and output the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

[0015] According to a fifth aspect of an embodiment of the present invention, a computer-readable medium is further provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0016] According to the sixth aspect of an embodiment of the present invention, an electronic device is also provided, which includes: a processor; a memory for storing instructions executable by the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in the first aspect or the second aspect.

[0017] An embodiment of the present invention provides a data fusion method and device based on two-stage time alignment, the method comprising a plurality of nodes for state estimation of an observed object; each of the nodes is in communication connection with a first device; the method is applied to the first device; comprising: firstly, generating a first state estimation queue based on a first local state estimation value of each of the nodes for a target observed object within a same time window; wherein each of the nodes sends a local state estimation value of the target observed object to the first device once within the same time window; secondly, for any first local state estimation value in the first state estimation queue: obtaining a sampling moment of the first local state estimation value; if the first local state estimation value meets a preset condition, obtaining a second state estimation value of the target observed object from a paired node; The second local state estimate closest to the acquisition time is obtained from the estimation sequence; based on the sampling time, the second local state estimate is time-aligned to obtain a first registered state estimate; the first registered state estimate and the first local state estimate are fused to output a fused local state estimate; based on the fusion time, the fused local state estimate is time-aligned to generate a second registered state estimate; wherein the paired node is used to indicate the node that can perform trajectory matching with the target node corresponding to the first local state estimate; finally, the second registered state estimate corresponding to each first local state estimate in the first state estimation queue is fused to output the global state estimate corresponding to the target observation object at the fusion time. The Kalman filtering method of this embodiment realizes the alignment and synchronization of the sampled data, and then uses a two-stage weighted asynchronous fusion technology to reduce the impact of communication delay on fusion accuracy; thereby improving the accuracy and real-time performance of the state estimate of the target observation object in high-speed scenarios, and effectively solving the problem of significantly increased registration error of existing time registration technology in high-speed scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0019] Figure 1 A flow chart of a data fusion method based on two-stage temporal registration provided by one embodiment of the present invention;

[0020] Figure 2 A flowchart of a data fusion method based on two-stage temporal registration provided by another embodiment of the present invention;

[0021] Figure 3 A schematic structural diagram of a data fusion device based on two-stage temporal registration provided by one embodiment of the present invention;

[0022] Figure 4 A schematic structural diagram of a data fusion device based on two-stage time registration is provided in another embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purposes, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0024] like Figure 1 FIG. 1 is a flow chart of a data fusion method based on two-stage temporal registration according to an embodiment of the present invention;

[0025] A data fusion method based on two-stage temporal registration includes a plurality of nodes for performing state estimation on an observed object; each of the nodes is communicatively connected to a first device; the method is applied to the first device and includes at least the following steps:

[0026] S101, generating a first state estimation queue based on a first local state estimation value of a target observation object by each node within a same time window; wherein each node sends a local state estimation value of the target observation object to a first device once within the same time window;

[0027] S102, for any first local state estimate value in the first state estimate queue: obtain the sampling time of the first local state estimate value; if the first local state estimate value meets the preset conditions, obtain the second local state estimate value closest to the sampling time from the second state estimate sequence of the paired node for the target observation object; based on the sampling time, perform time alignment on the second local state estimate value to obtain a first aligned state estimate value; fuse the first aligned state estimate value and the first local state estimate value to output a aligned local state estimate value; based on the fusion time, perform time alignment on the aligned local state estimate value to generate a second aligned state estimate value; wherein the paired node is used to indicate a node that can perform trajectory matching with the target node corresponding to the first local state estimate value for the target object;

[0028] S103, performing fusion processing based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and outputting the global state estimation value corresponding to the target observation object at the fusion moment.

[0029] It should be noted that in this embodiment, there is no limitation on the methods of temporal registration and data fusion, which can be implemented based on a training model or mathematical algorithm. For example, the Kalman filter method is used to achieve alignment and synchronization of sampled data, and then a two-stage weighted asynchronous fusion technique is used for data fusion.

[0030] In S101, the first device communicates with each of the nodes, and each node sends the local state estimation value corresponding to each observed object that can be observed to the first device. The first device receives the local state estimation value sent by each node and stores the received local state estimation value in a cache queue to generate a cache queue. Since there is a communication delay between each node and the first device, even if each node sends the first local state estimation value corresponding to the target observation object to the first device at the same time, the first device receives the first local state estimation value sent by each node at different times due to the communication delay between the node and the first device. Obtain the first local state estimation value of each node for the target observation object in the same time window, and obtain the first state estimation queue

[0031] It should be noted that the local state estimation value of each node is determined by the node based on the observation information collected by the sensor on the node.

[0032] In S102, exemplarily, a plurality of tracks generated by each of the plurality of nodes within a preset time are obtained; wherein the tracks are used to indicate the motion trajectory generated by the node for the observable observation object, and each of the observation objects has a corresponding track; each track is formed by a plurality of local state estimation values ​​arranged in chronological order; any two nodes are selected from the plurality of nodes as a pairing group: a plurality of first tracks corresponding to the target nodes in the pairing group and a plurality of second tracks corresponding to the pairing nodes are paired to generate paired tracks; based on the paired tracks, the target observation objects corresponding to the pairing group are determined.

[0033] After completing measurement data association, a node or even a single sensor can estimate the local state of the target object. To further improve the accuracy of target state estimation, it is necessary to fuse the local state estimates across multiple devices. Track association is the key foundation for distributed multi-source data fusion. Multiple devices share target state estimation pairs (x, P) and implement track association based on the Hungarian algorithm. Here, x represents the state estimate and P represents the state covariance matrix.

[0034] Asynchronous measurements and communication delays between multiple machines can lead to asynchronous local estimates. However, track correlation only requires accurate target association, without requiring real-time state estimation. Therefore, the initialization phase of track correlation only considers data asynchrony caused by differences in measurement start times and sampling frequencies. A curve fitting method is used to synchronize the data between multiple machines for subsequent track correlation.

[0035] The Mahalanobis distance is used as the cost weight between the state estimates to be matched between machines, as shown in formula (1):

[0036]

[0037] in, and represents the state estimation and state covariance matrix of target n in single machine i (or single sensor i), and Represents the state estimation and state covariance matrix of target m in single machine j (or single sensor j).

[0038] Construct a cost matrix D∈R based on possible matching targets n×m , as shown in formula (2):

[0039]

[0040] Under the assumption that the targets of any two nodes can only be uniquely paired, the Hungarian algorithm is used to minimize the total cost function to achieve unique matching of target tracks.

[0041] Under the assumption that target observations between any two nodes can only be uniquely paired, the Hungarian algorithm is applied to minimize the total cost function to achieve unique matching of targets. The Hungarian algorithm ensures that each target observation is accurately associated with the corresponding state estimate by finding the minimum total pairing cost in the cost matrix.

[0042] By using the Hungarian matching algorithm and Mahalanobis distance, combined with data synchronization methods, the initial association of targets can be effectively achieved in multi-sensor systems, which is crucial for improving target recognition and tracking capabilities in complex environments.

[0043] First, the first local state estimate The corresponding sampling time Determine whether the absolute value of the difference between the sampling time and the fusion time is less than the preset threshold; if so, determine that the first local state estimate does not meet the preset condition; if not, determine that the first local state estimate meets the preset condition, and query the second local state estimate that is before the acquisition time and close to the acquisition time from the second state estimate sequence of the paired node for the target observation object. If a second local state estimate that meets the requirements is found Based on the sampling time, the second local state estimation value is estimated using the Kalman prediction method Perform the first time registration to obtain the first registration state estimate. If no second local state estimate that meets the requirements is found, wait for the next second local state estimate sent by the paired node to the first device; express The local state estimate of the target observation object n at time node i.

[0044] Specifically, if the absolute value of the difference between the sampling moment and the fusion moment is less than the preset threshold, it is determined that the first local state estimation value does not meet the preset condition; if the absolute value of the difference between the sampling moment and the fusion moment is not less than the preset threshold, it is determined that the first local state estimation value meets the preset condition.

[0045] Based on the collection time The second local state estimate is calculated using the Kalman prediction method. Register to the acquisition time to obtain the first registration state estimate and the first registration state covariance matrix The specific calculation formula is shown in formula (3):

[0046]

[0047] Among them, F k Represents the state recursive model of discrete targets; the calculation formula is shown in formula (3):

[0048] F k =I+A k δt formula (4);

[0049] Where δt is and The time interval between k The equation representing the change of the target continuous state is shown in formula (5):

[0050]

[0051] in, express The derivative of wk is the process noise, which describes the prediction error between the physical model and the reference model, Q k is the variance matrix of the process noise. Based on the above formula, the following formula (6) can be derived:

[0052] V k =B k δ t Formula (6);

[0053] Among them, B k is the influence matrix of the control input.

[0054] Through the above steps and formulas, the Kalman filter can be effectively used to align and update the first state estimate. Upon receiving a valid first state estimate, the state transition matrix and covariance update rule are used to achieve efficient state prediction and estimation. If no valid data is received, the system is designed to wait for the next communication to ensure the accuracy and real-time nature of the state estimate. This design improves the robustness and accuracy of the multi-target tracking system, enabling it to effectively respond to changes in target state in dynamic environments.

[0055] As a result, not only the real-time and accuracy of target state estimation between multiple nodes are improved, but also the potential errors caused by data asynchrony are reduced, ultimately providing a reliable foundation for subsequent state fusion.

[0056] Secondly, after the acquisition time registration is completed, the first registration state estimation value and the first local state estimation value are fused based on the Kalman filter measurement update method to output the fused local state estimation value. And the fusion state covariance matrix Specifically, the steps are aimed at and The first stage of fusion is achieved to reduce the uncertainty of single-machine Kalman prediction under communication delay. and are not related to each other, so we can refer to the Kalman filter measurement update method to As a virtual measurement, the measurement matrix is ​​the unit matrix I, and the fusion formula is shown in the following formula (7):

[0057]

[0058] in, and They respectively represent the fused local state estimation value and the fused state covariance matrix of the local state estimation value of node i and node j for the target object n after completing the first stage of fusion.

[0059] Afterwards, based on the fusion time, the Kalman prediction method is used to register the fused local state estimates to the latest fusion time. Specifically, this step takes into account the real-time requirements of data fusion and the simplicity of the fusion method. By temporally registering the fused local state estimates, the temporal registration error is further reduced.

[0060] Based on the fusion time t f , the Kalman prediction method is used to estimate the fusion local state of the target observation object n and Perform a second temporal registration to generate a second registration state estimate and the second registration state covariance matrix Will Add to fusion cache queue The specific calculation formula is shown in formula (8):

[0061]

[0062] Among them, V k Calculated based on time deviation.

[0063] As a result, the system can effectively fuse state estimates from multiple nodes, reducing uncertainty introduced by communication delays. This approach not only improves the accuracy of state estimates but also ensures real-time and efficiency in dynamic environments. This is of great significance in applications such as multi-sensor fusion and target tracking.

[0064] Here, since each node stores its local state estimate for the observed object locally after generating it, each node generates corresponding tracks for different observed objects within a preset time, generating multiple tracks and sending them to the first device. This results in a second state estimate sequence for the tracks corresponding to the target observed objects in the paired node.

[0065] In S103, based on a preset rule or model, the second registered state estimate corresponding to each first local state estimate in the first state estimation queue is fused to output the global state estimate corresponding to the target observed object at the fusion moment. The two-stage fusion algorithm based on the covariance intersection method in this embodiment performs well in high-speed dynamic scenes, capable of updating and fusing the state information of multiple nodes in real time, providing reliable support for multi-sensor collaborative operation.

[0066] Exemplarily, based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, a fusion cache queue is obtained; for any second registration state estimation value in the fusion cache queue: a second registration covariance matrix corresponding to the second registration state estimation value is obtained; based on the second registration covariance matrix, an information matrix and a first weight corresponding to the information matrix are determined; based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue, several information matrices are obtained; for any information matrix among the several information matrices: a corresponding first weight is applied to the information matrix, and the information matrix after the first weight is applied is summed to generate an optimal information matrix; based on the optimal information matrix, and the information matrix and the first weight corresponding to each second registration state estimation value in the fusion cache queue, the second registration state estimation value is optimized to output the global state estimation value corresponding to the target observation object at the fusion moment.

[0067] Specifically, the covariance intersection method is used to fused the cache queue All fused local state estimates participating in the first stage fusion Perform fusion processing and output the global state estimation value corresponding to the target observation object at the fusion time; the calculation of the global state estimation value is specifically as follows (9);

[0068]

[0069] in, is the information matrix, represents the optimal information matrix; w ij is the first weight, satisfying the random constraint

[0070]

[0071] The first weight coefficient can be obtained based on the nonlinear optimization problem, as shown in formula (10):

[0072]

[0073] In order to improve the fusion efficiency, the first weight is updated based on the fast covariance cross method. The specific first weight update method is shown in the following formula (11):

[0074]

[0075] in,

[0076] Therefore, the covariance intersection method is used to consistently fuse information in asynchronous distributed systems, which not only improves the accuracy of local state estimates, but also enhances the fusion efficiency of the system in high-speed scenarios. The entire algorithm process covers the update from local state estimation to global state estimation, ensuring the reliability and consistency of information.

[0077] In a preferred implementation of this embodiment, if the first local state estimation value does not meet the preset conditions, the first local state estimation value is time-aligned based on the fusion moment to generate a first local state estimation value after alignment; and the first local state estimation value after alignment is determined as the second alignment state estimation value.

[0078] Specifically, if the first local state estimate does not meet the preset conditions, it is considered that the accuracy of the Kalman prediction method meets the registration requirements; based on the fusion moment, the Kalman prediction method is used to directly perform a second time registration on the first local state estimate to generate the first local state estimate after registration and the first state covariance matrix after registration.

[0079] Due to the fusion cache queue Including the second registration state estimate after the first stage fusion and / or the first local state estimate after registration When both exist at the same time, first fuse all based on the covariance cross method. Then, the covariance cross method is used to obtain the fusion and all Fusion is performed to generate a sub-global state estimate

[0080] Sub-global state estimate and the subglobal state covariance matrix It is calculated by the following formula (12):

[0081]

[0082] in, represents the first local state estimate after registration, Represents the first state covariance matrix after registration.

[0083] This embodiment comprehensively considers the communication delay and inherent sensor problems that lead to data asynchrony, and proposes a two-stage distributed asynchronous fusion framework combined with a fast covariance crossover method. This framework uses a two-stage time registration method to effectively reduce the state prediction error of time registration and provide a good initial value for state asynchronous fusion. It also solves the problem of significantly increased registration error in existing time registration technologies in high-speed scenarios.

[0084] like Figure 2FIG. 1 is a flow chart of a data fusion method based on two-stage time registration provided by another embodiment of the present invention.

[0085] A data fusion method based on two-stage temporal registration, wherein the node includes at least one sensor; each of the sensors is used to observe at least one observation object; and comprises:

[0086] S201, associating the current measurement result generated by the node with the target observation object to generate a target measurement result;

[0087] S202, determining a corresponding filtering method based on the sensor type corresponding to the current measurement result;

[0088] S203, filtering the target measurement result based on a filtering method, and outputting a filtered measurement result;

[0089] S204, optimizing the current state of the target observation object according to the current initialized state value of the target observation object and the filtered measurement result, and outputting the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

[0090] In S201 , there is no limitation on the method for establishing the association relationship between the current measurement result and the target observation object, which can be implemented based on a model or preset rules. For example, the association between the current measurement result and the target observation object can be implemented based on a gating concept.

[0091] Exemplarily, the measurement results corresponding to each of all the observation objects that can be observed by the node are received to generate at least one current measurement result; for any current measurement result among the at least one current measurement result: the observation object corresponding to the current measurement result is taken as the target observation object, and the predicted measurement result corresponding to the predicted state of the target observation object at the current moment is obtained; the Euclidean distance between the current measurement result and the predicted measurement result is determined; if the Euclidean distance is not greater than the gated threshold, it is determined that the current measurement result is associated with the target observation object; the number of current measurement results associated with the target observation object is counted to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, the current measurement result is used as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, the distance between the observation object of each current measurement result and the target observation object is determined; and the current measurement result with the smallest Euclidean distance is selected from the at least two current measurement results as the target measurement result of the target observation object.

[0092] Specifically, considering the need to track multiple target observation objects in actual scenarios, this step aims to complete the association between node measurement results and target observation states to reduce the probability of erroneous tracking.

[0093] The probability threshold of the measurement result is set based on the gate control idea. If the current measurement result and the predicted measurement results h(x i ) is less than or equal to the gate threshold, the match is considered successful, as shown in the following formula (13);

[0094]

[0095] Where h(·) is the observation equation, δ d Represents the gating threshold. The value of the gating threshold needs to comprehensively consider the error between the current predicted state and the current measurement result. When δ d When δ is too large, mismatching may occur. d If the time is too short, the correct current measurement result will be rejected.

[0096]

[0097] Among them, σ h and σ g denote the measurement error and state prediction error respectively, represents the normalized state prediction error, This represents the transformation from the state dimension to the measurement dimension. When multiple current measurement results are associated with the same target observation, a one-to-one matching is achieved using the universal bipartite matching method, the Hungarian algorithm. For example, a greedy strategy selects the measurement with the smallest L2 distance.

[0098] Therefore, the gating-based association of measurement results with local trajectory data can effectively reduce the probability of false tracking and improve the accuracy and robustness of multi-target tracking. This process is flexible and adaptable to dynamically changing environments, providing a good solution for multi-target tracking in practical applications.

[0099] In S202 to S203, an appropriate filtering method is selected based on the sensor's measurement type. For nonlinear measurements, the EKF, UKF, or CKF are used, while for linear measurements, the KF is used. The local state estimate corresponding to the current moment is updated by sequentially following the prior state prediction, uncertainty update, Kalman gain solution, and posterior state update process.

[0100] Therefore, by combining the selection of appropriate filtering methods and detailed state update procedures, the accuracy and robustness of multi-target tracking can be effectively improved.

[0101] In a preferred implementation of this embodiment, the method also includes: receiving target observation information for the target observation object sent by the sensor; if there is prior information of the target observation object, performing attribute identification on the target observation object based on the target observation information and the prior information to generate a target matching result; generating a target trajectory based on the target matching result; if there is no prior information of the target observation object and the data type of the target observation information meets the observability requirement, performing Hough detection on the target observation information to generate a target trajectory; if there is no prior information of the target observation object and the data type of the target observation information does not meet the observability requirement, after receiving the local state estimation value for the target object sent by other nodes, performing trajectory similarity matching on the target observation information and the local state estimation value to generate a target trajectory; obtaining the predicted state value for the target observation object at the current moment output at the previous moment from the target trajectory; and using the predicted state value as the current initialization state value.

[0102] Specifically, the local state estimation value is used to indicate the 3D or 2D position of the target observation object. or At the same time, the state covariance matrix needs to be initialized to obtain the initialized state covariance matrix P0. Since the state covariance reflects the uncertainty of the state of the target observation object, a larger initial state covariance matrix can accelerate the convergence of the filter.

[0103] In a preferred implementation of an embodiment of the present invention, based on the filtered measurement results corresponding to the target observation object at the previous moment adjacent to the current moment and the predicted state value at the previous moment, the current state of the target observation object at the previous moment is optimized, and the local state estimation value corresponding to the target object at the previous moment and the predicted state value at the current moment are generated.

[0104] The data fusion method based on two-stage temporal registration provided by this embodiment will be described in detail below with reference to a specific application scenario.

[0105] A data fusion method based on two-stage temporal registration includes: a plurality of nodes for performing state estimation on an observed object; the nodes include at least one sensor; each of the sensors is used to observe at least one observed object and perform state estimation on the observed object; each of the nodes is communicatively connected to a first device; and the method is applied to the first device;

[0106] S1, based on the gating idea, realizes the association between the current measurement result and the target observation object.

[0107] Receive measurement results corresponding to each of all observation objects that can be observed by the node, and generate at least one current measurement result; for any current measurement result among the at least one current measurement result: take the observation object corresponding to the current measurement result as the target observation object, and obtain the predicted measurement result corresponding to the predicted state of the target observation object at the current moment; determine the Euclidean distance between the current measurement result and the predicted measurement result; if the Euclidean distance is not greater than a gated threshold, determine that the current measurement result is associated with the target observation object; count the number of current measurement results associated with the target observation object to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, take the current measurement result as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, determine the distance between the observation object of each current measurement result and the target observation object; select the current measurement result with the smallest Euclidean distance from the at least two current measurement results as the target measurement result of the target observation object.

[0108] S2, determining the local state estimation value corresponding to the current measurement result based on the Kalman filter method.

[0109] Based on the sensor type corresponding to the current measurement result, a corresponding filtering method is determined; based on the filtering method, the target measurement result is filtered and the filtered measurement result is output; according to the predicted state value of the target observation object at the current moment and the filtered measurement result, the current state of the target observation object is optimized and the local state estimation value corresponding to the target observation object at the current moment is output.

[0110] S3, based on the Hungarian matching algorithm, completes the track association of the target observation object between the communicable nodes within the sliding window and assigns the target ID.

[0111] Acquire a plurality of tracks generated by each of the plurality of nodes within a preset time; wherein the tracks are used to indicate the motion trajectory generated by the node for the observable observation object, and each observation object has a corresponding track; each track is formed by a plurality of local state estimation values ​​arranged in chronological order; select any two nodes from the plurality of nodes as a pairing group: pair a plurality of first tracks corresponding to the target node in the pairing group with a plurality of second tracks corresponding to the pairing node to generate paired tracks; and determine the target observation object corresponding to the pairing group based on the paired tracks.

[0112] S4, when the first device receives the first local state estimation value sent from other nodes, it parses the data packet and searches for the local state estimation value closest in time dimension in the state estimation sequence corresponding to the target ID according to the sampling time of the first local state estimation value; then applies the Kalman filtering method to align the local state estimation value for time registration.

[0113] Based on the first local state estimation value of each node for the target observation object within the same time window, a first state estimation queue is generated; wherein, within the same time window, each node sends the local state estimation value of the target observation object to the first device once. For any first local state estimation value in the first state estimation queue: obtain the sampling time of the first local state estimation value; if the first local state estimation value meets the preset conditions, obtain the second local state estimation value closest to the sampling time from the second state estimation sequence of the paired node for the target observation object; based on the sampling time, perform time alignment on the second local state estimation value to obtain a first aligned state estimation value; then execute step S6;

[0114] If the corresponding local state estimation value is not obtained, wait for the node to send the local state estimation value next time; then return to step S4.

[0115] S5. If the first local state estimation value does not meet the preset conditions, then based on the fusion moment, time-align the first local state estimation value to generate a registered first local state estimation value; and determine the registered first local state estimation value as a second registered state estimation value.

[0116] S6, after completing the time alignment, performing the first stage fusion according to the Kalman measurement update method; and aligning the fused local state estimation value generated after the fusion to the latest fusion time.

[0117] The first registered state estimate and the first local state estimate are fused to output a fused local state estimate. Based on the fusion time, the fused local state estimate is temporally registered to generate a second registered state estimate; wherein the paired node indicates a node that can be trajectory-matched with the target node corresponding to the first local state estimate; and then S7 is executed.

[0118] S7. Obtain a fusion cache queue based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue; wherein the fusion cache queue includes: the second registration state estimation value after the first stage fusion processing and the first local state estimation value after registration.

[0119] S8, based on the covariance intersection method, all the second registration state estimation values ​​of the fusion cache queue participating in the first stage fusion processing are fused again to achieve information consistency fusion.

[0120] For any second registration state estimate in the fusion cache queue that has participated in the first-stage fusion processing: obtain the second registration covariance matrix corresponding to the second registration state estimate; determine an information matrix and a first weight corresponding to the information matrix based on the second registration covariance matrix. Based on the information matrix corresponding to each second registration state estimate in the fusion cache queue, obtain several information matrices; for any information matrix among the several information matrices: apply the corresponding first weight to the information matrix, and sum the information matrices after applying the first weight to generate an optimal information matrix; based on the optimal information matrix, the information matrix corresponding to each second registration state estimate in the fusion cache queue, and the first weight, optimize the second registration state estimate, and output the global state estimate corresponding to the target observation object at the fusion moment.

[0121] For the first local state estimate after registration in the fusion cache queue: use the covariance intersection method to calculate the global state estimate and all Fusion is performed to generate a sub-global state estimate

[0122] This embodiment's Kalman filter-based temporal registration method combines real-time and precision requirements, simultaneously addressing data asynchrony issues caused by differing sensor sampling frequencies and inconsistent sensor sampling start times. In particular, under the assumption of a highly maneuverable target motion model, typical temporal registration algorithms exhibit increased errors. Furthermore, considering communication latency, low communication bandwidth reduces both the registration accuracy of target observation data and the accuracy of state estimation. To this end, this embodiment utilizes a Kalman filter to achieve alignment and synchronization of sampled data, followed by a two-stage weighted asynchronous fusion technique to mitigate the impact of communication latency on fusion accuracy.

[0123] like Figure 3 FIG. 1 is a structural diagram of a data fusion device based on two-stage time registration provided by an embodiment of the present invention.

[0124] A data fusion device based on two-stage time alignment, comprising a plurality of nodes for state estimation of an observed object; each of the nodes is in communication connection with a first device; the method is applied to the first device; the device 300 comprises: a first generation module 301, for generating a first state estimation queue based on a first local state estimation value of each of the nodes for a target observed object within the same time window; wherein each of the nodes sends a local state estimation value of the target observed object to the first device once within the same time window; a second generation module 302, for obtaining a sampling moment of the first local state estimation value for any first local state estimation value in the first state estimation queue; if the first local state estimation value meets a preset condition, the second state estimation value of the target observed object from the paired node is obtained. Obtain a second local state estimation value closest to the acquisition moment in the estimation sequence; perform time registration on the second local state estimation value based on the sampling moment to obtain a first registered state estimation value; fuse the first registered state estimation value and the first local state estimation value to output a fused local state estimation value; perform time registration on the fused local state estimation value based on the fusion moment to generate a second registered state estimation value; wherein, the pairing node is used to indicate a node that can perform trajectory matching with the target node corresponding to the first local state estimation value; the fusion processing module 303 is used to perform fusion processing based on the second registered state estimation value corresponding to each first local state estimation value in the first state estimation queue, and output the global state estimation value corresponding to the target observation object at the fusion moment.

[0125] In a preferred implementation of this embodiment, the third generating module is configured to perform time registration on the first local state estimation value based on a fusion moment to generate a second registered state estimation value if the first local state estimation value does not meet a preset condition.

[0126] In a preferred implementation of this embodiment, the device also includes: an acquisition module for acquiring a number of tracks generated by each of the several nodes within a preset time; wherein the track is used to indicate the motion trajectory generated by the node for the observable observation object, and each of the observation objects has a corresponding track; each track is formed by a number of local state estimation values ​​arranged in chronological order; a fourth generation module for selecting any two nodes from the several nodes as a pairing group: pairing the several first tracks corresponding to the target node in the pairing group and the several second tracks corresponding to the pairing node to generate a pairing result; a selection module for selecting the target track corresponding to the pairing node from the several second tracks based on the pairing result.

[0127] In a preferred implementation manner of this embodiment, the fusion processing module includes: a first obtaining unit, which is used to obtain a fusion cache queue based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue; a determination unit, which is used to obtain, for any second registration state estimation value in the fusion cache queue: a second registration covariance matrix corresponding to the second registration state estimation value; determine an information matrix and a first weight corresponding to the information matrix based on the second registration covariance matrix; a second obtaining unit, which is used to obtain several information matrices based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue; a generation unit, which is used to apply the corresponding first weight to any information matrix among the several information matrices, and sum up the information matrices after applying the first weight to generate an optimal information matrix; an optimization processing unit, which is used to optimize the second registration state estimation value based on the optimal information matrix, the information matrix corresponding to each second registration state estimation value in the fusion cache queue and the first weight, and output the global state estimation value corresponding to the target observation object at the fusion moment.

[0128] like Figure 4 FIG. 1 is a structural diagram of a data fusion device based on two-stage time registration provided by another embodiment of the present invention.

[0129] A data fusion state based on two-stage time alignment is applied to the node: the node includes at least one sensor; each sensor is used to observe at least one observation object; the device 400 includes: a first generation module 401, used to associate the current measurement result generated by the node with the target observation object to generate a target measurement result; a first determination module 402, used to determine the corresponding filtering method based on the sensor type corresponding to the current measurement result; a filtering module 403, used to filter the target measurement result based on the filtering method, and output the filtered measurement result; an optimization module 404, used to optimize the current state of the target observation object according to the current initialization state value of the target observation object and the filtered measurement result, and output the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

[0130] In a preferred implementation manner of this embodiment, the first generation module includes: a receiving unit, configured to receive a measurement result corresponding to each of all observation objects that can be observed by the node, and generate at least one current measurement result; a determining unit, configured to, for any current measurement result among the at least one current measurement result: use the observation object corresponding to the current measurement result as a target observation object, and obtain a predicted measurement result corresponding to the predicted state of the target observation object at the current moment; determine a Euclidean distance between the current measurement result and the predicted measurement result; if the Euclidean distance is not greater than a gated threshold, determine that the current measurement result is associated with the target observation object; and a selecting unit, configured to count the number of current measurement results associated with the target observation object to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, use the current measurement result as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, determine the distance between the observation object of each current measurement result and the target observation object; and select the current measurement result with the smallest Euclidean distance from the at least two current measurement results as the target measurement result of the target observation object.

[0131] In a preferred implementation manner of this embodiment, the device also includes: a receiving module for receiving target observation information for a target observation object sent by a sensor; a second generating module for, if there is prior information of the target observation object, performing attribute identification on the target observation object based on the target observation information and the prior information to generate a target matching result; and generating a target trajectory based on the target matching result; a third generating module for, if there is no prior information of the target observation object and the data type of the target observation information meets the observability requirement, performing Hough detection on the target observation information to generate a target trajectory; a fourth generating module for, if there is no prior information of the target observation object and the data type of the target observation information does not meet the observability requirement, performing trajectory similarity matching on the target observation information and the local state estimation value after receiving the local state estimation value for the target object sent by other nodes to generate a target trajectory; a second determining module for obtaining, from the target trajectory, the predicted state value for the target observation object at the current moment output at the previous moment; and using the predicted state value as the current initialization state value.

[0132] The above-described device can implement a data fusion method based on two-stage temporal registration provided in one embodiment of the present invention, and possesses the corresponding functional modules and beneficial effects of implementing a data fusion method based on two-stage temporal registration. For technical details not fully described in this embodiment, please refer to the data fusion method based on two-stage temporal registration provided in one embodiment of the present invention.

[0133] The present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a data fusion method based on two-stage time alignment described in the present invention.

[0134] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0135] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0136] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to the following embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0137] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0138] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0139] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0140] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0141] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0142] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0143] In the description of this specification, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless otherwise inconsistent.

[0144] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0145] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A data fusion method based on two-stage temporal registration, characterized in that: The method comprises a plurality of nodes for performing state estimation on an observed object; each of the nodes is communicatively connected to a first device; the method is applied to the first device; and comprises: Generate a first state estimation queue based on a first local state estimation value of each of the nodes for the target observation object within the same time window; wherein each of the nodes sends a local state estimation value of the target observation object to the first device once within the same time window; For any first local state estimation value in the first state estimation queue: obtaining a sampling time of the first local state estimation value; if the first local state estimation value meets a preset condition, obtaining a second local state estimation value closest to the sampling time from a second state estimation sequence of a paired node for a target observation object; based on the sampling time, performing time registration on the second local state estimation value to obtain a first registered state estimation value; fusing the first registered state estimation value and the first local state estimation value to output a fused local state estimation value; based on the fusion time, performing time registration on the fused local state estimation value to generate a second registered state estimation value; wherein the paired node is used to indicate a node that can perform trajectory matching on the target object with the target node corresponding to the first local state estimation value; A fusion process is performed based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and a global state estimation value corresponding to the target observation object at the fusion moment is output.

2. The method according to claim 1, characterized in that Also includes: If the first local state estimation value does not meet the preset condition, performing time registration on the first local state estimation value based on the fusion time to generate a registered first local state estimation value; The first local state estimation value after registration is determined as a second registration state estimation value.

3. The method according to claim 1, characterized in that Also includes: Acquire a plurality of tracks generated by each of the plurality of nodes for performing state estimation on the observed objects within a preset time; wherein the track indicates the movement trajectory generated by the node for the observed objects, each observed object having a corresponding track; and each track is formed by arranging a plurality of local state estimation values ​​in chronological order; Selecting any two nodes from the plurality of nodes for state estimation of the observed object as a pairing group; performing track pairing on a plurality of first tracks corresponding to the target nodes in the pairing group and a plurality of second tracks corresponding to the pairing nodes to generate paired tracks; Based on the paired tracks, a target observation object corresponding to the paired group is determined.

4. The method according to claim 1, wherein The method comprises: performing fusion processing based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and outputting the global state estimation value corresponding to the target observation object at the fusion moment; Obtaining a fusion cache queue based on a second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue; For any second registration state estimation value in the fusion cache queue: obtaining a second registration covariance matrix corresponding to the second registration state estimation value; determining an information matrix and a first weight corresponding to the information matrix based on the second registration covariance matrix; Obtaining a plurality of information matrices based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue; For any information matrix among the plurality of information matrices: applying a corresponding first weight to the information matrix, and summing the information matrices after applying the first weight to generate an optimal information matrix; Based on the optimal information matrix, and the information matrix and the first weight corresponding to each second registration state estimation value in the fusion cache queue, the second registration state estimation value is optimized to output the global state estimation value corresponding to the target observation object at the fusion moment.

5. A data fusion method based on two-stage temporal registration, characterized in that: Applicable to the plurality of nodes for performing state estimation on an observed object in the method according to any one of claims 1 to 4: the nodes comprising at least one sensor; each of the sensors being configured to observe at least one observed object; comprising: Based on the gating idea, the current measurement result generated by the node is associated with the target observation object to generate the target measurement result; Determining a corresponding filtering method based on the sensor type corresponding to the current measurement result; Performing filtering on the target measurement result based on the filtering method, and outputting the filtered measurement result; According to the current initialized state value of the target observation object and the filtered measurement result, the current state of the target observation object is optimized, and the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment are output.

6. The method according to claim 5, characterized in that The step of associating the current measurement result generated by the node with the target object to generate a target measurement result includes: receiving a measurement result corresponding to each of all observed objects that can be observed by the node, and generating at least one current measurement result; For any current measurement result of the at least one current measurement result: taking the observation object corresponding to the current measurement result as the target observation object, obtaining a predicted measurement result corresponding to the predicted state of the target observation object at the current moment; determining a Euclidean distance between the current measurement result and the predicted measurement result; and determining that the current measurement result is associated with the target observation object if the Euclidean distance is not greater than a gating threshold; Counting the number of current measurement results associated with the target observation object to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, using the current measurement result as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, determining the distance between the observation object of each current measurement result and the target observation object; and selecting the current measurement result with the smallest Euclidean distance from the at least two current measurement results as the target measurement result of the target observation object.

7. The method according to claim 5, characterized in that Also includes: Receiving target observation information for a target observation object sent by a sensor; If there is prior information of the target observation object, then based on the target observation information and the prior information, attribute recognition is performed on the target observation object to generate a target matching result; generating a target trajectory based on the target matching result; If there is no prior information of the target observation object and the data type of the target observation information meets the observability requirement, then performing Hough detection on the target observation information to generate a target trajectory; If there is no prior information of the target observation object and the data type of the target observation information does not meet the observability requirement, then after receiving the local state estimation value of the target object sent by other nodes, the target observation information and the local state estimation value are matched for trajectory similarity to generate the target trajectory; The predicted state value of the target observation object at the current moment output at the previous moment is obtained from the target trajectory; and the predicted state value is used as the current initialization state value.

8. A data fusion device based on two-stage temporal registration, characterized in that: The device comprises a plurality of nodes for performing state estimation on an observed object; each of the nodes is communicatively connected to a first device; the device is applied to the first device; and comprises: A first generating module is configured to generate a first state estimation queue based on a first local state estimation value of each of the nodes for the target observation object within the same time window; wherein each of the nodes sends the local state estimation value of the target observation object to the first device once within the same time window; The second generation module is configured to: for any first local state estimation value in the first state estimation queue: obtain a sampling time of the first local state estimation value; if the first local state estimation value satisfies a preset condition, obtain a second local state estimation value closest to the sampling time from a second state estimation sequence of a paired node for a target observation object; perform time registration on the second local state estimation value based on the sampling time to obtain a first registered state estimation value; fuse the first registered state estimation value and the first local state estimation value to output a fused local state estimation value; perform time registration on the fused local state estimation value based on the fusion time to generate a second registered state estimation value; wherein the paired node is configured to indicate a node that can perform trajectory matching on the target object with the target node corresponding to the first local state estimation value; A fusion processing module is used to perform fusion processing based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and output the global state estimation value corresponding to the target observation object at the fusion moment.

9. A data fusion device based on two-stage temporal registration, characterized in that: Applicable to the plurality of nodes for performing state estimation on an observed object in the method according to any one of claims 1 to 4: the nodes comprising at least one sensor; each of the sensors being configured to observe at least one observed object; comprising: A first generating module is configured to associate the current measurement result generated by the node with the target observation object based on a gating concept to generate a target measurement result; A first determining module, configured to determine a corresponding filtering method based on a sensor type corresponding to the current measurement result; A filtering module, configured to filter the target measurement result based on the filtering method and output a filtered measurement result; The optimization module is used to optimize the current state of the target observation object according to the current initialized state value of the target observation object and the filtered measurement result, and output the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

10. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 4 or claims 5 to 7 is implemented.

Images