Data fusion method and device based on two-stage time registration

Abstract

The invention discloses a data fusion method and device based on two-stage time registration, and the method comprises a plurality of nodes, and each node is in communication connection with a first device. The method is applied to first equipment. Generating a first state estimation queue based on the first local state estimation value of each node for the target observation object in the same time window; for any first local state estimation value in the first state estimation queue, performing time registration and fusion processing on the first local state estimation value to generate a second registration state estimation value; and performing fusion processing based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and outputting a global state estimation value. Therefore, according to the embodiment of the invention, the two-stage weighted asynchronous fusion technology of Kalman filtering is applied, so that the influence of communication delay on fusion precision can be reduced, and the precision and real-time performance of the state estimation value of the target observation object in a high-speed scene are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a data fusion method and device based on two-stage time registration. Background Art

[0002] Multi-sensor fusion refers to the integration of data from multiple sensors to obtain more comprehensive and accurate information. Its advantage is that it can overcome the limitations of a single sensor and improve the robustness and reliability of the system. In target tracking, multi-sensor fusion can play a more significant advantage: high accuracy, robustness, wide-area perception range and information integrity. Multi-sensor fusion can be broadly divided into multi-machine data fusion and multi-modal data fusion. Multi-modal data fusion emphasizes the execution of sensor data fusion on a single machine, and generally can directly use the raw data of multiple sensors. When a single sensor fails or is interfered with, other sensors can provide supplementary information to ensure that the system can continue to track the target. In addition, the measurement range and information type of multi-modal sensors are different, which can effectively expand the perception range and improve information integrity. Radar can measure the distance to the target and the surrounding environment, and the camera can perceive the appearance attributes and texture changes of the target and the environment. Spatiotemporal synchronization is the key basis for multi-sensor data fusion. When multiple sensors are mounted on a single platform, data alignment can be achieved through hard synchronization methods such as pulse triggering.

[0003] Limited by the inherent properties of a single-machine platform, the scope of target tracking and searching is limited, and the system is not robust and efficient enough. Multi-machine platforms can effectively make up for the disadvantages of single-machine platforms due to their flexible distribution. Unlike the hard time synchronization method on a single-machine platform, multi-machine platforms only achieve global time synchronization through communication synchronization triggering, relying on the ideal assumption of reliable full connectivity and low communication latency. In actual scenarios, the centralized triggering method limits the movement of multiple machines, which may lead to target tracking failure.

[0004] The reasons for the asynchrony of multi-sensor data are mainly the following three points: 1) time base is not synchronized; 2) different sampling frequencies of sensors and inconsistent sampling start times of sensors; 3) communication delay. On the basis of time base synchronization, due to the communication delay problem, there is a time difference in the synchronization trigger, which makes the sampling start time of the sensor different. In view of the different sampling times of sensors, the existing curve fitting method, spline interpolation method and interpolation extrapolation method can be used to realize the registration of sampled data. The least squares virtual method and interpolation extrapolation method realize data registration under different sampling frequencies in a synchronous and asynchronous manner respectively. However, the interpolation extrapolation method and the least squares virtual method are only applicable to ideal motion models such as uniform speed and uniform acceleration. The curve fitting method and the spline interpolation method only have good approximate results for data interpolation, but the real-time performance of these methods is insufficient. Summary of the invention

[0005] The present invention provides a data fusion method and device based on two-stage time registration; the method can solve the distributed asynchronous data fusion problem caused by communication delay and inherent problems of sensors, thereby improving the accuracy and real-time performance of the state estimation value of the target observation object in high-speed scenarios.

[0006] According to a first aspect of an embodiment of the present invention, a data fusion method based on two-stage time alignment is provided, the method comprising a plurality of nodes for state estimation of an observed object; each of the nodes is in communication connection with a first device; the method is applied to the first device; comprising: generating a first state estimation queue based on a first local state estimation value of each of the nodes for a target observed object within the same time window; wherein each of the nodes sends a local state estimation value of the target observed object to the first device once within the same time window; for any first local state estimation value in the first state estimation queue: obtaining a sampling time of the first local state estimation value; if the first local state estimation value meets a preset condition, obtaining a second state estimation value of the target observed object from a paired node; The method comprises the following steps: obtaining a second local state estimation value closest to the sampling moment in the sequence; performing time registration on the second local state estimation value based on the sampling moment to obtain a first registered state estimation value; fusing the first registered state estimation value and the first local state estimation value to output a fused local state estimation value; performing time registration on the fused local state estimation value based on the fusion moment to generate a second registered state estimation value; wherein the paired node is used to indicate a node that can match the trajectory of a target object with the target node corresponding to the first local state estimation value; performing fusion processing on the second registered state estimation value corresponding to each first local state estimation value in the first state estimation queue to output a global state estimation value corresponding to the target observation object at the fusion moment.

[0007] Optionally, the method also includes: if the first local state estimation value does not meet a preset condition, based on the fusion moment, performing time alignment on the first local state estimation value to generate a first local state estimation value after alignment; and determining the first local state estimation value after alignment as a second alignment state estimation value.

[0008] Optionally, the method also includes: obtaining a number of tracks generated by each of the several nodes within a preset time; wherein the track is used to indicate the action trajectory generated by the node for the observable observation object, and each of the observation objects has a corresponding track; each track is formed by a number of local state estimation values ​​arranged in chronological order; selecting any two nodes from the several nodes as a pairing group: pairing a number of first tracks corresponding to the target nodes in the pairing group and a number of second tracks corresponding to the pairing nodes to generate paired tracks; based on the paired tracks, determining the target observation object corresponding to the pairing group.

[0009] Optionally, the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue is fused, and the global state estimation value corresponding to the target observation object at the fusion moment is output; including: obtaining a fusion cache queue based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue; for any second registration state estimation value in the fusion cache queue: obtaining the second registration covariance matrix corresponding to the second registration state estimation value; determining the information matrix and the first weight corresponding to the information matrix based on the second registration covariance matrix; obtaining several information matrices based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue; for any information matrix among the several information matrices: applying the corresponding first weight to the information matrix, and summing up the information matrices after applying the first weight to generate an optimal information matrix; optimizing the second registration state estimation value based on the optimal information matrix, the information matrix and the first weight corresponding to each second registration state estimation value in the fusion cache queue, and outputting the global state estimation value corresponding to the target observation object at the fusion moment.

[0010] According to the second aspect of an embodiment of the present invention, a data fusion method based on two-stage time alignment is also provided, which is applied to the node: the node includes at least one sensor; each of the sensors is used to observe at least one observation object; including: associating the current measurement result generated by the node with the target observation object to generate a target measurement result; determining the corresponding filtering method based on the sensor type corresponding to the current measurement result; filtering the target measurement result based on the filtering method, and outputting the filtered measurement result; optimizing the current state of the target observation object according to the current initialization state value of the target observation object and the filtered measurement result, and outputting the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

[0011] Optionally, associating the current measurement result generated by the node with the target object to generate a target measurement result includes: receiving the measurement result corresponding to each of all the observation objects that can be observed by the node to generate at least one current measurement result; for any current measurement result among the at least one current measurement result: taking the observation object corresponding to the current measurement result as the target observation object, and obtaining the predicted measurement result corresponding to the predicted state of the target observation object at the current moment; determining the Euclidean distance between the current measurement result and the predicted measurement result; if the Euclidean distance is not greater than a gated threshold, determining that the current measurement result is associated with the target observation object; counting the number of current measurement results associated with the target observation object to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, taking the current measurement result as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, determining the distance between the observation object of each current measurement result and the target observation object; selecting the current measurement result with the smallest Euclidean distance from the at least two current measurement results as the target measurement result of the target observation object.

[0012] Optionally, the method also includes: receiving target observation information for the target observation object sent by the sensor; if there is prior information of the target observation object, performing attribute identification on the target observation object based on the target observation information and the prior information to generate a target matching result; generating a target trajectory based on the target matching result; if there is no prior information of the target observation object and the data type of the target observation information meets the observability requirements, performing Hough detection on the target observation information to generate a target trajectory; if there is no prior information of the target observation object and the data type of the target observation information does not meet the observability requirements, after receiving the local state estimation value for the target object sent by other nodes, performing trajectory similarity matching on the target observation information and the local state estimation value to generate a target trajectory; obtaining the predicted state value for the target observation object at the current moment output at the previous moment from the target trajectory; and using the predicted state value as the current initialization state value.

[0013] According to the third aspect of an embodiment of the present invention, there is also provided a data fusion device based on two-stage time alignment, the device comprising a plurality of nodes for state estimation of an observed object; each of the nodes is communicatively connected to a first device; the method is applied to the first device; comprising: a first generating module, for generating a first state estimation queue based on a first local state estimation value of each of the nodes for a target observed object within a same time window; wherein each of the nodes sends a local state estimation value of the target observed object to the first device once within the same time window; a second generating module, for obtaining a sampling time of the first local state estimation value for any first local state estimation value in the first state estimation queue; if the first local state estimation value meets a preset condition, the first local state estimation value for the target observed object from the paired node is sent to the first device; A second local state estimation value closest to the sampling moment is obtained from two state estimation sequences; based on the sampling moment, the second local state estimation value is time-aligned to obtain a first aligned state estimation value; the first aligned state estimation value and the first local state estimation value are fused to output a fused local state estimation value; based on the fusion moment, the fused local state estimation value is time-aligned to generate a second aligned state estimation value; wherein the pairing node is used to indicate a node that can match the trajectory of the target object with the target node corresponding to the first local state estimation value; a fusion processing module is used to perform fusion processing based on the second aligned state estimation value corresponding to each first local state estimation value in the first state estimation queue, and output a global state estimation value corresponding to the target observation object at the fusion moment.

[0014] According to the fourth aspect of an embodiment of the present invention, a data fusion state based on two-stage time alignment is also provided, which is applied to the node: the node includes at least one sensor; each of the sensors is used to observe at least one observation object; including: a generation module, which is used to associate the current measurement result generated by the node with the target observation object to generate a target measurement result; a determination module, which is used to determine the corresponding filtering method based on the sensor type corresponding to the current measurement result; a filtering module, which is used to filter the target measurement result based on the filtering method, and output the filtered measurement result; an optimization module, which is used to optimize the current state of the target observation object according to the current initialization state value of the target observation object and the filtered measurement result, and output the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

[0015] According to a fifth aspect of an embodiment of the present invention, there is further provided a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0016] According to the sixth aspect of an embodiment of the present invention, an electronic device is also provided, which includes: a processor; a memory for storing instructions executable by the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in the first aspect or the second aspect.

[0017] The embodiment of the present invention provides a data fusion method and device based on two-stage time alignment, the method comprising a plurality of nodes for state estimation of an observed object; each of the nodes is in communication connection with a first device; the method is applied to the first device; comprising: firstly, based on the first local state estimation value of each of the nodes for the target observed object in the same time window, a first state estimation queue is generated; wherein each of the nodes sends the local state estimation value of the target observed object to the first device once in the same time window; secondly, for any first local state estimation value in the first state estimation queue: obtaining the sampling time of the first local state estimation value; if the first local state estimation value meets the preset condition, the second state estimation value of the target observed object from the paired node is obtained; The second local state estimation value closest to the acquisition time is obtained from the estimation sequence; based on the sampling time, the second local state estimation value is time-aligned to obtain the first registered state estimation value; the first registered state estimation value and the first local state estimation value are fused and the fused local state estimation value is output; based on the fusion time, the fused local state estimation value is time-aligned to generate the second registered state estimation value; wherein the paired node is used to indicate the node that can match the trajectory with the target node corresponding to the first local state estimation value; finally, the second registered state estimation value corresponding to each first local state estimation value in the first state estimation queue is fused and the global state estimation value corresponding to the target observation object at the fusion time is output. The Kalman filtering method of this embodiment realizes the alignment and synchronization of the sampling data, and then uses the two-stage weighted asynchronous fusion technology to reduce the influence of the communication delay on the fusion accuracy; thereby improving the accuracy and real-time performance of the state estimation value of the target observation object in the high-speed scene; effectively solving the problem of the obvious increase in the registration error of the existing time registration technology in the high-speed scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0019] Figure 1 A schematic flow chart of a data fusion method based on two-stage time registration provided by an embodiment of the present invention;

[0020] Figure 2 A schematic flow chart of a data fusion method based on two-stage time registration provided by another embodiment of the present invention;

[0021] Figure 3 A schematic diagram of the structure of a data fusion device based on two-stage time registration provided by one embodiment of the present invention;

[0022] Figure 4 A schematic structural diagram of a data fusion device based on two-stage time registration provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0024] like Figure 1 FIG. 1 is a flow chart of a data fusion method based on two-stage time registration provided by an embodiment of the present invention;

[0025] A data fusion method based on two-stage time registration includes a plurality of nodes for state estimation of an observed object; each of the nodes is communicatively connected to a first device; the method is applied to the first device; and at least includes the following steps:

[0026] S101, generating a first state estimation queue based on a first local state estimation value of a target observation object by each node in a same time window; wherein each node sends a local state estimation value of the target observation object to a first device once in the same time window;

[0027] S102, for any first local state estimation value in the first state estimation queue: obtain the sampling time of the first local state estimation value; if the first local state estimation value meets the preset condition, obtain the second local state estimation value closest to the sampling time from the second state estimation sequence of the paired node for the target observation object; based on the sampling time, perform time alignment on the second local state estimation value to obtain a first aligned state estimation value; fuse the first aligned state estimation value and the first local state estimation value to output a fused local state estimation value; based on the fusion time, perform time alignment on the fused local state estimation value to generate a second aligned state estimation value; wherein the paired node is used to indicate a node that can perform trajectory matching with the target node corresponding to the first local state estimation value for the target object;

[0028] S103, performing fusion processing based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and outputting the global state estimation value corresponding to the target observation object at the fusion moment.

[0029] It should be noted that in this embodiment, there is no limitation on the method of time registration and data fusion, which can be implemented based on a training model or a mathematical algorithm. For example, the Kalman filter method is used to achieve the alignment and synchronization of the sampled data, and then the two-stage weighted asynchronous fusion technology is used for data fusion.

[0030] In S101, the first device communicates with each of the plurality of nodes, each node sends a local state estimation value corresponding to each observed object that can be observed to the first device, the first device receives the local state estimation value sent by each node, and stores the received local state estimation value in a cache queue to generate a cache queue. Since there is a communication delay between each node and the first device, even if each node sends the first local state estimation value corresponding to the target observation object to the first device at the same time, due to the communication delay between the node and the first device, the first device receives the first local state estimation value sent by each node at different times. The first local state estimation value of each node for the target observation object in the same time window is obtained to obtain the first state estimation queue

[0031] It should be noted that the local state estimation value of each node is determined by the node based on the observation information collected by the sensor on the node.

[0032] In S102, exemplarily, a plurality of tracks generated by each of the plurality of nodes within a preset time are obtained; wherein the track is used to indicate the action trajectory generated by the node for the observable observation object, and each of the observation objects has a corresponding track; each track is formed by a plurality of local state estimation values ​​arranged in chronological order; any two nodes are selected from the plurality of nodes as a pairing group: a plurality of first tracks corresponding to the target nodes in the pairing group and a plurality of second tracks corresponding to the pairing nodes are paired to generate paired tracks; based on the paired tracks, the target observation objects corresponding to the pairing group are determined.

[0033] After completing the measurement data association, the node or single sensor can complete the local state estimation of the target observation object. In order to further improve the accuracy of the state estimation of the target, it is necessary to fuse the local state estimation between multiple machines. Track association is the key basis for realizing distributed multi-source data fusion. Multiple machines share the state estimation pair (x, P) of the target with each other and realize track association based on the Hungarian algorithm; where x represents the state estimation value and P represents the state covariance matrix.

[0034] The asynchrony of multi-machine measurements and communication delays will lead to asynchrony of local estimates. However, for track association, only the accurate association of the target needs to be completed, and the real-time state estimation does not need to be considered. Therefore, the initialization phase of track association only considers the data asynchrony problem caused by different measurement start times and sampling frequencies. The data between multiple machines are synchronously aligned based on the curve fitting method for subsequent track association.

[0035] The Mahalanobis distance is used as the cost weight between the state estimates to be matched between machines, as shown in formula (1):

[0036]

[0037] in, and represents the state estimation and state covariance matrix of target n in single machine i (or single sensor i), and Represents the state estimation and state covariance matrix of target m in single machine j (or single sensor j).

[0038] Construct a cost matrix D∈R based on possible matching targets n×m , as shown in formula (2):

[0039]

[0040] Under the assumption that the targets of any two nodes can only be uniquely paired, the Hungarian algorithm is used to minimize the total cost function to achieve unique matching of target tracks.

[0041] Under the assumption that the target observation objects between any two nodes can only be uniquely paired, the Hungarian algorithm is applied to minimize the total cost function to achieve unique matching of the target. The Hungarian algorithm ensures that each target observation object can be accurately associated with the corresponding state estimate by finding the minimum total pairing cost in the cost matrix.

[0042] By using the Hungarian matching algorithm and Mahalanobis distance, combined with data synchronization methods, the initialization association of targets can be effectively achieved in multi-sensor systems, which is crucial to improving target recognition and tracking capabilities in complex environments.

[0043] First, the first local state estimate The corresponding sampling time Determine whether the absolute value of the difference between the sampling time and the fusion time is less than the preset threshold; if so, determine that the first local state estimate does not meet the preset condition; if not, determine that the first local state estimate meets the preset condition, and query the second local state estimate that is located before the acquisition time and close to the acquisition time from the second state estimate sequence of the paired node for the target observation object. If a second local state estimate that meets the requirements is found Based on the sampling time, the second local state estimate is calculated using the Kalman prediction method. Perform the first time registration to obtain the first registration state estimate. If no second local state estimate that meets the requirements is found, wait for the next second local state estimate sent by the paired node to the first device; express The local state estimate of the target observation object n in node i at time instant.

[0044] Specifically, if the absolute value of the difference between the sampling moment and the fusion moment is less than the preset threshold, it is determined that the first local state estimation value does not meet the preset condition; if the absolute value of the difference between the sampling moment and the fusion moment is not less than the preset threshold, it is determined that the first local state estimation value meets the preset condition.

[0045] Based on the collection time The second local state estimate is calculated using the Kalman prediction method. Register to the acquisition time to obtain the first registration state estimate And the first registration state covariance matrix The specific calculation formula is shown in formula (3):

[0046]

[0047] Among them, F k Represents the state recursive model of discrete targets; the calculation formula is shown in formula (3):

[0048] F k =I+A k δt Formula (4);

[0049] Where δt is and The time interval between k The equation representing the change of the target continuous state is shown in formula (5):

[0050]

[0051] in, express The derivative of wk is the process noise, describing the prediction error between the physical model and the reference model, Q k is the variance matrix of process noise. Based on the above formula, the following formula (6) can be derived:

[0052] V k =B k δ t Formula (6);

[0053] Among them, B k is the influence matrix of the control input.

[0054] Through the above steps and formulas, the Kalman filter can be effectively used to align and update the first state estimate. When a valid first state estimate is received, efficient state prediction and estimation can be achieved by using the state transfer matrix and covariance update rule. In the case of no valid data, the system is designed to wait for the next communication to ensure the accuracy and real-time performance of the state estimate. This design improves the robustness and accuracy of the multi-target tracking system, making it possible to effectively respond to changes in target states in a dynamic environment.

[0055] As a result, not only the real-time and accuracy of target state estimation between multiple nodes are improved, but also the potential errors caused by data asynchrony are reduced, ultimately providing a reliable foundation for subsequent state fusion.

[0056] Secondly, after the acquisition time registration is completed, the first registration state estimation value and the first local state estimation value are fused based on the Kalman filter measurement update method to output a fused local state estimation value. And the fusion state covariance matrix Specifically, the steps are aimed at and The first stage of fusion is achieved to reduce the uncertainty of single-machine Kalman prediction under communication delay. and are unrelated, so we can refer to the Kalman filter measurement update method to As a virtual measurement, the measurement matrix is ​​the unit matrix I, and the fusion formula is shown in the following formula (7):

[0057]

[0058] in, and They respectively represent the fused local state estimation value and the fused state covariance matrix of the local state estimation value of node i and node j for the target object n after completing the first stage of fusion.

[0059] Afterwards, based on the fusion time, the Kalman prediction method is continued to be used to align the fused local state estimation value to the latest fusion time. Specifically, this step takes into account the real-time requirements of data fusion and the simplicity of the fusion method, and further reduces the error of time alignment by temporally aligning the fused local state estimation value.

[0060] Based on the fusion time t f , the Kalman prediction method is used to estimate the fusion local state of the target observation object n and Perform a second temporal registration to generate a second registration state estimate and the second registration state covariance matrix Will Add to fusion cache queue The specific calculation formula is shown in formula (8):

[0061]

[0062] Among them, V k Calculated based on time deviation.

[0063] As a result, the system can effectively fuse the state estimates of multiple nodes, thereby reducing the uncertainty caused by communication delays. This approach not only improves the accuracy of state estimation, but also ensures the real-time and effectiveness of the system in dynamic environments. This is of great significance in applications such as multi-sensor fusion and target tracking.

[0064] Here, since each node generates a local state estimation value of the observed object and stores it on the local machine, each node generates a corresponding track for different observed objects within a preset time, obtains a number of tracks, and sends the tracks to the first device. Thus, the track corresponding to the target observed object in the paired node obtains a second state estimation sequence.

[0065] In S103, based on a preset rule or model, the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue is fused, and the global state estimation value corresponding to the target observation object at the fusion time is output. The two-stage fusion algorithm based on the covariance crossover method in this embodiment performs well in high-speed dynamic scenes, can update and fuse the state information of multiple nodes in real time, and provides reliable support for the collaborative work of multiple sensors.

[0066] Exemplarily, based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, a fusion cache queue is obtained; for any second registration state estimation value in the fusion cache queue: a second registration covariance matrix corresponding to the second registration state estimation value is obtained; based on the second registration covariance matrix, an information matrix and a first weight corresponding to the information matrix are determined; based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue, several information matrices are obtained; for any information matrix among the several information matrices: a corresponding first weight is applied to the information matrix, and the information matrix with the first weight applied is added to generate an optimal information matrix; based on the optimal information matrix, and the information matrix and the first weight corresponding to each second registration state estimation value in the fusion cache queue, the second registration state estimation value is optimized to output the global state estimation value corresponding to the target observation object at the fusion moment.

[0067] Specifically, the covariance crossover method is used to fusion cache queue All fused local state estimates participating in the first stage fusion Perform fusion processing and output the global state estimation value corresponding to the target observation object at the fusion time; the calculation of the global state estimation value is specifically as follows:

[0068]

[0069] in, is the information matrix, represents the optimal information matrix; w ij is the first weight, satisfying the random constraint

[0070]

[0071] The first weight coefficient can be obtained based on the nonlinear optimization problem, as shown in formula (10):

[0072]

[0073] In order to improve the fusion efficiency, the first weight is updated based on the fast covariance crossover method. The specific first weight updating method is shown in the following formula (11):

[0074]

[0075] in,

[0076] Therefore, the covariance crossover method is used to consistently fuse the information in the asynchronous distributed system, which not only improves the accuracy of the local state estimation value, but also enhances the fusion efficiency of the system in high-speed scenarios. The entire algorithm process covers the update from local state estimation to global state estimation, ensuring the reliability and consistency of information.

[0077] In a preferred implementation of this embodiment, if the first local state estimation value does not meet the preset conditions, the first local state estimation value is time-aligned based on the fusion moment to generate a first local state estimation value after alignment; and the first local state estimation value after alignment is determined as the second alignment state estimation value.

[0078] Specifically, if the first local state estimate does not meet the preset conditions, it is considered that the accuracy of the Kalman prediction method meets the registration requirements; based on the fusion moment, the Kalman prediction method is used to directly perform a second time registration on the first local state estimate to generate the first local state estimate after registration and the first state covariance matrix after registration.

[0079] Due to the fusion cache queue Includes the second registration state estimate after the first stage fusion And / or the first local state estimate after registration When both exist at the same time, first fuse all based on the covariance cross method. Then, based on the covariance crossover method, the fusion and all Fusion is performed to generate a sub-global state estimate

[0080] Sub-global state estimate and the subglobal state covariance matrix It is calculated by the following formula (12):

[0081]

[0082] in, represents the first local state estimate after registration, Represents the first state covariance matrix after registration.

[0083] This embodiment comprehensively considers the communication delay and inherent problems of sensors that cause data asynchrony, and proposes a two-stage distributed asynchronous fusion framework combined with a fast covariance crossover method; the framework uses a two-stage time registration method to effectively reduce the state prediction error of time registration and provide a good initial value for state asynchronous fusion; it solves the problem of significantly increased registration error of existing time registration technologies in high-speed scenarios.

[0084] like Figure 2FIG. 1 is a flow chart of a data fusion method based on two-stage time registration provided by another embodiment of the present invention.

[0085] A data fusion method based on two-stage time registration, wherein the node comprises at least one sensor; each of the sensors is used to observe at least one observation object; comprising:

[0086] S201, associating the current measurement result generated by the node with the target observation object to generate a target measurement result;

[0087] S202, determining a corresponding filtering method based on the sensor type corresponding to the current measurement result;

[0088] S203, filtering the target measurement result based on a filtering method, and outputting a filtered measurement result;

[0089] S204, optimizing the current state of the target observation object according to the current initialization state value of the target observation object and the filtered measurement result, and outputting the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

[0090] In S201, there is no limitation on the method of establishing the association relationship between the current measurement result and the target observation object, which can be implemented based on a model or preset rules. For example, the association between the current measurement result and the target observation object is implemented based on the gating concept.

[0091] Exemplarily, the measurement results corresponding to each of all the observation objects that can be observed by the node are received to generate at least one current measurement result; for any current measurement result among the at least one current measurement result: the observation object corresponding to the current measurement result is taken as the target observation object, and the predicted measurement result corresponding to the predicted state of the target observation object at the current moment is obtained; the Euclidean distance between the current measurement result and the predicted measurement result is determined; if the Euclidean distance is not greater than the gated threshold, it is determined that the current measurement result is associated with the target observation object; the number of current measurement results associated with the target observation object is counted to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, the current measurement result is taken as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, the distance between the observation object of each current measurement result and the target observation object is determined; and the current measurement result with the smallest Euclidean distance is selected from the at least two current measurement results as the target measurement result of the target observation object.

[0092] Specifically, considering the need to track multiple target observation objects in actual scenarios, this step aims to complete the association between node measurement results and target observation states to reduce the probability of erroneous tracking.

[0093] The probability threshold of the measurement result is set based on the gating idea. If the current measurement result and the predicted measurement result h(x i ) is less than or equal to the gate threshold, the match is considered successful, as shown in the following formula (13);

[0094]

[0095] Among them, h(·) is the observation equation, δ d Represents the gating threshold. The value of the gating threshold needs to comprehensively consider the error between the current predicted state and the current measurement result. d When δ is too large, it is easy to cause mismatching. d If the time is too small, the correct current measurement result will be rejected.

[0096]

[0097] Among them, σ h and σ g denote the measurement error and state prediction error respectively, represents the normalized state prediction error, Indicates the transformation from the state dimension to the measurement dimension. When multiple current measurement results are associated with the same target observation object, the Hungarian algorithm is used to achieve one-to-one matching based on the general bipartite matching method. For example: based on the greedy strategy, the measurement with the smaller L2 distance is selected.

[0098] Therefore, the gating-based measurement results and local trajectory data association can effectively reduce the probability of false tracking and improve the accuracy and robustness of multi-target tracking. This process is flexible and can adapt to dynamically changing environments, providing a good solution for multi-target tracking in practical applications.

[0099] In S202 to S203, a suitable filtering method is selected according to the measurement type of the sensor, and EKF, UKF or CKF is selected for nonlinear measurement, and KF is selected for linear measurement. The local state estimate corresponding to the current moment is updated in sequence according to the prior state prediction, uncertainty update, Kalman gain solution and posterior state update process.

[0100] Therefore, combined with the selection of appropriate filtering methods and detailed state update procedures, the accuracy and robustness of multi-target tracking can be effectively improved.

[0101] In a preferred implementation manner of this embodiment, the method also includes: receiving target observation information for the target observation object sent by the sensor; if there is prior information of the target observation object, then based on the target observation information and the prior information, performing attribute recognition on the target observation object to generate a target matching result; generating a target trajectory based on the target matching result; if there is no prior information of the target observation object and the data type of the target observation information meets the observability requirement, then performing Hough detection on the target observation information to generate a target trajectory; if there is no prior information of the target observation object and the data type of the target observation information does not meet the observability requirement, then after receiving the local state estimation value for the target object sent by other nodes, performing trajectory similarity matching on the target observation information and the local state estimation value to generate a target trajectory; obtaining the predicted state value for the target observation object at the current moment output at the previous moment from the target trajectory; and using the predicted state value as the current initialization state value.

[0102] Specifically, the local state estimate is used to indicate the 3D or 2D position of the target observation object. or At the same time, it is also necessary to initialize the state covariance matrix to obtain the initialized state covariance matrix P0. Since the state covariance reflects the uncertainty of the state of the target observation object, a larger initial state covariance matrix can accelerate the filter convergence.

[0103] In a preferred implementation of an embodiment of the present invention, based on the filtered measurement result corresponding to the previous moment adjacent to the current moment of the target observation object and the predicted state value at the previous moment, the current state of the target observation object at the previous moment is optimized to generate a local state estimation value corresponding to the target object at the previous moment and the predicted state value at the current moment.

[0104] The following will describe in detail a data fusion method based on two-stage time registration provided by this embodiment in conjunction with a specific application scenario.

[0105] A data fusion based on two-stage time registration, comprising: comprising a plurality of nodes for performing state estimation on an observed object; the nodes comprising at least one sensor; each of the sensors being used to observe at least one observed object and perform state estimation on the observed object; each of the nodes being communicatively connected to a first device; the method being applied to the first device;

[0106] S1, based on the gating idea, realizes the association between the current measurement result and the target observation object.

[0107] Receive the measurement results corresponding to each of all the observation objects that can be observed by the node, and generate at least one current measurement result; for any current measurement result among the at least one current measurement result: take the observation object corresponding to the current measurement result as the target observation object, and obtain the predicted measurement result corresponding to the predicted state of the target observation object at the current moment; determine the Euclidean distance between the current measurement result and the predicted measurement result; if the Euclidean distance is not greater than the gated threshold, determine that the current measurement result is associated with the target observation object; count the number of current measurement results associated with the target observation object to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, take the current measurement result as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, determine the distance between the observation object of each current measurement result and the target observation object; select the current measurement result with the smallest Euclidean distance from the at least two current measurement results as the target measurement result of the target observation object.

[0108] S2, determining the local state estimation value corresponding to the current measurement result based on the Kalman filtering method.

[0109] Based on the sensor type corresponding to the current measurement result, a corresponding filtering method is determined; based on the filtering method, the target measurement result is filtered and the filtered measurement result is output; according to the predicted state value of the target observation object at the current moment and the filtered measurement result, the current state of the target observation object is optimized, and the local state estimation value corresponding to the target observation object at the current moment is output.

[0110] S3, based on the Hungarian matching algorithm, completes the track association of the target observation object between the communicable nodes within the sliding window and specifies the target ID.

[0111] Acquire a plurality of tracks generated by each of the plurality of nodes within a preset time; wherein the track is used to indicate the action trajectory generated by the node for the observable observation object, and each of the observation objects has a corresponding track; each track is formed by a plurality of local state estimation values ​​arranged in chronological order; select any two nodes from the plurality of nodes as a pairing group: pair a plurality of first tracks corresponding to the target nodes in the pairing group and a plurality of second tracks corresponding to the pairing nodes to generate paired tracks; determine the target observation object corresponding to the pairing group based on the paired tracks.

[0112] S4, when the first device receives the first local state estimation value sent from other nodes, it parses the data packet and searches for the local state estimation value closest in time dimension in the state estimation sequence corresponding to the target ID according to the sampling time of the first local state estimation value; then applies the Kalman filtering method to align the local state estimation for time registration.

[0113] Based on the first local state estimation value of each node for the target observation object in the same time window, a first state estimation queue is generated; wherein each node sends the local state estimation value of the target observation object to the first device once in the same time window. For any first local state estimation value in the first state estimation queue: obtain the sampling time of the first local state estimation value; if the first local state estimation value meets the preset conditions, obtain the second local state estimation value closest to the sampling time from the second state estimation sequence of the paired node for the target observation object; based on the sampling time, perform time alignment on the second local state estimation value to obtain a first aligned state estimation value; then execute step S6;

[0114] If the corresponding local state estimation value is not obtained, wait for the node to send the local state estimation value next time; then return to step S4.

[0115] S5: If the first local state estimation value does not meet the preset conditions, then based on the fusion time, the first local state estimation value is time-aligned to generate a first local state estimation value after alignment; and the first local state estimation value after alignment is determined as a second alignment state estimation value.

[0116] S6, after completing the time alignment, perform the first stage fusion according to the Kalman measurement update method; and align the fused local state estimation value generated after the fusion to the latest fusion time.

[0117] The first registration state estimation value and the first local state estimation value are fused to output a fused local state estimation value. Based on the fusion time, the fused local state estimation value is temporally registered to generate a second registration state estimation value; wherein the paired node is used to indicate a node that can perform trajectory matching with the target node corresponding to the first local state estimation value; and then S7 is executed.

[0118] S7, obtaining a fusion cache queue based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue; wherein the fusion cache queue includes: the second registration state estimation value after the first stage fusion processing and the first local state estimation value after registration.

[0119] S8, based on the covariance crossover method, all the second registration state estimation values ​​participating in the first stage fusion processing in the fusion cache queue are fused again to achieve information consistency fusion.

[0120] For any second registration state estimation value in the fusion cache queue that has participated in the first stage fusion processing: obtain the second registration covariance matrix corresponding to the second registration state estimation value; determine the information matrix and the first weight corresponding to the information matrix based on the second registration covariance matrix. Based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue, obtain several information matrices; for any information matrix among the several information matrices: apply the corresponding first weight to the information matrix, and add the information matrix after applying the first weight to generate an optimal information matrix; based on the optimal information matrix, and the information matrix and the first weight corresponding to each second registration state estimation value in the fusion cache queue, optimize the second registration state estimation value, and output the global state estimation value corresponding to the target observation object at the fusion moment.

[0121] For the first local state estimate after registration in the fusion cache queue: use the covariance intersection method to calculate the global state estimate and all Fusion is performed to generate a sub-global state estimate

[0122] The time registration method based on Kalman filtering in this embodiment has both real-time and precision requirements, and can simultaneously solve the data asynchrony problem caused by different sensor sampling frequencies and inconsistent sensor sampling start times. Especially in the assumption of high-maneuverability motion model of the target, the error of the typical time registration algorithm increases. Further considering the communication delay factor, the low communication bandwidth leads to a decrease in the registration accuracy and state estimation accuracy of the target observation data. To this end, this embodiment uses the Kalman filtering method to achieve alignment and synchronization of the sampled data, and then uses a two-stage weighted asynchronous fusion technology to reduce the impact of communication delay on fusion accuracy.

[0123] like Figure 3 FIG. 1 is a schematic diagram of the structure of a data fusion device based on two-stage time registration provided by an embodiment of the present invention.

[0124] A data fusion device based on two-stage time alignment, comprising a plurality of nodes for state estimation of an observed object; each of the nodes is in communication connection with a first device; the method is applied to the first device; the device 300 comprises: a first generation module 301, for generating a first state estimation queue based on a first local state estimation value of each node for a target observed object within the same time window; wherein each node sends a local state estimation value of the target observed object to the first device once within the same time window; a second generation module 302, for obtaining a sampling time of the first local state estimation value for any first local state estimation value in the first state estimation queue; if the first local state estimation value meets a preset condition, the second state estimation value of the target observed object from the paired node is generated. Obtain a second local state estimation value closest to the acquisition time in the estimation sequence; based on the sampling time, perform time alignment on the second local state estimation value to obtain a first aligned state estimation value; fuse the first aligned state estimation value and the first local state estimation value to output a fused local state estimation value; based on the fusion time, perform time alignment on the fused local state estimation value to generate a second aligned state estimation value; wherein the paired node is used to indicate a node that can perform trajectory matching with the target node corresponding to the first local state estimation value; a fusion processing module 303 is used to perform fusion processing based on the second aligned state estimation value corresponding to each first local state estimation value in the first state estimation queue, and output a global state estimation value corresponding to the target observation object at the fusion time.

[0125] In a preferred implementation manner of this embodiment, the third generating module is used to perform time registration on the first local state estimation value based on a fusion time to generate a second registered state estimation value if the first local state estimation value does not meet a preset condition.

[0126] In a preferred implementation manner of this embodiment, the device also includes: an acquisition module, used to acquire a number of tracks generated by each of the several nodes within a preset time; wherein the track is used to indicate the action trajectory generated by the node for the observable observation object, and each of the observation objects has a corresponding track; each track is formed by a number of local state estimation values ​​arranged in chronological order; a fourth generation module, used to select any two nodes from the several nodes as a pairing group: pair the several first tracks corresponding to the target nodes in the pairing group and the several second tracks corresponding to the pairing nodes to generate a pairing result; a selection module, used to select the target track corresponding to the pairing node from the several second tracks based on the pairing result.

[0127] In a preferred implementation manner of this embodiment, the fusion processing module includes: a first obtaining unit, which is used to obtain a fusion cache queue based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue; a determining unit, which is used to obtain, for any second registration state estimation value in the fusion cache queue: a second registration covariance matrix corresponding to the second registration state estimation value; determine an information matrix and a first weight corresponding to the information matrix based on the second registration covariance matrix; a second obtaining unit, which is used to obtain several information matrices based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue; a generating unit, which is used to apply the corresponding first weight to the information matrix for any information matrix among the several information matrices, and add the information matrices after applying the first weight to generate an optimal information matrix; an optimization processing unit, which is used to optimize the second registration state estimation value based on the optimal information matrix, the information matrix corresponding to each second registration state estimation value in the fusion cache queue and the first weight, and output the global state estimation value corresponding to the target observation object at the fusion moment.

[0128] like Figure 4 FIG. 1 is a schematic diagram of the structure of a data fusion device based on two-stage time registration provided by another embodiment of the present invention.

[0129] A data fusion state based on two-stage time alignment is applied to the node: the node includes at least one sensor; each of the sensors is used to observe at least one observation object; the device 400 includes: a first generation module 401, used to associate the current measurement result generated by the node with the target observation object to generate the target measurement result; a first determination module 402, used to determine the corresponding filtering method based on the sensor type corresponding to the current measurement result; a filtering module 403, used to filter the target measurement result based on the filtering method, and output the filtered measurement result; an optimization module 404, used to optimize the current state of the target observation object according to the current initialization state value of the target observation object and the filtered measurement result, and output the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

[0130] In a preferred implementation manner of this embodiment, the first generation module includes: a receiving unit, which is used to receive the measurement results corresponding to each of all the observation objects that can be observed by the node, and generate at least one current measurement result; a determination unit, which is used to: for any current measurement result among the at least one current measurement result: take the observation object corresponding to the current measurement result as the target observation object, and obtain the predicted measurement result corresponding to the predicted state of the target observation object at the current moment; determine the Euclidean distance between the current measurement result and the predicted measurement result; if the Euclidean distance is not greater than the gated threshold, determine that the current measurement result is associated with the target observation object; a selection unit, which is used to count the number of current measurement results associated with the target observation object to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, use the current measurement result as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, determine the distance between the observation object of each current measurement result and the target observation object; select the current measurement result with the smallest Euclidean distance from the at least two current measurement results as the target measurement result of the target observation object.

[0131] In a preferred implementation manner of this embodiment, the device also includes: a receiving module, which is used to receive target observation information for the target observation object sent by the sensor; a second generating module, which is used to, if there is prior information of the target observation object, perform attribute recognition on the target observation object based on the target observation information and the prior information to generate a target matching result; and generate a target trajectory based on the target matching result; a third generating module, which is used to, if there is no prior information of the target observation object and the data type of the target observation information meets the observability requirement, perform Hough detection on the target observation information to generate a target trajectory; a fourth generating module, which is used to, if there is no prior information of the target observation object and the data type of the target observation information does not meet the observability requirement, perform trajectory similarity matching on the target observation information and the local state estimation value after receiving the local state estimation value for the target object sent by other nodes to generate a target trajectory; a second determining module, which is used to obtain the predicted state value for the target observation object at the current moment output at the previous moment from the target trajectory; and use the predicted state value as the current initialization state value.

[0132] The above device can execute a data fusion method based on two-stage time registration provided by an embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing a data fusion method based on two-stage time registration. For technical details not described in detail in this embodiment, please refer to a data fusion method based on two-stage time registration provided by an embodiment of the present invention.

[0133] The present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a data fusion method based on two-stage time alignment described in the present invention.

[0134] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0135] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0136] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method according to the following embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0137] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0138] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0139] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0140] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0141] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0142] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0143] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0144] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0145] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A data fusion method based on two-stage temporal registration, characterized in that: The method comprises a plurality of nodes for performing state estimation on an observed object; each of the nodes is communicatively connected to a first device; the method is applied to the first device; and comprises: Generate a first state estimation queue based on a first local state estimation value of each of the nodes for the target observation object within the same time window; wherein each of the nodes sends a local state estimation value of the target observation object to the first device once within the same time window; For any first local state estimation value in the first state estimation queue: obtain the sampling time of the first local state estimation value; if the first local state estimation value meets the preset condition, obtain the second local state estimation value closest to the sampling time from the second state estimation sequence of the paired node for the target observation object; based on the sampling time, perform time alignment on the second local state estimation value to obtain a first aligned state estimation value; fuse the first aligned state estimation value and the first local state estimation value to output a fused local state estimation value; based on the fusion time, perform time alignment on the fused local state estimation value to generate a second aligned state estimation value; wherein the paired node is used to indicate a node that can perform trajectory matching with the target node corresponding to the first local state estimation value for the target object; A fusion process is performed based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and a global state estimation value corresponding to the target observation object at the fusion moment is output.

2. The method according to claim 1, characterized in that: Also includes: If the first local state estimation value does not meet the preset condition, based on the fusion time, performing time registration on the first local state estimation value to generate a registered first local state estimation value; The registered first local state estimation value is determined as a second registered state estimation value.

3. The method according to claim 1, characterized in that Also includes: Acquire a plurality of tracks generated by each of the plurality of nodes within a preset time; wherein the track is used to indicate the action trajectory generated by the node for the observable observation object, and each observation object has a corresponding track; each track is formed by a plurality of local state estimation values ​​arranged in time order; Selecting any two nodes from the plurality of nodes as a pairing group: performing track pairing on a plurality of first tracks corresponding to the target nodes in the pairing group and a plurality of second tracks corresponding to the pairing nodes to generate paired tracks; Based on the paired tracks, a target observation object corresponding to the paired group is determined.

4. The method according to claim 1, characterized in that: The method comprises: performing fusion processing based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and outputting the global state estimation value corresponding to the target observation object at the fusion time; comprising: Obtaining a fusion cache queue based on a second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue; For any second registration state estimation value in the fusion cache queue: obtaining a second registration covariance matrix corresponding to the second registration state estimation value; determining an information matrix and a first weight corresponding to the information matrix based on the second registration covariance matrix; Based on the information matrix corresponding to each second registration state estimation value in the fusion cache queue, a plurality of information matrices are obtained; For any information matrix among the plurality of information matrices: applying a corresponding first weight to the information matrix, and summing the information matrices after applying the first weight to generate an optimal information matrix; Based on the optimal information matrix, and the information matrix and the first weight corresponding to each second registration state estimation value in the fusion cache queue, the second registration state estimation value is optimized to output the global state estimation value corresponding to the target observation object at the fusion moment.

5. A data fusion method based on two-stage temporal registration, characterized in that: Applied to the node: the node includes at least one sensor; each of the sensors is used to observe at least one observation object; including: Associating the current measurement result generated by the node with the target observation object to generate a target measurement result; Determining a corresponding filtering method based on the sensor type corresponding to the current measurement result; Performing filtering processing on the target measurement result based on the filtering method, and outputting the filtered measurement result; According to the current initialization state value of the target observation object and the filtered measurement result, the current state of the target observation object is optimized, and the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment are output.

6. The method according to claim 5, characterized in that The step of associating the current measurement result generated by the node with the target object to generate the target measurement result comprises: Receiving a measurement result corresponding to each of all the observation objects that can be observed by the node, and generating at least one current measurement result; For any current measurement result of the at least one current measurement result: taking the observation object corresponding to the current measurement result as the target observation object, obtaining the predicted measurement result corresponding to the predicted state of the target observation object at the current moment; determining the Euclidean distance between the current measurement result and the predicted measurement result; if the Euclidean distance is not greater than the gating threshold, determining that the current measurement result is associated with the target observation object; Count the number of current measurement results associated with the target observation object to obtain a statistical result; if the statistical result indicates that only one current measurement result is associated with the target observation object, use the current measurement result as the target measurement result of the target observation object; if the statistical result indicates that at least two current measurement results are associated with the target observation object, determine the distance between the observation object of each current measurement result and the target observation object; select the current measurement result with the smallest Euclidean distance from the at least two current measurement results as the target measurement result of the target observation object.

7. The method according to claim 5, characterized in that Also includes: Receiving target observation information for a target observation object sent by a sensor; If there is prior information of the target observation object, then based on the target observation information and the prior information, attribute recognition is performed on the target observation object to generate a target matching result; generating a target trajectory based on the target matching result; If there is no prior information of the target observation object and the data type of the target observation information meets the observability requirement, perform Hough detection on the target observation information to generate a target trajectory; If there is no prior information of the target observation object and the data type of the target observation information does not meet the observability requirement, then after receiving the local state estimation value for the target object sent by other nodes, the target observation information and the local state estimation value are matched for trajectory similarity to generate the target trajectory; The predicted state value of the target observation object at the current moment output at the previous moment is obtained from the target trajectory; and the predicted state value is used as the current initialization state value.

8. A data fusion device based on two-stage time registration, characterized in that: The method comprises a plurality of nodes for performing state estimation on an observed object; each of the nodes is communicatively connected to a first device; the method is applied to the first device; and comprises: A first generating module, configured to generate a first state estimation queue based on a first local state estimation value of each of the nodes for a target observation object within a same time window; wherein each of the nodes sends a local state estimation value of the target observation object to the first device once within the same time window; The second generation module is used for obtaining the sampling time of the first local state estimation value for any first local state estimation value in the first state estimation queue; if the first local state estimation value meets the preset condition, obtaining the second local state estimation value closest to the sampling time from the second state estimation sequence of the paired node for the target observation object; based on the sampling time, performing time registration on the second local state estimation value to obtain a first registered state estimation value; fusing the first registered state estimation value and the first local state estimation value to output a fused local state estimation value; based on the fusion time, performing time registration on the fused local state estimation value to generate a second registered state estimation value; wherein the paired node is used to indicate a node that can perform trajectory matching with the target node corresponding to the first local state estimation value for the target object; A fusion processing module is used to perform fusion processing based on the second registration state estimation value corresponding to each first local state estimation value in the first state estimation queue, and output the global state estimation value corresponding to the target observation object at the fusion moment.

9. A data fusion state based on two-stage temporal registration, characterized in that: Applied to the node: the node includes at least one sensor; each of the sensors is used to observe at least one observation object; including: A first generating module, configured to associate a current measurement result generated by the node with a target observation object to generate a target measurement result; A first determination module, configured to determine a corresponding filtering method based on a sensor type corresponding to the current measurement result; A filtering module, used for filtering the target measurement result based on the filtering method, and outputting the filtered measurement result; The optimization module is used to optimize the current state of the target observation object according to the current initialization state value of the target observation object and the filtered measurement result, and output the local state estimation value corresponding to the target observation object at the current moment and the predicted state value at the next moment.

10. A computer readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 4 or claims 5 to 7 is implemented.

Images