A multi-sensing information fusion method based on an interacting multiple model algorithm

Abstract

The application discloses a kind of multi-sensing information fusion methods based on interactive multiple model algorithm, comprising: obtaining the camera target sequence and radar target sequence of two perception objects, time-space alignment is carried out, and the two perception objects after alignment are obtained;Multi-sensing source data correlation is carried out to the two perception objects after alignment, and the object of different perception sources with correlation is obtained;The objects of different perception sources are fused, and the associated object with target information consistent is obtained;The associated object with target information consistent is associated with tracking object based on the nearest neighbor method, and the radar and camera motion state value of the same vehicle obstacle vehicle in association are obtained;The radar and camera motion state value of the same vehicle obstacle vehicle in association obtained are input into interactive multiple model for fusion estimation, and a fusion target list is obtained.The application effectively solves the estimation error fusion problem under different sensors tracking, improves the accuracy and reliability of obstacle vehicle tracking.

Description

Technical Field

[0001] This invention belongs to the field of vehicle intelligent driving and active safety technology, and in particular relates to a multi-sensor information fusion method based on interactive multi-model algorithm. Background Technology

[0002] Intelligent driving assistance systems are considered a key technology for improving vehicle driving safety. However, due to the high cost of systems using lidar, multi-sensor configurations such as millimeter-wave radar and cameras are commonly adopted. While this configuration improves the performance of the perception system to some extent, several challenges remain. Multi-sensor systems may have blind spots, meaning they cannot fully perceive information about traffic vehicles in localized areas. Furthermore, due to the different characteristics and operating principles of each sensor, a single sensor may produce false detections or missed detections, affecting the accurate perception of traffic vehicles. To improve perception accuracy and ensure stable tracking throughout the entire process, it is essential to comprehensively utilize multi-sensor data to achieve accurate fusion of the motion state of traffic vehicles. Literature review reveals that when processing multi-sensor data on the same obstacle vehicle, a key issue is that the motion states acquired by these sensors each carry their own estimation errors. Effectively fusing these erroneous estimates is a challenging problem, requiring consideration of how to comprehensively utilize information from different sensors to improve the system's fusion accuracy. Particularly during stable tracking of obstacle vehicles, significant challenges arise, necessitating methods that comprehensively consider multi-source information to improve the system's robustness. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes a multi-sensor information fusion method based on an interactive multi-model algorithm. By introducing this algorithm, the adaptability of the intelligent driving assistance system is significantly improved, thereby enhancing the accuracy and reliability of obstacle and vehicle tracking.

[0004] To achieve the above objectives, the present invention provides a multi-sensor information fusion method based on an interactive multi-model algorithm, comprising:

[0005] Acquire camera target sequences and radar target sequences of two sensing objects, and perform spatiotemporal alignment on the camera target sequences and radar target sequences to obtain the two aligned sensing objects;

[0006] Multi-sensor source data association is performed on the two aligned sensing objects to obtain objects from different sensing sources with related relationships; the objects from different sensing sources are fused to obtain associated objects with consistent target information; the associated objects with consistent target information are associated with the tracking object based on the nearest neighbor method to obtain the radar and camera motion state values ​​associated with the same obstacle vehicle.

[0007] The obtained radar and camera motion state values ​​associated with the same obstacle vehicle are input into the interactive multi-model for fusion estimation to obtain a fused target list.

[0008] According to the multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention, before spatiotemporally aligning the camera target sequence and the radar target sequence, spatial calibration of the sensors is required to obtain the coordinate transformation relationship from the radar Cartesian coordinate system to the vehicle Cartesian coordinate system; wherein the spatial calibration method is as follows:

[0009] Let the radar rectangular coordinate system be denoted as S. r The vehicle body is marked in Cartesian coordinate system as S v Establish S r To S v The coordinate transformation relationship between them is as follows:

[0010] S v =RS r +T

[0011]

[0012]

[0013] Where R is the rotation matrix, T is the translation matrix, Δx is the translation distance in the x-direction, Δy is the translation distance in the y-direction, and Δz is the translation distance in the z-direction.

[0014] The multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention includes the following method for associating multi-sensor source data of the aligned two sensing objects to obtain objects with different sensing sources having an association relationship:

[0015] Multi-sensor data association is performed on the two aligned sensing objects. The DS evidence theory is introduced, and mass function calculations considering positional similarity and velocity similarity are performed. The DS synthesis rule is used to determine the successful association of the two aligned sensing objects, thereby obtaining objects from different sensing sources with association relationships.

[0016] The multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention fuses objects from different sensing sources to obtain associated objects with consistent target information as follows:

[0017]

[0018] Among them, X A To associate the motion state of the fused objects, P C P represents the error covariance of the camera. R Let X be the error covariance of the radar.R For the radar's motion state, X C For the motion state of the camera, P A The error covariance of the associated fusion objects.

[0019] The multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention performs data association between the associated objects with consistent target information and the tracked objects based on the nearest neighbor method to obtain the radar and camera motion state values ​​associated with the same obstacle vehicle.

[0020] |Z(k)-H(k)X(k|k-1)|=|d i (k)|≤P G ·σ i i = 1, 2, ..., m

[0021]

[0022] d 2 (z(k))=[z(k)-z(k|k-1)] T S -1 (k)[z(k)-z(k|k-1)]=v T (k)S -1 (k)v(k)

[0023] Where Z(k) is the candidate measurement, H(k) is the measurement transformation matrix, k is the k-th time, X(k|k-1) is the prior estimate, and P G The tracking gate for the target contains the probability of the actual measurement, σ i Let m be the standard deviation of the i-th residual, where i is the i-th measurement and m is the number of measurements. Let R be the observation noise covariance moment and R be the i-th diagonal element of the prediction covariance matrix P(k|k-1), respectively; d be the distance between the measurement and the prior estimate; z(k|k-1) be the prior estimate of the measurement; S(k) be the covariance of the filter residual; and v(k) be the filter residual.

[0024] The multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention includes inputting the obtained radar and camera motion state values ​​associated with the same obstacle vehicle into the interactive multi-model for fusion estimation to obtain a fused target list.

[0025] The radar and camera motion state values ​​associated with the same obstacle vehicle are used as input values. Kalman filtering is used to filter and estimate the initial state of the two filtering models to obtain the state estimate and corresponding covariance matrix of each model at a certain time.

[0026] Based on the input values ​​and the state estimates and corresponding covariance matrices of each model, the residuals of the prior estimates and their corresponding covariance matrices are obtained.

[0027] Based on the prior estimated residuals and the corresponding covariance matrix, the likelihood function of each model is calculated, and the probability of each model is updated based on Bayesian probability calculation.

[0028] Based on the updated model probabilities, the final motion state estimate and the corresponding covariance matrix are obtained.

[0029] The final motion state estimate and the corresponding covariance matrix are input into the interactive multi-model to obtain the fusion target list.

[0030] According to the multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention, the method for obtaining the residuals of the prior estimate and the corresponding covariance matrix based on the input value and the state estimates and corresponding covariance matrices of each model is as follows:

[0031]

[0032]

[0033] in, The residuals are the prior estimates. To obtain the radar and camera motion state values ​​associated with the same obstacle vehicle, H is the measurement transformation matrix. These are the prior estimates for the Kalman filter. For the measured predicted value, The covariance matrix is ​​the prior estimate. For the error covariance, R i To observe the noise covariance.

[0034] According to the multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention, the likelihood function of each model is calculated based on the prior estimated residuals and the corresponding covariance matrix, and the probability of each model is further updated based on Bayesian probability calculation as follows:

[0035]

[0036]

[0037]

[0038] in, Let i be the probability density function of model i. The residuals are the prior estimates. The covariance matrix is ​​the prior estimate. MTo measure the dimension of the model, For model weights, C represents the probability that the input interaction is within this model. k For model transition probabilities, r This represents the number of models.

[0039] The multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention obtains the final motion state estimate and the corresponding covariance matrix based on the updated model probabilities as follows:

[0040]

[0041]

[0042] in, This is the final state vector. r For the number of models, Let k be the target motion state at time k. Let P be the weight value corresponding to the i-th model. k For the final covariance, Let be the covariance of the i-th model at time k.

[0043] The multi-sensor information fusion method based on the interactive multi-model algorithm provided by the present invention, wherein the final motion state estimate and the corresponding covariance matrix are input into the interactive multi-model to obtain the fusion target list, is as follows:

[0044]

[0045]

[0046] in, The probability that the input interaction is within this model. k For time k, i Let π represent the i-th model, and let π be the state transition matrix. j|i Given the Markov state transition matrix a priori, This represents the model probability.

[0047] Technical advantages of this invention: This invention discloses a multi-sensor information fusion method based on an interactive multi-model algorithm. It employs an interactive multi-model fusion algorithm as the solution, enabling the intelligent driving assistance system to adaptively correct the weights of sub-models in real time and fuse the output results of the filter estimators established by each sensor based on these weights. This strategy effectively solves the problem of estimation error fusion under different sensor tracking conditions. By introducing the interactive multi-model algorithm, the adaptability of the intelligent driving assistance system is significantly improved, thereby enhancing the accuracy and reliability of obstacle and vehicle tracking. Attached Figure Description

[0048] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0049] Figure 1 This is a flowchart illustrating a multi-sensor information fusion method based on an interactive multi-model algorithm according to an embodiment of the present invention.

[0050] Figure 2 This is a time alignment scheme for millimeter-wave radar and camera according to an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of a data association method based on DS evidence theory according to an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of a data association method based on the nearest neighbor method according to an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram illustrating information fusion based on the interactive multi-model algorithm in an embodiment of the present invention. Detailed Implementation

[0054] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0055] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0056] like Figure 1 As shown, this embodiment provides a multi-sensor information fusion method based on the interactive multi-model algorithm, including the following steps:

[0057] The camera target sequence and radar target sequence of two sensing objects are obtained, and the camera target sequence and radar target sequence are spatiotemporally aligned to obtain the two sensing objects after alignment.

[0058] The two aligned sensing objects are associated with multi-sensor data based on DS evidence theory to obtain objects from different sensing sources with related relationships; maximum likelihood fusion is performed on objects from different sensing sources to obtain associated objects with consistent target information; the associated objects with consistent target information are associated with the tracking object based on the nearest neighbor method to obtain the radar and camera motion state values ​​of the vehicle associated with the same obstacle.

[0059] The obtained radar and camera motion state values ​​associated with the same obstacle vehicle are input into the interactive multi-model for fusion estimation to obtain a fused target list.

[0060] Furthermore, before performing spatiotemporal alignment of the camera target sequence and the radar target sequence, it is necessary to perform spatial calibration of the sensors to obtain the coordinate transformation relationship from the radar Cartesian coordinate system to the vehicle body Cartesian coordinate system, such as... Figure 2 As shown, a synchronous update approach is adopted, which defines a fusion frequency and periodically synchronizes the latest data from each sensor to the same point in time for updates. This method allows for updates to the target list at regular intervals, reducing the instability that frequent updates might cause. This synchronous update strategy better adapts to the actual needs of the system, balancing real-time performance and stability. The key to achieving spatial alignment lies in the accurate spatial calibration of the sensors. In this invention, the aim is to obtain the coordinate transformation relationship from the radar Cartesian coordinate system to the vehicle body Cartesian coordinate system. This process involves two main aspects: translation and rotation, specifically the translation of the origin of the coordinate system and the rotation of each axis.

[0061] Let the radar rectangular coordinate system be denoted as S. r The vehicle body is marked in Cartesian coordinate system as S v Establish S r To S v The coordinate transformation relationship between them is as follows:

[0062] S v =RS r +T (1)

[0063]

[0064]

[0065] Where R is the rotation matrix, T is the translation matrix, Δx is the translation distance in the x-direction, Δy is the translation distance in the y-direction, and Δz is the translation distance in the z-direction.

[0066] In the calibration test, this invention selected a straight rod on a horizontal surface and recorded its coordinates in the vehicle coordinate system and the corresponding radar measurement results. To ensure the accuracy of the calibration, this invention collected multiple sets of data, typically three or more. Using laser rangefinder and radar scanning data, this invention obtained the coordinates of the straight rod in the vehicle coordinate system. and radar measurement results Subsequently, the rotation transformation matrix R (3×3) and translation transformation matrix T (3×1) were obtained using Matlab software.

[0067] Furthermore, methods for associating multi-sensor data between two aligned sensing objects to obtain objects from different sensing sources with related relationships include:

[0068] Multi-sensor data association is performed on the two aligned sensing objects. The DS evidence theory is introduced to calculate the mass function considering both positional similarity and velocity similarity. The DS synthesis rule is used to determine the successful association of the two aligned sensing objects, thereby obtaining objects from different sensing sources that have an association relationship.

[0069] Specifically, such as Figure 3 As shown, for a given two objects Obj x and Obj y First, the identification framework for whether they are the same object (i.e., whether they are related) is defined as Θ = {1, 0} (4).

[0070] In the identification framework, 1 represents Obj. x and Obj y {1} represents the same object, while 0 represents different objects. Furthermore, to more comprehensively account for uncertainty, this invention defines three possible subsets: {1} represents the same object, {0} represents different objects, and {1,0} represents the situation where it cannot be determined whether they are the same object (i.e., the unknown case).

[0071] Pro1: The basic probability assignment function for position is as follows:

[0072]

[0073] In the formula

[0074]

[0075] Where, λ pos For distance-related proportional parameters, The Mahalanobis distance between the two locations is:

[0076] Pro2: The basic probability assignment function for velocity is as follows:

[0077]

[0078] In the formula

[0079]

[0080] Where, λ vel For speed-related proportional parameters, The Mahalanobis distance representing the velocities of the two objects:

[0081] Pro3: Finally, using the Dempster-Shafer synthesis formula, the probability of each subset under the recognition frame Θ is calculated. Let A, B, and C be three subsets with non-zero probabilities under Θ. Then, the probability of inferring that it is subset A is:

[0082]

[0083]

[0084] By comparison as well as The probability of a subset, if If the probability is highest, then Obj is determined. x and Obj y The two objects are the same object, thus confirming the successful association.

[0085] Furthermore, the method for fusing objects from different sensing sources to obtain associated objects with consistent target information specifically includes: In the associated object fusion stage, this invention introduces a maximum likelihood estimation method, assuming that for the same traffic vehicle associated with it, the motion state of the radar and camera sensing objects and the corresponding error covariance are respectively (X... R ,P R ) and (X C ,P C The result after fusion is:

[0086]

[0087] Among them, X A To associate the motion state of the fused objects, P C P represents the error covariance of the camera. R Let X be the error covariance of the radar. R For the radar's motion state, X C For the motion state of the camera, P A The error covariance of the associated fusion objects.

[0088] like Figure 4 As shown, the method for obtaining the radar and camera motion state values ​​of vehicles associated with the same obstacle by performing data association between the associated objects with consistent target information and the tracked objects based on the nearest neighbor method is as follows:

[0089] Pro1: A rectangular tracking gate is a rectangular region centered on the predicted position of the target object in one step. A measurement is considered to fall within the tracking gate of the target when the following relationship is satisfied:

[0090] |Z(k)-H(k)X(k|k-1)|=|v i (k)|≤P G·σ i ,i=1,2,...m (12)

[0091] The standard deviation σ of the i-th residual i for:

[0092] Pro2: The basic idea of ​​the nearest neighbor (NN) method is to consider the measurement that falls within the target tracking gate and is closest to the center of the gate as the target's measurement at the current moment.

[0093] d 2 (z(k))=[z(k)-z(k|k-1)] T S -1 (k)[z(k)-z(k|k-1)]=v T (k)S -1 (k)v(k) (14)

[0094] Where Z(k) is the candidate measurement, H(k) is the measurement transformation matrix, k is the k-th time, X(k|k-1) is the prior estimate, and P G The tracking gate for the target contains the probability of the actual measurement, σ i Let m be the standard deviation of the i-th residual, where i is the i-th measurement and m is the number of measurements. Let R be the observation noise covariance moment and R be the i-th diagonal element of the prediction covariance matrix P(k|k-1), respectively; d be the distance between the measurement and the prior estimate; z(k|k-1) be the prior estimate of the measurement; S(k) be the covariance of the filter residual; and v(k) be the filter residual.

[0095] Furthermore, the methods for obtaining a fused target list by inputting the obtained radar and camera motion state values ​​associated with the same obstacle vehicle into the interactive multi-model for fusion estimation include:

[0096] Input the final motion state estimates and corresponding covariance matrices of each model in the previous frame into the interactive multi-model to obtain the initial motion state estimates and covariance matrices for each model in this frame.

[0097] The radar and camera motion state values ​​associated with the same obstacle vehicle are used as input values. Kalman filtering is used to filter and estimate the initial state of the two filtering models to obtain the state estimate and corresponding covariance matrix of each model at the current time.

[0098] Based on the input values ​​and the state estimates and corresponding covariance matrices of each model, the residuals of the prior estimates and their corresponding covariance matrices are obtained.

[0099] The likelihood function of each model is calculated based on the prior estimated residuals and the corresponding covariance matrix. Then, the probability of each model is updated based on Bayesian probability calculation.

[0100] Based on the updated model probabilities, the final motion state estimate and the corresponding covariance matrix are obtained, and all the final information is summarized into the fusion target list.

[0101] Specifically, such as Figure 5 As shown, Pro1: Input Interaction. First, assume that the target motion state of the i-th model at time k-1 has been obtained. covariance Model Probability And the state transition matrix π.

[0102]

[0103]

[0104] in, Let be the probability of the input interaction being in this model, k-1 be the (k-1)th time step, i represent the i-th model, and π be the prior Markov state transition matrix. This represents the model probability.

[0105] The calculation of the two parameters above is mainly for the purpose of interacting with the target motion state and covariance obtained at time k-1:

[0106]

[0107]

[0108] In the formula, and The goal after interaction is used for the initial state estimation and covariance matrix of each model.

[0109] Pro2: Filtering. Assume that radar and camera state estimates associated with the same obstacle vehicle have been successfully obtained. Then, using these associated measurements as input, Kalman filtering is applied to estimate the initial states of the two filtered models, thus obtaining the state estimates and corresponding covariance matrices of each model at time k. and

[0110] Measurements were also obtained. Residuals compared with prior estimates and the corresponding covariance matrix

[0111]

[0112]

[0113] in, The residuals are the prior estimates. To obtain the radar and camera motion state values ​​associated with the same obstacle vehicle, H is the measurement transformation matrix. These are the prior estimates for the Kalman filter. For the measured predicted value, The covariance matrix is ​​the prior estimate. For the error covariance, R i To observe the noise covariance.

[0114] Pro3: Model Probability Update. The probability obtained in the previous step... and S k Calculate the likelihood function for each model:

[0115]

[0116] Next, update the model probability according to the Bayesian probability formula:

[0117]

[0118]

[0119] in, Let i be the probability density function of model i. The residuals are the prior estimates. The covariance matrix is ​​the prior estimate. M To measure the dimension of the model, For model weights, C represents the probability that the input interaction is within this model. k For model transition probabilities, r This represents the number of models.

[0120] Pro4: The method for obtaining the final motion state estimate and corresponding covariance matrix based on the updated model probabilities is as follows:

[0121]

[0122]

[0123] in, This is the final state vector. r For the number of models, Let k be the target motion state at time k. Let P be the weight value corresponding to the i-th model. k For the final covariance, Let be the covariance of the i-th model at time k.

[0124] This invention discloses a multi-sensor information fusion method based on an interactive multi-model algorithm (IMA). The IMA is used as the solution, enabling the intelligent driving assistance system to adaptively correct the weights of sub-models in real time and fuse the outputs of the filter estimators established by each sensor based on these weights. This strategy effectively solves the problem of estimation error fusion under different sensor tracking conditions. By introducing the IMA, the adaptability of the intelligent driving assistance system is significantly improved, thereby enhancing the accuracy and reliability of obstacle and vehicle tracking.

[0125] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-sensor information fusion method based on interactive multi-model algorithm, characterized in that, include: Acquire camera target sequences and radar target sequences of two sensing objects, and perform spatiotemporal alignment on the camera target sequences and radar target sequences to obtain the two aligned sensing objects; The process of associating the aligned two sensing objects using multi-sensor data to obtain objects from different sensing sources with a correlation relationship includes: associating the aligned two sensing objects using multi-sensor data, introducing DS evidence theory, calculating mass functions considering positional similarity and velocity similarity, using DS synthesis rules to determine the successful association of the aligned two sensing objects, and obtaining objects from different sensing sources with a correlation relationship; fusing the objects from different sensing sources to obtain associated objects with consistent target information; and associating the associated objects with consistent target information with the tracking object using the nearest neighbor method to obtain the radar and camera motion state values ​​of the vehicle associated with the same obstacle. The obtained radar and camera motion state values ​​associated with the same obstacle vehicle are input into the interactive multi-model for fusion estimation to obtain a fused target list, specifically including: The radar and camera motion state values ​​associated with the same obstacle vehicle are used as input values. Kalman filtering is used to filter and estimate the initial state of the two filtering models to obtain the state estimate and corresponding covariance matrix of each model at a certain time. Based on the input values ​​and the state estimates and corresponding covariance matrices of each model, the residuals of the prior estimates and their corresponding covariance matrices are obtained. Based on the prior estimated residuals and the corresponding covariance matrix, the likelihood function of each model is calculated, and the probability of each model is updated based on Bayesian probability calculation. Based on the updated model probabilities, the final motion state estimate and the corresponding covariance matrix are obtained. The final motion state estimate and the corresponding covariance matrix are input into the interactive multi-model to obtain the fusion target list.

2. The multi-sensor information fusion method based on the interactive multi-model algorithm as described in claim 1, characterized in that, Before performing spatiotemporal alignment of the camera target sequence and the radar target sequence, spatial calibration of the sensors is required to obtain the coordinate transformation relationship from the radar Cartesian coordinate system to the vehicle Cartesian coordinate system; wherein the spatial calibration method is as follows: Let the radar rectangular coordinate system be denoted as... The vehicle body is marked in Cartesian coordinate system as ,Establish arrive The coordinate transformation relationship between them is as follows: in, For rotation matrix, It is a translation matrix. The distance is the translation in the x-direction. The distance is the translation in the y-direction. The distance is the translation in the z-direction.

3. The multi-sensor information fusion method based on the interactive multi-model algorithm as described in claim 1, characterized in that, The method for fusing objects from different sensory sources to obtain associated objects with consistent target information is as follows: Among them, To associate the motion state of the fusion objects, Let $\mathbf{a}$ be the error covariance of the camera. The error covariance of the radar. The radar's motion status, The motion state of the camera. The error covariance of the associated fusion objects.

4. The multi-sensor information fusion method based on the interactive multi-model algorithm as described in claim 1, characterized in that, The method for obtaining the radar and camera motion state values ​​of the vehicle associated with the same obstacle by performing data association between the associated object with the tracked object based on the nearest neighbor method is as follows: in, As a candidate measurement, For measurement transformation matrix, For time k, These are prior estimates. The tracking gate for the target contains the probability of the actual measurement. For the first The standard deviation of each residual For the i-th measurement, To measure the number, These are the observation noise covariance moments. and predicting the covariance matrix The One diagonal element, The distance between the measured value and the prior estimate, For prior estimates of the measurement value, The covariance of the filtered residuals. This is the filtering residual.

5. The multi-sensor information fusion method based on the interactive multi-model algorithm as described in claim 1, characterized in that, The method for obtaining the prior estimate residuals and corresponding covariance matrices based on the input values, the state estimates of each model, and the corresponding covariance matrices is as follows: in, The residuals are the prior estimates. To obtain the radar and camera motion state values ​​associated with the same obstacle vehicle, For measurement transformation matrix, These are the prior estimates for the Kalman filter. For the measured predicted value, The covariance matrix is ​​the prior estimate. For error covariance, To observe the noise covariance.

6. The multi-sensor information fusion method based on the interactive multi-model algorithm as described in claim 1, characterized in that, The likelihood function of each model is calculated based on the prior estimated residuals and the corresponding covariance matrix. Further updates to the probabilities of each model are then performed using Bayesian probability calculations. in, Let i be the probability density function of model i. The residuals are the prior estimates. The covariance matrix is ​​the prior estimate. To measure the dimension of the model, For model weights, The probability that the input interaction is within this model. For model transition probabilities, This represents the number of models.

7. The multi-sensor information fusion method based on the interactive multi-model algorithm as described in claim 1, characterized in that, The method for obtaining the final motion state estimate and corresponding covariance matrix based on the updated model probabilities is as follows: in, This is the final state vector. For the number of models, Let k be the target motion state at time k. Let i be the weight value corresponding to the i-th model. For the final covariance, Let be the covariance of the i-th model at time k.

8. The multi-sensor information fusion method based on the interactive multi-model algorithm as described in claim 1, characterized in that, The method for inputting the final motion state estimate and the corresponding covariance matrix into the interactive multi-model to obtain the fusion target list is as follows: in, The probability that the input interaction is within this model. For time k, To represent the i-th model, Here is the state transition matrix. Given the Markov state transition matrix a priori, This represents the model probability.

Images