Data fusion vehicle positioning method based on Kalman filtering and reinforcement learning

Abstract

The invention discloses a data fusion vehicle positioning method based on Kalman filtering and reinforcement learning, and belongs to the technical field of multi-sensor data fusion, and the method comprises the following steps: 1, preprocessing observation data obtained by each sensor, and carrying out data interpolation and time alignment; 2, fusing state data at different moments through a Kalman filtering method so as to improve data precision; and 3, the optimal fusion weight of the data of each sensor is adjusted through a reinforcement learning method, and the fusion data precision is further improved. According to the invention, multi-sensor data can be fully fused, the data utilization efficiency is effectively improved, and the vehicle positioning precision is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-sensor data fusion, and in particular to a data fusion vehicle positioning method and system based on Kalman filtering and reinforcement learning. Background Art

[0002] Vehicle positioning technology is an integral component of modern transportation systems, using a variety of sensors and positioning devices to determine a vehicle's location. With the development of intelligent transportation systems, the accuracy and reliability of vehicle positioning data are becoming increasingly important. Current vehicle positioning systems typically rely on multiple positioning technologies, such as the Global Positioning System (GPS), Inertial Navigation System (INS), and other geo-tagging-based sensor systems. However, these systems often suffer from positioning errors when used alone, especially in environments where GPS signals are obstructed, such as urban canyons or tunnels.

[0003] At present, the fusion objects of the multi-sensor fusion algorithm are random. The detection information of multiple sensors is weighted averaged through the weighted averaging method to obtain the fusion result. This method is simple and intuitive, and can directly process the original dynamic data of the sensor in real time. It can effectively eliminate certain accidental errors, but adjusting and setting the weight coefficients takes a lot of time and has a certain degree of subjectivity. Summary of the Invention

[0004] In order to solve the technical problems raised in the above background, the present invention provides a data fusion vehicle positioning method based on Kalman filtering and reinforcement learning, which automatically adjusts and sets weight coefficients in real time to solve the problems of insufficient adaptability to dynamic environments and low positioning accuracy in the existing technology. Kalman filtering is an efficient state estimation method for linear dynamic systems that can effectively handle noise and uncertainty in observation data. Reinforcement learning is a learning method based on environmental feedback that can optimize decision-making strategies in interaction with the environment. By combining Kalman filtering with reinforcement learning, the present invention can not only achieve effective fusion of multi-source data, but also dynamically adjust data processing strategies to adapt to various complex environments, thereby improving the accuracy and reliability of vehicle positioning.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions:

[0006] A vehicle positioning method based on data fusion of Kalman filtering and reinforcement learning, comprising the following steps:

[0007] Step 1: Preprocess sensor data from different sources and propose a method based on cubic B-spline interpolation curve fitting to perform data interpolation and time alignment;

[0008] Step 2: Use the Kalman filter method to fuse the preprocessed data. Specifically, the state data at different times are fused through the state estimation method of feedback control to preliminarily improve the data accuracy.

[0009] Step 3: Based on the reinforcement learning method, the state space, action space and reward function required for multi-sensor fusion are designed to train the fusion model and obtain the optimal fusion weight;

[0010] Step 4: Load the trained fusion model and optimal fusion weights to fuse multi-sensor data. It also supports loading real-time data to optimize the fusion model and adjust the fusion weights, thereby further improving the accuracy of the fusion data.

[0011] Furthermore, the step 1 includes the following contents:

[0012] The discrete measurement values ​​observed by each sensor are , r control points are required Perform interpolation curve fitting. For a cubic B-spline curve, the control points and interpolation points must satisfy the following relationship

[0013]

[0014] After cubic B-spline interpolation fitting, a smooth curve based on the observed values ​​is obtained. This curve can be used to obtain the observed value of the corresponding sensor at any time, achieving data interpolation. Then, by aligning with other sensors, selecting the required sampling time, and extracting the observed value at the corresponding time from the fitted curve of each sensor to achieve time alignment.

[0015] Furthermore, the step 2 includes the following:

[0016] The Kalman filter equation based on feedback control iteration is established. First, the state at a certain moment in the process is estimated, and then feedback is obtained in the form of measurement values. The Kalman filter can be divided into two parts: the time update equation and the measurement update equation.

[0017] The time update equation is responsible for forward calculation of the current state variables and error covariance estimation The value of , which provides the basis for constructing a priori estimates for the next time step, is expressed as follows:

[0018]

[0019]

[0020] The measurement update equation is responsible for feedback, that is, the prior estimate With the new measurement Combined to construct an improved posterior estimate for:

[0021]

[0022]

[0023]

[0024] The time update equation can be regarded as an estimation equation, and the measurement update equation can be regarded as a correction equation. The combination of the two forms an estimation-correction algorithm with a numerical solution, which can obtain the current time estimate and the uncertainty of this state estimate .

[0025] Furthermore, step 3 includes the following:

[0026] Define a finite set of states S, a finite set of actions A, and a reward function R(s, a): At each time step t, the data fusion system based on reinforcement learning receives the state in the state set S , and select an action from action set A , to fuse the multi-sensor observations. Once the operation is performed , the data fusion system calculates a scalar reward according to the reward function R , at which point the system transitions to the next state The data fusion system repeatedly learns through this process until the termination condition is reached and the optimal fusion value is output. The Q function of the multi-sensor learning fusion method based on weight adjustment is defined as

[0027]

[0028] in, is the discount factor; is the horizontal range, i.e. the number of iterations.

[0029] Furthermore, the step 4 includes the following contents:

[0030] First, the fusion model and its corresponding optimal weights, obtained through training using historical datasets, are loaded. Next, the sensor data is received in real time and interpolated and time-aligned. This processed data serves as training data for model optimization and weight adjustment, further improving the system's adaptability and robustness. To continuously monitor the accuracy of the fused data, the following real-time performance evaluation metric, Acc, is used:

[0031]

[0032] Where N is the number of evaluation samples, is the model prediction value, is the actual value and 1 is the indicator function.

[0033] In summary, the present invention mainly has the following beneficial effects:

[0034] 1. By combining Kalman filtering with reinforcement learning, we achieve efficient fusion of multi-sensor data, significantly improving the accuracy and reliability of vehicle positioning. Furthermore, reinforcement learning adaptively adjusts sensor fusion weights to effectively address changes in sensor characteristics under different environments and conditions, enhancing system robustness.

[0035] 2. A time alignment and data interpolation method based on cubic B-spline interpolation was proposed to address the inconsistency of sensor data at different time points. The accuracy of the fused data was continuously monitored through real-time performance evaluation indicators to ensure the system's efficient performance in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is an overall flow chart of multi-source vehicle positioning data fusion according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of a Kalman filtering method according to an embodiment of the present invention;

[0038] Figure 3 This is a comparison chart of the multi-source vehicle positioning data fusion results of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention provides a data fusion vehicle positioning method based on Kalman filtering and reinforcement learning. The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and embodiments:

[0040] In a specific embodiment of the present invention, the task scenario is: a multi-sensor collaborative vehicle positioning method, the implementation scenario is that three radar sensors measure the distance of the vehicle respectively at the same position, and it is necessary to combine the data of each sensor to obtain accurate vehicle distance data.

[0041] Step 1: Preprocess sensor data from different sources and propose a method based on cubic B-spline interpolation curve fitting to perform data interpolation and time alignment;

[0042] This example uses observation data from three radars with sampling frequencies of 60Hz, 30Hz, and 20Hz respectively. The following table shows some of the data used in this simulation experiment, where I1, I2, and I3 are the target distance data sent back by the three sensors. The actual distance data of the target.

[0043] Table 1 Sensor raw data display table

[0044]

[0045] The discrete measurement values ​​observed by each sensor are , r control points are required Perform interpolation curve fitting. For a cubic B-spline curve, the control points and interpolation points must satisfy the following relationship

[0046]

[0047] Also consider the boundary conditions

[0048]

[0049] The following equations can be obtained together

[0050]

[0051] So based on the above equations, we can solve each control point . Further, the corresponding fitting curve for

[0052]

[0053] in, It is a B-spline basis function, and the cubic B-spline interpolation curve is used for fitting, so there is a B-spline basis function

[0054]

[0055] After cubic B-spline interpolation fitting, a smooth curve based on the observed value fitting can be obtained. Through this curve, the observed value of the corresponding sensor at any time can be obtained to achieve data interpolation. Then align with other sensors, select the required sampling time, and extract the observed value at the corresponding time from the fitting curve of each sensor to achieve time alignment. The data obtained after interpolation and patching are shown in Table 2.

[0056] Table 2 Processed sensor data display table

[0057]

[0058] Step 2: Use the Kalman filter method to fuse the state data at different times to improve data accuracy.

[0059] First, a Kalman filter equation based on feedback control iteration is established to estimate the state at a certain moment in the process, and then obtain feedback in the form of measurement values. The Kalman filter can be divided into two parts: the time update equation and the measurement update equation.

[0060] The time update equation is responsible for forward calculation of the current state variables and error covariance estimation The value of , which provides the basis for constructing a priori estimates for the next time step, is expressed as follows:

[0061]

[0062]

[0063] The measurement update equation is responsible for feedback, that is, the prior estimate With the new measurement Combined to construct an improved posterior estimate for:

[0064]

[0065]

[0066]

[0067] The time update equation can be regarded as an estimation equation, and the measurement update equation can be regarded as a correction equation. The combination of the two forms an estimation-correction algorithm with a numerical solution, which can obtain the current time estimate and the uncertainty of this state estimate .

[0068] Then, the three sets of processed sensor data are filtered using the same filter, and the initial filter parameters used are:

[0069]

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] After Kalman filtering, the sensor data are shown in the following table

[0076] Table 3 Sensor data display after filtering

[0077]

[0078] Step 3: Based on the reinforcement learning method, design the state space, action space and reward function required for multi-sensor fusion to train the fusion model and obtain the optimal fusion weight.

[0079] Define a finite set of states S, a finite set of actions A, and a reward function R(s, a): At each time step t, the data fusion system based on reinforcement learning receives the state in the state set S , and select an action from action set A , to fuse the multi-sensor observations. Once the operation is performed , the data fusion system calculates a scalar reward according to the reward function R , at which point the system transitions to the next state The data fusion system repeatedly learns through this process until the termination condition is reached and the optimal fusion value is output. The Q function of the multi-sensor learning fusion method based on weight adjustment is defined as

[0080]

[0081] in, is the discount factor; is the horizontal range, i.e. the number of iterations. And the Q value table is used to store the total expected reward of taking a certain action in all states.

[0082] To obtain the optimal fusion value, a corresponding fusion weight is set for each radar. Based on the weight adjustment, the data of the three radars are fused by learning, that is, the weights w1, w2, and w3 of each radar are determined. w1 and w2 can be adjusted in three ways: increasing by 0.01, decreasing by 0.01, and remaining unchanged. Since the sum of the weights is 1, w3 is calculated from w1 and w2, that is, w3=1-w1-w2. So there are a total of Possible actions.

[0083] The size range of w1 and w2 is (0.0, 1.0), so the state space size is , so the Q value table model parameter size is .

[0084] For a multi-sensor detection system, assuming that the number of sensors is ns, in order to obtain the optimal fusion value, each sensor corresponds to a corresponding weight, and the fusion output Fo can be calculated by the following formula:

[0085]

[0086] in, is the observation value of ns sensors, is the corresponding weight coefficient of each sensor, and .

[0087] For each data fusion result, it is required to find a set of optimal weights to maximize the cumulative reward of Q-learning. The following defines the state, action and reward of the multi-sensor learning fusion method based on weight adjustment. Each change in weight is used as the set of multi-sensor learning fusion actions based on weight adjustment. Secondly, the definition of the multi-sensor learning fusion state based on weight adjustment refers to the calculation formula of the fusion value, and defines the state under time step t. This is the fusion value at this moment

[0088]

[0089] in, Status The corresponding weight coefficients of each sensor are shown below.

[0090] The error function is used as the reward function of multi-sensor learning fusion based on weight adjustment. When the fusion error is small, a larger reward is obtained, and vice versa. Take action Post-arrival state The reward can be defined as

[0091]

[0092] in, and Status and The fusion value of is the true value obtained based on prior knowledge. < 0, means the status Better than , on the contrary, the state better.

[0093] The training iteration results of the weight selection simulation of the reinforcement learning method are shown in the following table:

[0094] Table 4 Reinforcement learning training weight iteration change table

[0095]

[0096] Where t is the number of training iterations. From the sensor observation data in Table 1, we can see that Radar 1's measurement performance is significantly better than Radar 2 and Radar 3. Therefore, Radar 3's weight should account for a larger percentage in the data fusion results, which is consistent with the results shown in Table 4.

[0097] The Bayesian method and the reinforcement learning method, the data comparison curve after fusion is as follows Figure 3Simulation results show that the weights of multiple sensors affect the accuracy of the data fusion system. By adjusting the weights of multiple sensors, an effective fusion value can be obtained. Compared with the Bayesian algorithm, the reinforcement learning algorithm adjusts the fusion strategy in real time based on feedback, resulting in better fusion results and smaller errors.

[0098] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless defined as such, will not be interpreted in an idealized or overly formal sense.

[0099] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data fusion vehicle positioning method based on Kalman filtering and reinforcement learning, characterized in that: The following steps are involved: Step 1: Preprocess the raw sensor data and use a cubic B-spline interpolation curve fitting method to perform data interpolation and time alignment; Step 2: The pre-processed data is fused using the Kalman filter method. Specifically, the state data at different times are fused through the state estimation method of feedback control to improve the accuracy of the original data. Step 3: Based on the reinforcement learning method, the state space, action space and reward function required for multi-sensor fusion are designed to train the fusion model and obtain the optimal fusion weight; Step 4: Load the trained fusion model and optimal fusion weights to fuse the multi-sensor data, load real-time data to optimize the fusion model, adjust the fusion weights, and further improve the accuracy of the fusion data.

2. The multi-sensor data fusion vehicle positioning method based on Kalman filtering and reinforcement learning according to claim 1 is characterized in that: In step 1, the cubic B-spline interpolation curve is used to fit the sensor sampling data. The designed B-spline basis function is shown as follows: In the formula, is the sampling time, That is The interpolated curve obtained within the time period.

3. The vehicle positioning method based on data fusion of Kalman filtering and reinforcement learning according to claim 1 is characterized in that: In step 2, the Kalman filter equation based on feedback control iteration is established to obtain the current moment estimate and the uncertainty of this state estimate ; ； ； ； ； ； is the prior state prediction at time k, yes The optimal estimate result at time, is the covariance of the prior state prediction error, express The uncertainty of the optimal estimate at the moment, Kk is the Kalman gain.

4. The vehicle positioning method based on data fusion of Kalman filtering and reinforcement learning according to claim 1 is characterized in that: The Q function of the multi-sensor learning fusion method based on weight adjustment in step 3 is defined as ； Among them, γ∈[0, 1] is the discount factor; is the horizontal range, i.e. the number of iterations; Secondly, define each change in weight as a set of multi-sensor learning fusion actions based on weight adjustment; and define the multi-sensor learning fusion state based on weight adjustment at time step t This is the fusion value at this moment ; ； in, Status The corresponding weight coefficients of each sensor below; Finally, the error function is used as the reward function of multi-sensor learning fusion based on weight adjustment; Take action Post-arrival state The compensation is defined as ； in, and Status and The fusion value of It is the true value obtained based on prior knowledge.

Images