A roadside perception fusion system and optimization method based on vehicle-road collaboration

Abstract

An optimization method based on a roadside perception fusion system, which uses vehicle-side positioning (vehicle-side high-precision GNSS, inertial navigation or high-precision map combined positioning) as the positioning truth value to be sent to the roadside unit based on a vehicle-road collaborative mode, and determines the accuracy of the perception of the roadside perception device by comparing the vehicle positioning value obtained by the roadside perception device with the positioning truth value fed back by the vehicle-side based on the vehicle-road collaborative mode. When the difference between the two is within a certain range, the factor graph positioning method is used to correct the positioning of all targets perceived by the roadside perception device based on the vehicle-side positioning data, thereby improving the accuracy of the road target data perceived by the roadside perception device, and the correction information is also sent to the cloud platform through the roadside edge computing unit; when the difference exceeds the set error allowable range, the data abnormality warning is reported to the platform to realize the monitoring of the roadside perception data. Under the condition of vehicle-road collaboration, the present invention uses the factor graph method to correct the data error obtained by the roadside perception device to ensure the accuracy of the data obtained by the roadside perception device.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-road collaboration and roadside perception fusion, and in particular to a roadside perception fusion system and optimization method based on vehicle-road collaboration. Background Art

[0002] Vehicle-road collaborative technology refers to the use of advanced wireless communications and new-generation Internet technology to implement all-round dynamic real-time information interaction between vehicles and roads, and to carry out vehicle active safety control and road collaborative management based on the collection and integration of dynamic traffic information in all time and space, so as to fully realize the effective coordination between people, vehicles and roads, ensure traffic safety, improve traffic efficiency, and thus form a safe, efficient and environmentally friendly road traffic system.

[0003] At present, the positioning accuracy requirements for autonomous vehicles are getting higher and higher. With the large-scale installation of high-precision roadside equipment, its perception accuracy is directly related to the perception of blind spot obstacles by intelligent connected vehicles. However, in actual situations, due to inaccurate early calibration, loose installation, wind and other factors, the angle of the perception equipment is slightly deviated, which has a great impact on the accuracy of the perception data, resulting in a certain deviation in the sensor's perception of the target position on the road.

[0004] Taking into account that most intelligent connected vehicles have high-precision positioning capabilities, in order to solve the above problems, the present invention proposes a roadside perception fusion optimization method based on vehicle-road collaboration to improve and monitor the accuracy of the external broadcast data of roadside perception equipment.

[0005] Some situations of existing patents:

[0006] Patent: Vehicle positioning method and device based on vehicle-road collaboration and high-precision positioning fusion (patent application number: 202011548379.1), proposes to use vehicle-road collaboration technology, taking into account the shortcomings of high-precision satellite navigation and inertial navigation, and adding specific anchor points within a certain distance in certain scenes where the roads are complex or not conducive to satellite positioning, so as to achieve high-precision positioning of vehicles in all road conditions. This method is only applicable to sections with anchor point marks set, and cannot be used to achieve high-precision positioning of vehicles on sections without anchor point marks.

[0007] Patent: Positioning method and device, vehicle, auxiliary positioning system (patent application number: 201911241849.7), proposes to transmit position information within the transmission range of the position transmitter, obtain the current position of the vehicle according to the current geographical location reserved by the position information and the driving information of the vehicle, and accurately locate the vehicle. This method can only perform accurate positioning when the position transmitter is continuously deployed and the vehicle is guaranteed to be within the deployment range of the transmitter during driving.

[0008] Patent: High-precision positioning and path planning method for autonomous driving (patent application number: 201911027832.1), proposes to deploy RSU (the RSU shell should be painted with a rectangular bright color logo and serve as a target for optical positioning) in different road scenarios (urban roads and highways), use PCM optical ranging, and use elevation and azimuth measurements to achieve precise positioning of the vehicle.

[0009] It can be seen that the existing technologies have not solved the problem of changes in perception data accuracy errors caused by calibration and long-term slight displacement of sensors on the roadside perception equipment of the entire road section, thereby causing deviations in the perception data of the surrounding road environment provided to the connected vehicles. Summary of the invention

[0010] According to the problems raised by the background technology, in order to solve the technical difficulty that the roadside perception equipment is caused by the calibration and long-term slight displacement of the sensor, thereby causing the deviation of the surrounding road environment perception data provided to the connected vehicles, the present invention proposes a roadside perception fusion optimization method based on vehicle-road collaboration.

[0011] Since the vehicle side generally has high-precision GNSS, inertial navigation or high-precision map combined positioning capabilities, its positioning accuracy is relatively high, so the vehicle side positioning can be used as the true value to be sent to the roadside unit based on the vehicle-road collaboration mode, and the accuracy of the roadside perception device's perception is determined by comparing the vehicle positioning value obtained by the roadside perception device with the positioning true value fed back by the vehicle side based on the vehicle-road collaboration mode. When the difference between the two is within a certain range, the factor graph positioning method is used to correct the positioning of all targets perceived by the roadside perception device based on the vehicle side positioning data, improve the accuracy of the road target data perceived by the roadside perception device, and also send the correction information to the cloud platform through the roadside edge computing unit (MEC); when the difference exceeds the set error allowable range, the data abnormality warning is reported to the platform side to realize the monitoring of the roadside perception data.

[0012] A roadside perception fusion system based on vehicle-road collaboration, comprising:

[0013] The vehicle-mounted unit broadcasts vehicle information to the roadside unit through the vehicle-road cooperative PC5 direct communication method;

[0014] Roadside units are arranged at certain intervals on the roadside and are used to receive vehicle information broadcast by the vehicle-mounted units and broadcast road and vehicle information externally;

[0015] Roadside sensing equipment obtains road and vehicle information and sends the information to roadside edge computing through a wired network;

[0016] Roadside edge computing, deployed on the roadside, is used to receive and process information sent by roadside units and roadside sensing devices through wired networks;

[0017] The cloud control platform is used to receive road and vehicle structured data sent by roadside edge computing and monitor and warn of abnormal data.

[0018] An optimization method based on a roadside perception fusion system comprises the following steps:

[0019] Step 1: When the connected vehicle is driving on the road, the vehicle-mounted unit broadcasts vehicle information to the outside through the vehicle-road cooperative PC5 communication method;

[0020] Step 2: When entering the communication range of the roadside unit, the roadside unit receives the vehicle information broadcast by the vehicle-mounted unit, obtains the vehicle real-time positioning information as the vehicle real-time positioning truth value, and sends the positioning truth value to the roadside edge calculation;

[0021] Step 3: Roadside sensing equipment (radar, camera, etc.) senses and obtains road and vehicle information, and sends the road and vehicle information to roadside edge computing;

[0022] Step 4: Roadside edge computing identifies, tracks, and fuses the acquired road and vehicle information to obtain structured information, and uses the vehicle positioning data sensed by the roadside sensing device as the value to be tested;

[0023] Step 5: The roadside edge computing compares the true positioning value sent by the vehicle-mounted unit to the roadside unit with the vehicle positioning data to be verified value sensed by the roadside sensing device;

[0024] Step 6: If the error between the true positioning value and the roadside perception positioning value to be tested is less than the set allowable error value, it is considered that the perception ability of the roadside perception equipment is relatively accurate; if the error between the true positioning value and the positioning value to be estimated exceeds the set allowable error but is within the abnormal error, the factor graph method is used to correct the estimated positioning value; if the error between the true positioning value and the positioning value to be estimated reaches the abnormal error range, an early warning is sent to the cloud platform.

[0025] Beneficial effect: Compared with the prior art, the present invention uses a factor graph method to correct the data errors obtained by the roadside sensing device under the vehicle-road collaborative condition, thereby ensuring the accuracy of the data broadcast by the roadside device to the outside. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 : Optimize the specific scene diagram based on the fusion of roadside perception equipment of vehicle-road collaboration;

[0027] Figure 2 : Flowchart of roadside perception fusion optimization based on vehicle-road collaboration;

[0028] Figure 3 :Factor graph method correction flow chart;

[0029] Figure 4 : Relationship diagram between variable nodes and factor graph nodes. DETAILED DESCRIPTION

[0030] Next, a specific embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0031] Reference Figure 1 A roadside perception fusion system based on vehicle-road collaboration includes an on-board unit (OBU), a roadside unit (RSU), a roadside edge computing (MEC), roadside perception equipment (radar, camera, etc.) and a cloud control platform.

[0032] The on-board unit is installed inside the vehicle and broadcasts vehicle information (BSM messages, including latitude and longitude position, speed, heading angle, braking, vehicle type, etc.) to the roadside unit through the vehicle-road cooperative PC5 direct communication method;

[0033] The roadside units are arranged on the roadside at a certain distance (generally 300-500 meters for urban roads) to receive vehicle information broadcast by the vehicle-mounted units and to broadcast road and vehicle information externally;

[0034] The roadside sensing equipment is composed of radar, camera and other equipment to obtain road and vehicle information and send the information to the roadside edge computing through a wired network;

[0035] The roadside edge computing is deployed on the roadside to receive and process information sent by the roadside unit and the roadside sensing device through the wired network;

[0036] The cloud control platform is used to receive road and vehicle structured data sent by roadside edge computing, and monitor and warn of abnormal data.

[0037] First, the vehicle side has high-precision GNSS, inertial navigation or high-precision map combined positioning, and the vehicle positioning accuracy is high, generally with centimeter-level positioning capabilities. In the present invention, the vehicle's own equipment positioning data is used as the true value and broadcasted to the outside through the on-board unit (OBU) based on the communication method of the vehicle-road collaborative PC5.

[0038] When the connected vehicle is on the road and within the communication range of the roadside unit (generally between 300-500m for urban roads), the on-board unit broadcasts its true positioning value to the roadside unit. When the connected vehicle is also within the perception range of the roadside unit, the roadside unit sends the data sensed by the sensor to the roadside edge computing through the wired network, and generates target structured data after identification and tracking by the roadside edge computing. The vehicle attribute information such as license plate and body color obtained by the roadside unit is matched with the target structured information obtained within the perception range of the roadside edge computing to obtain the connected vehicle detection data corresponding to the structured data in the roadside edge computing. After time synchronization according to the timestamp, the error caused by the delay is eliminated (the delay of vehicle-road collaborative communication is about 20ms, and the estimated position error is 0.4m based on the maximum speed of 80km / h on urban roads), and the true positioning value of the connected vehicle is compared with the perceived vehicle positioning value to be detected. When the error between the true positioning value and the positioning value to be tested is less than the set allowable error (the allowable error includes delay error, reasonable calibration error, and target center point estimation error, etc.), the perception ability of the roadside perception device is considered to be relatively accurate. When the error between the true positioning value and the positioning value to be estimated exceeds the set allowable error value but is less than the maximum acceptable error, the factor graph method is used to correct the estimated positioning value to improve the perception accuracy of the roadside perception device.

[0039] Reference Figure 2 The optimization method based on the roadside perception fusion system provided by the present invention comprises the following steps:

[0040] Step 1: When a connected vehicle with high-precision positioning capability is driving on the road, the on-board unit broadcasts vehicle information (BSM messages, including latitude and longitude position, speed, heading angle, braking, vehicle type, etc.) to the outside through the vehicle-road collaborative PC5 communication method;

[0041] Step 2: When the connected vehicle enters the communication range of the roadside unit, the roadside unit receives the vehicle information broadcast by the vehicle-mounted unit, obtains the vehicle real-time positioning information as the vehicle real-time position truth value, and sends the positioning truth value to the roadside edge computing through the wired network;

[0042] Step 3: The roadside perception device obtains road and vehicle information through road sensors, such as video, LiDAR point cloud, millimeter wave perception targets, etc., and sends the sensor data representing the road and vehicle information to the roadside edge computing through the wired network;

[0043] Step 4: Roadside edge computing identifies, tracks, and fuses sensor data to obtain structured information including vehicle attribute characteristics, location, speed, and heading angle, and uses the vehicle positioning data sensed by the roadside sensing device as the value to be tested;

[0044] Step 5: The roadside edge computing compares the true value of the vehicle positioning sent by the vehicle-mounted unit to the roadside unit with the vehicle positioning value to be verified perceived by the sensor;

[0045] Step 6: If the error between the true positioning value and the roadside perception positioning value to be tested is less than the set allowable error value, it is considered that the perception ability of the roadside perception device is relatively accurate; if the error between the true positioning value and the positioning value to be estimated exceeds the set allowable error but is within the abnormal error, the factor graph method is used to correct the estimated positioning value to improve the perception accuracy of the roadside perception device; if the error between the true positioning value and the positioning value to be estimated is large and reaches the abnormal error range, an early warning is sent to the cloud platform.

[0046] The present invention is mainly applicable to situations where the angle of the sensing device is slightly deviated due to factors such as early calibration errors, loose installation, wind, etc., which leads to a certain deviation in the sensor's perception of the target position. This embodiment uses the factor graph method to correct the vehicle positioning value obtained by the roadside sensing device to improve the perception accuracy of the roadside sensing device. Figure 3 , the factor graph method in step 6 specifically includes the following steps:

[0047] S61, obtaining parameters required by variable nodes and factor nodes in the factor graph through roadside sensors.

[0048] S62, define variable nodes and factor nodes.

[0049] The factor graph includes two types of nodes, namely variable nodes and factor nodes. In this embodiment, the data sensed by the sensor is used as the variable node, and the probability relationship between the data value sensed by the sensor and the vehicle positioning value is used as the factor node.

[0050] The variable nodes include the vehicle's position x i (t i The vehicle position is always perceived, including the vehicle's position, speed, direction, etc.), t i to k The vehicle's posture parameters at this moment are as follows:

[0051] X k ={x i} i＝1，2,…k ; (1)

[0052] x i =[p T ,v T ,θ T ];(2)

[0053] Where: p = [x, y, z] is the position vector of the vehicle, v = [v x ,vy ,v z ] is the vehicle velocity vector, and θ is the vehicle heading angle.

[0054] z i Indicates t i The vehicle positioning truth data is received at all times, from time t1 to time t k All the vehicle positioning truth data received at the moment is recorded as Z k :

[0055] Z k ={z i} i＝1,2,…k ; (3)

[0056] S63, constructing a factor graph model based on variable nodes and factor nodes.

[0057] For a given vehicle positioning truth value Z, the most commonly used method for estimating the probability of the perceived unknown state variable X is the maximum a posteriori probability, which is the most commonly used estimation algorithm based on factor graph modeling. The algorithm is as follows:

[0058] X MAP =argmaxp(X|Z); (4)

[0059] Where p is the joint probability function.

[0060] For any factor graph, MAP inference can be reduced to maximizing the product of all factors, expressed as:

[0061] X MAP = argmax∏ i Φ i (X i ); (5)

[0062] where Φ i (X i ) is a factor and is proportional to the product of multiple groups of errors, and,

[0063] φ i ∝f i ; (6)

[0064]

[0065] where h i (X i ) is the perception function, which is a function related to the perception state variable; i is the actual positioning value of the vehicle.

[0066] Under the premise of the Gaussian noise model assumption, by mathematically transforming the formula, the final MAP inference becomes the form of minimizing nonlinear least squares:

[0067]

[0068] The nonlinear least squares problem is linearized and h i (X i ) and Taylor expansion is performed to obtain:

[0069]

[0070] Among them, H i is the observation Jacobian matrix, which is given by the linearization point About h i (.) is the multivariate partial differential, where

[0071]

[0072] Furthermore, formula (8) Translates to:

[0073]

[0074] in, is the difference between the actual value and the perceived value at the linearization point. Convert the Mahalanobis norm to the 2-norm and get:

[0075]

[0076] in:

[0077]

[0078]

[0079] S64, correcting the roadside sensor perception data value according to the factor graph model.

[0080] In fact, optimizing formula (12) is actually solving the least squares problem, that is, solving A i The inverse of A i is a reversible matrix, where each row represents a factor and each column represents the variable contained in the factor. Solve for A i The inverse of X can be solved MAP , that is, find f i , and then make error correction within the allowable error.

[0081] The specific relationship between variable nodes and factor nodes is shown in the attached Figure 4 As shown in Figure 1, since the data obtained by the roadside sensing device will accumulate errors over time and may be affected by external conditions, resulting in errors caused by its own drift, the data value obtained by the sensor is added through f 融合定位Correction is performed. Among them, f pior It is a unary factor and is only related to the first step state x1.

[0082] The present invention uses a factor graph method to correct the data error obtained by the roadside sensing device under the vehicle-road cooperative condition, so as to maintain the accuracy of the data obtained by the roadside sensing device.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A roadside perception fusion optimization method based on vehicle-road collaboration, Features The steps include: Step 1: When the connected vehicle is driving on the road, the vehicle-mounted unit broadcasts vehicle information to the outside through the vehicle-road cooperative PC5 communication method; Step 2: When entering the communication range of the roadside unit, the roadside unit receives the vehicle information broadcast by the vehicle-mounted unit, obtains the vehicle real-time positioning information as the vehicle real-time positioning truth value, and sends the positioning truth value to the roadside edge calculation; Step 3: The roadside sensing device senses and obtains road and vehicle information, and sends the information to the roadside edge computing; Step 4: Roadside edge computing identifies, tracks, and fuses the information sensed by the sensing device to obtain the structured information of the vehicle, and uses the vehicle positioning data sensed by the roadside sensing device as the value to be tested; Step 5: The roadside edge computing compares the true positioning value sent by the vehicle-mounted unit to the roadside unit with the vehicle positioning data to be verified value sensed by the roadside sensing device; Step 6: If the error between the true positioning value and the roadside perception positioning value to be tested is less than the set allowable error value, it is considered that the perception ability of the roadside perception device is relatively accurate; if the error between the true positioning value and the estimated positioning value exceeds the set allowable error but is within the maximum acceptable error, the estimated positioning value is corrected using the factor graph method; if the error between the true positioning value and the estimated positioning value reaches within the abnormal error range, an early warning is sent to the cloud platform; The factor graph method described in step 6 is as follows: The factor graph includes two types of nodes: variable nodes and factor nodes. The sensed data is used as the variable node, and the probability relationship between the sensed data value and the true value of the vehicle positioning is used as the factor node; The variable nodes include the vehicle's position x i , t i to k The vehicle's posture parameters at this moment are as follows: X k ={x i } i＝1，2,…k , (1) x i =[p T ,v T ,i T ], (2) Where p = [x, y, z] is the position vector of the vehicle, v = [v x ,v y ,v z ] is the vehicle velocity vector, θ is the vehicle heading angle; Define z i Indicates t i The vehicle positioning truth data is received at time t 1 From time to t k All the true positioning data received at the moment is Z k : WITH k ={z i } i＝1,2,…k ; (3) For a given vehicle positioning true value data Z, the probability estimation of the perceived unknown state variable X adopts the maximum a posteriori probability estimation algorithm, and the algorithm is as follows: X MAP =argmaxp(X|Z), (4) Where p is the joint probability function; For any factor graph, MAP inference can be reduced to maximizing the product of all factors: X MAP =argmax∏ i F i (X i ), (5) Among them, Φ i (X i ) is a factor and is proportional to the product of multiple groups of errors, and, f i ∝f i , (6) Among them, h i (X i ) is the perception function, which is a function related to the perception state variable; i The actual positioning truth value of the vehicle; Under the premise of Gaussian noise model assumption, the formula is mathematically transformed to convert MAP inference into the form of minimizing nonlinear least squares: The nonlinear least squares problem is linearized and h i (X i ) and Taylor expansion is performed to obtain: Among them, H i is the observation Jacobian matrix, which is given by the linearization point About h i (.) is the multivariate partial differential, where Then, transform equation (8) into: in, In order to locate the difference between the true value and the perceived value at the linearization point, the Mahalanobis norm is converted into the 2-norm, and the result is: in: Formula (12) is actually solving the least squares problem, that is, solving A i The inverse of A i is a reversible matrix, where each row represents a factor and each column represents the variable contained in the factor. Solve for A i The inverse of X can be solved MAP , that is, find f i , make error correction for the error within the allowable range.

2. A roadside perception fusion system based on vehicle-road collaboration using the method of claim 1, Features include: The vehicle-mounted unit broadcasts vehicle information to the roadside unit through the vehicle-road cooperative PC5 direct communication method; Roadside units are arranged at certain intervals on the roadside and are used to receive vehicle information broadcast by the vehicle-mounted units and broadcast road and vehicle information externally; Roadside sensing equipment obtains road and vehicle information through sensing and sends the information to roadside edge computing through a wired network; Roadside edge computing, deployed on the roadside, is used to receive and process information sent by roadside units and roadside sensing devices through wired networks; The cloud control platform is used to receive road and vehicle structured data sent by roadside edge computing and monitor and warn of abnormal data.

Images