A fusion perception and positioning system based on vehicle-road collaboration

Abstract

The present invention relates to a fusion perception and positioning system based on vehicle-road collaboration, and belongs to the field of vehicle-road collaboration technology. The system includes an intelligent vehicle-mounted perception and positioning subsystem, a roadside perception and positioning subsystem, a V2X communication subsystem, and a roadside edge collaborative perception subsystem. The roadside edge collaborative perception subsystem uses a particle filtering algorithm and, after spatial coordinate conversion, fuses the positioning information of surrounding vehicles obtained by the vehicle-mounted perception and positioning subsystem and the positioning information of road environment vehicles obtained by the roadside perception and positioning subsystem, and performs position compensation on the fusion set target according to the designed collaborative perception position and time offset compensation method. Based on advanced V2X communication technology, the present invention utilizes roadside perception information and vehicle-side perception information, and through the perception fusion technology of the edge collaborative perception subsystem in the MEC server, improves the positioning accuracy and perception field of view of road vehicles, and makes up for the problem of roadside perception failure.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle-road collaboration and relates to a fusion perception and positioning system based on vehicle-road collaboration. Background Art

[0002] In recent years, autonomous vehicle technology has rapidly developed, with vehicles of varying levels of autonomy already on the road. This means that for a long time to come, intelligent and non-intelligent vehicles will coexist on the road. Autonomous vehicles also require increasingly precise positioning and perception. Relying solely on onboard sensors for positioning and perception still faces limitations, such as blind spots. To address these challenges, vehicle-infrastructure collaboration (V2I) has emerged. V2I utilizes advanced wireless communications and next-generation internet technologies to enable comprehensive, real-time information exchange between vehicles and the road, effectively achieving effective vehicle-road collaboration and ensuring traffic safety. However, roadside perception and positioning can also experience discrepancies in the accuracy of roadside perception equipment due to factors such as initial calibration and installation errors, leading to perception failures. Therefore, it is necessary to integrate autonomous driving and roadside perception information for complementary fusion.

[0003] Taking into account that most autonomous driving vehicles have high-precision positioning capabilities, combined with vehicle networking technology, and making full use of autonomous driving perception and positioning of surrounding environment information, the present invention proposes a road vehicle fusion perception and positioning system based on vehicle-road collaboration to improve the accuracy of perception data broadcast externally by roadside perception equipment. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a fusion perception and positioning system based on vehicle-road collaboration, based on advanced vehicle networking technologies such as LTE-V2X and 5G, making full use of vehicle-side perception and positioning information, combined with roadside MAP messages, to improve the vehicle's positioning accuracy, and through complementary fusion with roadside perception and positioning target information, solve the problem of incomplete perception road target data information broadcast by the roadside.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A fusion perception and positioning system based on vehicle-road collaboration, including:

[0007] The intelligent vehicle-mounted perception and positioning subsystem, including a vehicle-side perception module and a vehicle-side data fusion module, uses the intelligent vehicle-side sensors to obtain perception and positioning information of the vehicle itself and surrounding non-intelligent vehicles. It uses lane matching positioning methods to improve the positioning accuracy of the vehicle-mounted perception terminal, and then transmits the information to the roadside edge collaborative perception subsystem through the V2X communication subsystem.

[0008] The roadside perception and positioning subsystem, which includes a roadside perception module and a roadside data fusion module, is used to obtain perception and positioning information of vehicles on the roadside end, and compensate for positioning information of failed targets during the fusion process through positioning estimation methods. The road environment and vehicle information are then transmitted to the roadside edge collaborative perception subsystem;

[0009] A roadside edge collaborative perception subsystem that complementarily fuses vehicle perception and positioning information calculated by the intelligent on-board perception and positioning subsystem and the roadside perception and positioning subsystem based on the vehicle's operating status, including a fusion method and a collaborative perception position and time offset compensation method. The fused positioning information is then transmitted to the roadside vehicle via the V2X communication subsystem in a standard application layer data format.

[0010] The V2X communication subsystem includes an intelligent vehicle-mounted communication module and a roadside communication module. The V2X communication subsystem is used for communication between the roadside edge perception and positioning subsystem and the intelligent vehicle-mounted perception and positioning subsystem, including vehicle network wireless communication technology communication and Ethernet communication.

[0011] Furthermore, the vehicle-side perception module includes a vehicle-mounted OBU, a vehicle-side high-definition camera, and a vehicle-side laser radar. The vehicle-mounted OBU is used to obtain the vehicle's own position information, speed and acceleration information, and lane information and transmit them to the vehicle-side data fusion module. The vehicle-side high-definition camera is used to obtain images of the target vehicle's surrounding environment and transmit them to the vehicle-side data fusion module. The vehicle-side laser radar is used to obtain feature information of vehicles around the target vehicle and transmit it to the vehicle-side data fusion module (including information such as azimuth and relative distance information).

[0012] Furthermore, the vehicle-side data fusion module improves positioning accuracy and obtains high-precision perception and positioning information of surrounding vehicles based on the designed lane matching positioning method; it uses the pre-trained deep learning lane line detection model through the image information of the target vehicle's surrounding environment to obtain the lane positioning information of its own vehicle (including lane number, lane confidence, etc.), and obtains high-precision positioning information of the intelligent vehicle based on the designed lane matching positioning method; and then obtains high-precision perception and positioning information of surrounding adjacent vehicles after relative position coordinate conversion calculation based on the point cloud feature information of surrounding adjacent vehicles (including vehicle azimuth and relative distance information, etc.);

[0013] Furthermore, the vehicle-side data fusion module uses a comprehensive weighted lane matching positioning method to improve positioning accuracy and obtain the perception positioning information of surrounding vehicles. The specific method steps are as follows:

[0014] S101 obtains the vehicle's own GPS location information and azimuth, and MAP map message transmitted by the roadside RSU through the vehicle OBU;

[0015] S102. Combined with the MAP map message, the GPS location information obtained by the vehicle is projected onto the road section. The vertical distance of the projection is recorded as d, and the angle between the vehicle GPS track and the road section is recorded as θ.

[0016] S103. Calculate the weight of each candidate lane based on the two factors d and θ, and record them as W d 、W θ ;

[0017] S104. Based on the image acquired by the on-board camera and the pre-trained deep learning lane detection model, the confidence level of each lane is obtained, denoted as W μ ;

[0018] S105. Set a comprehensive weight value W, which is the sum of the weight values ​​of the three aspects, and finally select the lane with the largest comprehensive weight value as the final positioning lane of the target vehicle;

[0019] S106. Based on the vehicle point cloud feature information obtained by the laser radar, the relative position coordinate information of the surrounding vehicles is obtained, and then the positioning information of the surrounding vehicles is obtained through coordinate conversion calculation based on the GPS coordinates and lane information of the target vehicle itself.

[0020] Furthermore, the roadside perception module includes a roadside high-definition camera and a roadside millimeter-wave radar. The roadside perception module and the roadside communication module are installed at the intersection. The roadside high-definition camera is used to obtain road environment image data and transmit it to the roadside data fusion module. The roadside millimeter-wave radar is used to detect road environment target data and transmit it to the roadside data fusion module.

[0021] Furthermore, the roadside data fusion module compensates for the positioning information of the failed targets in the fusion process according to the designed positioning estimation method and the fusion matching algorithm, thereby improving the roadside perception accuracy; based on the road environment image data, a pre-trained target detection model is used to obtain road vehicle perception information, and then the road target information detected by the millimeter wave radar and the vehicle target information detected by the camera are used to perform target-level data fusion and matching to obtain the fused road vehicle perception positioning information, record the failed targets of the fusion matching, and mainly use visual detection to locate the failed targets using the visual grid positioning estimation method to obtain the final road vehicle perception positioning information.

[0022] Furthermore, the specific steps of the roadside data fusion module using the visual grid positioning estimation method and the fusion matching method to improve the roadside perception accuracy and thus obtain the final road vehicle perception positioning information are as follows:

[0023] S201. Based on the pre-trained lightweight target detection model, road vehicle perception information is obtained according to the road environment image data, including vehicle target category, vehicle target bounding box related information, and confidence level;

[0024] S202 obtains the road target information of each frame detected by the millimeter-wave radar after filtering, calculates the target GPS position by the distance and azimuth information between the target point and the millimeter-wave radar and the GPS coordinates of the millimeter-wave radar, and maps the millimeter-wave radar target data frame to the image space through coordinate transformation;

[0025] S203. Using the Hungarian matching algorithm, the road vehicle perception information obtained based on the image data and the road target data obtained based on the millimeter-wave radar are fused and matched to obtain the fused road vehicle perception positioning information, and the vehicles that were not successfully matched and missed are recorded;

[0026] S204. Divide the roadside camera sensing area into grid areas and obtain high-precision GPS coordinate positions corresponding to the grid areas through RTK equipment calibration;

[0027] S205. Select the center point of the lower edge of the bounding box of the missed target that was not successfully matched, match the missed target with the grid information, obtain the high-precision GPS coordinates of the missed target, and merge it with the fused road vehicle perception and positioning information to obtain the final road vehicle perception and positioning information.

[0028] Furthermore, the roadside edge collaborative perception subsystem is based on a particle filtering algorithm to fuse the vehicle perception and positioning information calculated by the intelligent vehicle-mounted perception and positioning subsystem and the roadside perception and positioning subsystem. After the fusion, the fusion result is compensated and positioned based on the designed collaborative perception position and time offset compensation method based on the vehicle operation state estimation, thereby compensating for the timeliness problem existing in the fused perception data, making the fused information unique and with the highest confidence, thereby enabling the smart car to obtain a broader perception perspective.

[0029] The specific steps for integrating the vehicle perception and positioning information obtained by the onboard perception and positioning system and the roadside perception and positioning system are as follows:

[0030] S301. The vehicle perception and positioning information obtained by the intelligent vehicle perception and positioning subsystem and the roadside perception and positioning subsystem is converted to a unified coordinate system with the WGS84 coordinate system's true north direction as the positive direction. The vehicle's target latitude and longitude are projected onto the coordinate system. The perception information of the vehicle perception and positioning system and the roadside perception and positioning system are then synchronized using a time synchronization model.

[0031] S302. In the initialization step, the vehicle initial positioning information obtained by the roadside perception and positioning subsystem is input as prior information. The initialization mathematical relationship is:

[0032]

[0033] in, and They represent the detection values ​​of the roadside sensing and positioning equipment at time t, ε is the maximum value of the target positioning error range, rand(·) is the random function of the generated interval, and x i and y i is the horizontal and vertical coordinates of the generated i-th target particle, N is the number of random particle samples, ω i is the initial weight of each target particle;

[0034] S303. During the prediction process, the target vehicle's control input (including speed, heading angle, etc.) is added to all target particles. The next position of each target particle is predicted according to the equation of motion. The mathematical relationship of the prediction model is as follows:

[0035]

[0036] Among them, x t-1 and y t-1 is the horizontal and vertical coordinates of the target particle at the previous moment, x t and y t is the horizontal and vertical coordinates of the current state prediction, ΔT is the time difference, v t is the speed of the vehicle at time t, γ t-1 is the heading angle of the vehicle at the previous moment, θ t is the heading angle increment at the current moment, and is the system environmental noise;

[0037] S304. During the update process, the vehicle-based target location information is used as the observation value. The weights corresponding to each target particle are updated based on the geometric distance between the roadside location information and the observation value. The optimal location information is obtained by weighted summing the target particle set. The weight update formula is as follows:

[0038]

[0039] in, is the set of horizontal and vertical coordinates of the i-th target particle in the roadside perception positioning target is the horizontal and vertical coordinate set of the vehicle-mounted perception target at time t+1 To find the distance between two sets;

[0040] S305. In the resampling stage, the resampling principle is introduced to screen the target particles. Resampling generates a new target particle set based on the original target particles according to the weight of the target particles and returns to step S302 for the next cycle. If the target state information suddenly changes during the process, the roadside perception and positioning target information is used as the input of the initial position.

[0041] Furthermore, the method process of collaborative perception position and time offset compensation positioning designed based on vehicle operation state estimation is as follows: calculate the difference between the timestamp of the vehicles in the set after vehicle perception fusion and the timestamp of the current moment, recorded as the time offset compensation Δt, and then calculate the position offset value of the target vehicle in the x and y directions in the unified coordinate system based on the movement speed and acceleration of the target in the x and y directions under the unified coordinate system. The offset value is used as the compensation value of the target in the perception fusion result, and then converted into longitude and latitude through the coordinates and broadcast to the road intelligent vehicle through the V2X communication subsystem.

[0042] Furthermore, the communication between the on-board communication module and the roadside communication module adopts vehicle network wireless communication technology, including two technical routes: LTE-V2X and DSRC.

[0043] The beneficial effects of the present invention are:

[0044] 1. This invention utilizes vehicle environment perception and positioning information from intelligent vehicle sensing devices in vehicle-road collaboration, combined with a pre-trained deep learning lane detection model and roadside broadcast MAP messages, to improve lane-level positioning of intelligent vehicles and the positioning of surrounding non-intelligent vehicles through a comprehensive weighting method.

[0045] 2. This invention addresses missed targets during the roadside sensing fusion process, including those detected by millimeter-wave radar and cameras. It primarily uses visual detection and a grid-based method to locate missed targets, thereby improving the information integrity of roadside sensing data.

[0046] 3. The present invention makes full use of vehicle-side perception and positioning target data and roadside perception and positioning information through vehicle-to-vehicle communication technology, fuses the perception data on both sides based on the particle filtering algorithm, and compensates for the position offset of the fused target to make up for the error problem caused by the time calculation in the fusion calculation process. In a mixed traffic environment, it effectively improves the accuracy of the target data broadcast by the roadside broadcast unit, ensuring that roadside intelligent vehicles can obtain perception information with a higher perspective and a wider range.

[0047] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0049] Figure 1 It is a structural block diagram of the system of the present invention;

[0050] Figure 2 It is the relationship diagram between vehicle GPS position coordinates and road section nodes;

[0051] Figure 3 This is an overview diagram of the data processing flow of a fusion perception and positioning system based on vehicle-road collaboration in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0053] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0054] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0055] See also Figure 1 and Figure 3 As shown in FIG, the architecture of the fusion perception and positioning system based on vehicle-road collaboration designed by the present invention includes: an intelligent vehicle perception and positioning subsystem, a roadside perception and positioning subsystem, a roadside edge collaborative perception subsystem, and a V2X communication subsystem, wherein:

[0056] The intelligent vehicle-mounted perception and positioning subsystem, including a vehicle-side perception module and a vehicle-side data fusion module, uses the intelligent vehicle-side sensors to obtain perception and positioning information of the vehicle itself and surrounding non-intelligent vehicles. It uses lane matching positioning methods to improve the positioning accuracy of the vehicle-mounted perception terminal, and then transmits the information to the roadside edge collaborative perception subsystem through the V2X communication subsystem.

[0057] The roadside perception and positioning subsystem, which includes a roadside perception module and a roadside data fusion module, is used to obtain perception and positioning information of vehicles on the roadside end, and compensate for positioning information of failed targets during the fusion process through positioning estimation methods. The road environment and vehicle information are then transmitted to the roadside edge collaborative perception subsystem;

[0058] A roadside edge collaborative perception subsystem that complementarily fuses vehicle perception and positioning information calculated by the intelligent on-board perception and positioning subsystem and the roadside perception and positioning subsystem based on the vehicle's operating status, including a fusion method and a collaborative perception position and time offset compensation method. The fused positioning information is then transmitted to the roadside vehicle via the V2X communication subsystem in a standard application layer data format.

[0059] The V2X communication subsystem includes an intelligent vehicle-mounted communication module and a roadside communication module. The V2X communication subsystem is used for communication between the roadside edge perception and positioning subsystem and the intelligent vehicle-mounted perception and positioning subsystem, including vehicle network wireless communication technology communication and Ethernet communication.

[0060] In this embodiment, first, the intelligent vehicle needs to be equipped with a vehicle-side perception module, which includes an on-board OBU, a vehicle-side high-definition camera, and a vehicle-side laser radar. The on-board OBU is used to obtain the vehicle's own GPS position information, speed and acceleration information, and lane information and transmit them to the vehicle-side data fusion module. The vehicle-side high-definition camera is used to obtain the target vehicle's surrounding environment image and transmit it to the vehicle-side data fusion module. The vehicle-side laser radar is used to obtain the feature information of vehicles around the target vehicle and transmit it to the vehicle-side data fusion module (including azimuth and relative distance information, etc.); the vehicle-side data fusion module obtains the lane information of its own vehicle (including lane number, lane confidence, etc.) based on the target vehicle's surrounding environment image information and a pre-trained deep learning lane line detection model, and fuses and matches it with the lane information transmitted by the on-board OBU; then, the point cloud feature information of the vehicles around the target vehicle detected by the laser radar is converted and calculated through relative position coordinates to obtain the perception positioning information of the surrounding vehicles.

[0061] In this embodiment, the vehicle-side data fusion module uses a comprehensive weighted lane matching positioning method to improve positioning accuracy and obtain the perception positioning information of surrounding vehicles. The specific steps are as follows:

[0062] S101 obtains the vehicle's own GPS location information and azimuth, and MAP map message transmitted by the roadside RSU through the vehicle OBU;

[0063] S102. See Figure 2 As shown, the GPS position information obtained by the vehicle is projected onto the road segment in combination with the MAP map message broadcast by the RUS. The vertical distance of the projection is recorded as d. The smaller d is, the closer the GPS coordinate position is to the lane. The angle between the vehicle GPS trajectory and the road segment is recorded as θ. The smaller θ is, the higher the matching degree between the vehicle trajectory and the lane is. The calculation expression is as follows:

[0064]

[0065]

[0066] Among them, (x i ,y i ) is the coordinate of each lane node, coordinate (x p1 ,y p1 ) is the GPS position coordinate of the vehicle, θ r The azimuth of the road segment in the MAP map message broadcast by the RSU;

[0067] S103. Calculate the weight of each candidate lane based on the two factors d and θ, and record them as W d 、W θ , W θThe average of the angles formed by the lines connecting multiple sets of adjacent GPS coordinate points and the lanes is calculated using the following expression:

[0068]

[0069]

[0070] Among them, A d is the weight factor, which takes an indefinite value according to the road conditions. θ =k·A d (0<k<1), when the parameters k and A d After confirmation, A θ is also fixed, then θ i The smaller it is, the greater the weight of the angle formed by the vehicle trajectory GPS position point and the lane;

[0071] According to the vertical distance between the vehicle position point and the road section, the parameter k is adjusted to change the size of each weight value. Usually, when the distance d is small, the k value takes a relatively small value.

[0072] S104. Based on the image acquired by the on-board camera and the pre-trained deep learning lane detection model, the confidence level of each lane is obtained, denoted as W μ ;

[0073] S105. Set a comprehensive weight value W, which is the sum of the weight values ​​of the three aspects. Finally, select the lane with the largest comprehensive weight value as the final positioning lane of the target vehicle, where W = W d +W θ +W μ ,Through the comprehensive weight fusion of the three, the positioning accuracy of the vehicle can be effectively improved,,achieving lane-level positioning;

[0074] S106. Based on the vehicle point cloud feature information obtained by the laser radar, the relative position coordinate information of the surrounding vehicles is obtained, and then the positioning information of the surrounding vehicles is obtained through coordinate conversion calculation based on the GPS coordinates and lane information of the target vehicle itself.

[0075] In this embodiment, the roadside perception module includes a roadside high-definition camera and a roadside millimeter-wave radar. The roadside perception module and the roadside communication module are installed on the elevated road at the intersection. The roadside high-definition camera is used to acquire road environment image data and transmit it to the roadside data fusion module, while the roadside millimeter-wave radar is used to detect road environment target data and transmit it to the roadside data fusion module. The roadside data fusion module uses a pre-trained target detection model based on the road environment image data to obtain road vehicle perception information. The target detection model must meet the requirements of real-time and lightweight roadside perception. The road target information detected by the millimeter-wave radar is then fused and matched with the vehicle target information detected by the camera. During the radar-visual fusion process, the radar coordinates and the camera coordinates need to be time- and spatially aligned to obtain the fused road vehicle perception positioning information. Targets that fail in fusion matching are recorded and located primarily through visual detection to obtain the final road vehicle perception positioning information.

[0076] In this embodiment, the roadside data fusion module uses the visual grid positioning estimation method and the fusion matching method to improve the roadside perception accuracy and obtain the final road vehicle perception positioning information. The specific steps are:

[0077] S201. Based on the pre-trained lightweight target detection model, the model is inferred based on the road environment image data to obtain road vehicle perception information, including vehicle target category, vehicle target bounding box related information, confidence level, etc.;

[0078] S202. Obtain each frame of road target information detected by the millimeter wave radar after filtering, and use the distance d and azimuth information β between the target point and the millimeter wave radar and the GPS coordinates of the millimeter wave radar (R x , R y ), calculate the detected target GPS position (p x , p x ), and through the spatial calibration coordinate transformation, the millimeter wave radar target data frame is mapped to the image space. The transformation relationship includes the rotation matrix and translation vector. The formula for calculating the target GPS position is as follows:

[0079] (p x , p x )=(R x +d-sinβ, R y +d*cosβ)

[0080] S203. In the image coordinate system, the Euclidean distance between the road vehicle information obtained from the image data and the road target data obtained from the millimeter-wave radar is used as a cost matrix. The targets perceived by the two sensors are fused and matched using the Hungarian matching algorithm to obtain fused road vehicle perception and positioning information. The ID information of any vehicles that were not successfully matched and were missed is recorded.

[0081] S204. Divide the roadside camera's sensing area into grid areas and obtain high-precision GPS coordinates corresponding to the grids using RTK equipment calibration. Each grid corresponds to its ID information, coordinate information in the graph, and GPS location information, as shown below:

[0082] c i ={(u i , v i );(lo i , la i )}

[0083] Among them, (u i , v i ) is the graphic coordinate information, (lo i , la i ) are the longitude and latitude coordinates corresponding to the road. The density of the grid division is related to the calculation error of the vehicle position point. The finer the grid division density, the higher the positioning accuracy.

[0084] S205. Select the center point of the lower edge of the bounding box of the missed target that was not successfully matched, match the missed target with the grid information, obtain the high-precision GPS coordinates of the missed target, and merge it with the fused road vehicle perception and positioning information to obtain the final road vehicle perception and positioning information.

[0085] In this embodiment, the roadside edge collaborative perception subsystem, based on a particle filtering algorithm, fuses vehicle perception and positioning information calculated by the intelligent vehicle-mounted perception and positioning subsystem and the roadside perception and positioning subsystem to compensate for roadside perception omissions and inaccurate positioning accuracy. The fused positioning information is then transmitted to roadside vehicles via the Internet of Vehicles wireless communication technology, effectively addressing issues such as limited roadside perception range and perception failures, enabling intelligent vehicles on the road to obtain a wider perception range.

[0086] In this embodiment, the specific steps of integrating the vehicle perception and positioning information obtained by the onboard perception and positioning system and the roadside perception and positioning system are as follows:

[0087] S301. The vehicle perception and positioning information obtained by the intelligent on-board perception and positioning subsystem and the roadside perception and positioning subsystem is converted to a unified coordinate system with the WGS84 coordinate system's true north direction as the positive direction. The vehicle's target latitude and longitude are projected onto the coordinate system. The perception information of the on-board perception and positioning system and the roadside perception and positioning system are then synchronized using a time synchronization model. A time period is set, and the time difference between the first target in the perception information set of the on-board perception and positioning subsystem and the first target in the perception information set of the roadside perception and positioning subsystem is compared with the current timestamp at regular intervals. If the time difference is less than a time threshold, fusion is performed. The set time period is determined based on the transmission frequency of the on-board and roadside perception data, and the time threshold is based on the perception frequency of the vehicle-side and road-side perception units.

[0088] S302. During the initialization process, the vehicle initial positioning information obtained by the roadside sensing and positioning subsystem is input as prior information. The initialization mathematical relationship is:

[0089]

[0090] in, and They represent the detection values ​​of the roadside sensing and positioning equipment at time t, ε is the maximum value of the target positioning error range, rand(·) is a random function, and x i and y i is the horizontal and vertical coordinates of the generated i-th target particle, N is the number of random particle samples, ω i is the initial weight of each target particle;

[0091] S303. During the prediction process, the target vehicle's control input (including speed, heading angle, etc.) is added to all target particles. The next position of each target particle is predicted according to the equation of motion. The mathematical relationship of the prediction model is as follows:

[0092]

[0093] Among them, x t-1 and y t-1 is the horizontal and vertical coordinates of the target particle at the previous moment, x t and y t is the horizontal and vertical coordinates of the current state prediction, ΔT is the time difference, v t is the speed of the vehicle at time t, γ t-1 is the heading angle of the vehicle at the previous moment, θ t is the heading angle increment at the current moment, and is the system environmental noise;

[0094] S304. During the update process, the vehicle-based target location information is used as the observation value. The weights corresponding to each target particle are updated based on the geometric distance between the roadside location information and the observation value. The optimal location information is obtained by weighted summing the target particle set. The weight update formula is as follows:

[0095]

[0096] in, is the set of horizontal and vertical coordinates of the i-th target particle in the roadside perception positioning target is the horizontal and vertical coordinate set of the vehicle-mounted perception target at time t+1 Indicates finding the distance between two sets;

[0097] S305. In the resampling stage, the resampling principle is introduced to screen the target particles. Resampling generates a new target particle set based on the original target particles according to the weight of the target particles and returns to step S302 for the next cycle. If the target state information suddenly changes during the process, the roadside perception and positioning target information is used as the input of the initial position.

[0098] Since the time offset caused by the amount of calculation during the fusion process leads to position offset, after fusion, the collaborative perception position and time offset compensation positioning method designed based on vehicle operation state estimation is used to compensate for the position offset of the targets in the fusion result set. The difference between the timestamp of the vehicles in the set after vehicle perception fusion and the timestamp of the current moment is calculated, which is recorded as the time offset compensation Δt. Then, the position offset value of the target vehicle in the x and y directions in the unified coordinate system is calculated based on the movement speed and acceleration of the target in the x and y directions in the unified coordinate system. The offset value is used as the compensation value of the target in the perception fusion result, and then the coordinates are converted into longitude and latitude and broadcast to the road intelligent vehicles through the V2X communication subsystem.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A fusion perception and positioning system based on vehicle-road collaboration, characterized by: The system includes an intelligent vehicle-mounted perception and positioning subsystem, a roadside perception and positioning subsystem, a roadside edge collaborative perception subsystem, and a V2X communication subsystem, among which: The intelligent vehicle-mounted perception and positioning subsystem, including a vehicle-side perception module and a vehicle-side data fusion module, uses the intelligent vehicle-side sensors to obtain perception and positioning information of the vehicle itself and surrounding non-intelligent vehicles. It uses lane matching positioning methods to improve the positioning accuracy of the vehicle-mounted perception terminal, and then transmits the information to the roadside edge collaborative perception subsystem through the V2X communication subsystem. The roadside perception and positioning subsystem, which includes a roadside perception module and a roadside data fusion module, is used to obtain perception and positioning information of vehicles on the roadside end, and compensate for positioning information of failed targets during the fusion process through positioning estimation methods. The road environment and vehicle information are then transmitted to the roadside edge collaborative perception subsystem; A roadside edge collaborative perception subsystem that complementarily fuses vehicle perception and positioning information calculated by the intelligent on-board perception and positioning subsystem and the roadside perception and positioning subsystem based on the vehicle's operating status, including a fusion method and a collaborative perception position and time offset compensation method. The fused positioning information is then transmitted to the roadside vehicle via the V2X communication subsystem in a standard application layer data format. The V2X communication subsystem includes an intelligent vehicle communication module and a roadside communication module. The V2X communication subsystem is used for communication between the roadside edge perception and positioning subsystem and the intelligent vehicle perception and positioning subsystem, including vehicle-to-vehicle wireless communication technology and Ethernet communication; The vehicle-side data fusion module improves positioning accuracy and obtains high-precision perception and positioning information of surrounding vehicles based on the designed lane matching positioning method; it uses the pre-trained deep learning lane line detection model through the target vehicle's surrounding environment image information to obtain the lane positioning information of its own vehicle, including the lane number and lane confidence; and obtains high-precision positioning information of the intelligent vehicle based on the designed lane matching positioning method; then, through the point cloud feature information of the surrounding adjacent vehicles, including the vehicle azimuth and relative distance information; after relative position coordinate conversion calculation, high-precision perception and positioning information of the surrounding adjacent vehicles is obtained; wherein, the vehicle positioning information, surrounding environment image information, and point cloud feature information are obtained by the vehicle-side perception module using the on-board OBU, vehicle-side high-definition camera, and vehicle-side lidar; The specific method steps for the vehicle-side data fusion module to improve positioning accuracy and obtain high-precision perception positioning information of surrounding vehicles based on the designed lane matching positioning method are as follows: S101 obtains the vehicle's own GPS location information and azimuth, and MAP map message transmitted by the roadside RSU through the vehicle OBU; S102. Combined with the MAP map message, the GPS location information obtained by the vehicle is projected onto the road section. The vertical distance of the projection is recorded as d, and the angle between the vehicle GPS track and the road section is recorded as θ. S103. Calculate the weight of each candidate lane based on the two factors d and θ, and record them as W d 、W θ ; S104. Based on the image acquired by the on-board camera and the pre-trained deep learning lane detection model, the confidence level of each lane is obtained, denoted as W μ ; S105. Set a comprehensive weight value W, which is the sum of the weight values ​​of the three aspects, and finally select the lane with the largest comprehensive weight value as the final positioning lane of the target vehicle; S106. Based on the vehicle point cloud feature information acquired by the LiDAR, the relative position coordinates of surrounding vehicles are obtained. Then, based on the target vehicle's own lane-level GPS positioning coordinates, high-precision positioning information of surrounding vehicles is calculated through coordinate conversion. This information is then transmitted to the roadside edge cooperative perception subsystem via the V2X communication subsystem. The roadside data fusion module compensates for positioning information of failed targets during the fusion process based on a designed positioning estimation method combined with a fusion matching algorithm, thereby improving roadside perception accuracy. Based on road environment image data, a pre-trained target detection model is used to obtain road vehicle perception information. Then, target-level data fusion and matching are performed using the road target information detected by the millimeter-wave radar and the vehicle target information detected by the camera to obtain fused road vehicle perception positioning information. Failed targets are recorded and located using a visual grid positioning estimation method based on visual detection to obtain final road vehicle perception positioning information. The specific steps of the roadside data fusion module using the visual grid positioning estimation method and the fusion matching method to improve the roadside perception accuracy and obtain the final road vehicle perception positioning information are as follows: S201. Based on the pre-trained lightweight target detection model, road vehicle perception information is obtained according to the road environment image data, including vehicle target category, vehicle target bounding box related information, and confidence level; S202 obtains the road target information of each frame detected by the millimeter-wave radar after filtering, calculates the target GPS position by the distance and azimuth information between the target point and the millimeter-wave radar and the GPS position coordinates of the millimeter-wave radar, and maps the millimeter-wave radar target data frame to the image space through coordinate transformation; S203. Using the Hungarian matching algorithm, the road vehicle perception information obtained based on the image data and the road target data obtained based on the millimeter-wave radar are fused and matched to obtain the fused road vehicle perception positioning information, and the vehicles that were not successfully matched and missed are recorded; S204. Divide the roadside camera sensing area into grid areas and obtain high-precision GPS coordinate positions corresponding to the grid areas through RTK equipment calibration; S205. Select the center point of the lower edge of the bounding box of the missed target that was not successfully matched, match the missed target with the grid information, obtain the high-precision GPS coordinates of the missed target, and merge it with the fused road vehicle perception and positioning information to obtain the final road vehicle perception and positioning information; The roadside edge collaborative perception subsystem, based on a particle filtering algorithm, fuses vehicle perception and positioning information calculated by the intelligent vehicle perception and positioning subsystem and the roadside perception and positioning subsystem. After fusion, the fusion result is compensated for using the designed collaborative perception position and time offset compensation method based on vehicle operating state estimation. This compensates for the timeliness issues existing in the fused perception data, making the fused information unique and highly confident, thereby enabling the intelligent vehicle to obtain a broader perception perspective. The specific steps of obtaining vehicle perception and positioning information by the intelligent vehicle-mounted perception and positioning subsystem and the roadside perception and positioning subsystem are as follows: S301. Transform the vehicle perception and positioning information obtained by the intelligent vehicle perception and positioning subsystem and the roadside perception and positioning subsystem into a unified coordinate system with the WGS84 coordinate system's true north as the positive direction. Project the vehicle's target latitude and longitude onto the coordinate system. Then, synchronize the perception information of the vehicle perception and positioning subsystem with that of the roadside perception and positioning subsystem using a time synchronization model. S302. In the initialization step, the vehicle initial positioning information obtained by the roadside perception and positioning subsystem is input as prior information. The initialization mathematical relationship is: in, and They represent the detection values ​​of the roadside sensing and positioning equipment at time t, ε is the maximum value of the target positioning error range, rand(·) is the random function of the generated interval, and x i and y i is the horizontal and vertical coordinates of the generated i-th target particle, N is the number of random particle samples, ω i is the initial weight of each target particle; S303. During the prediction process, the target vehicle's control input, including speed and heading angle, is added to all target particles. The next position of each target particle is predicted according to the equation of motion. The mathematical relationship of the prediction model is as follows: Among them, x t-1 and y t-1 is the horizontal and vertical coordinates of the target particle at the previous moment, x t and y t is the horizontal and vertical coordinates of the current state prediction, ΔT is the time difference, v t is the speed of the vehicle at time t, γ t-1 is the heading angle of the vehicle at the previous moment, θ t is the heading angle increment at the current moment, and is the system environmental noise; S304. During the update process, the vehicle-based target location information is used as the observation value. The weights corresponding to each target particle are updated based on the geometric distance between the roadside location information and the observation value. The optimal location information is obtained by weighted summing the target particle set. The weight update formula is as follows: in, is the set of horizontal and vertical coordinates of the i-th target particle in the roadside perception positioning target is the horizontal and vertical coordinate set of the vehicle-mounted perception target at time t+1 To find the distance between two sets; In the resampling phase, the resampling principle is introduced to screen the target particles. Based on the weights of the target particles, a new set of target particles is generated from the original target particles. The process returns to step S302 for the next cycle. If a sudden change in the target state information is encountered during the process, the target information is used as the input for the initial position. The method for compensating the fusion result based on the designed collaborative perception position and time offset compensation based on vehicle running state estimation is as follows: the difference between the timestamp of the vehicles in the set after vehicle perception fusion and the timestamp of the current moment is calculated, recorded as the time offset compensation Δt, and then the position offset value of the target vehicle in the x and y directions in the unified coordinate system is calculated based on the movement speed and acceleration of the target in the x and y directions in the unified coordinate system. The offset value is used as the compensation value of the target in the perception fusion result, and then the coordinates are converted into longitude and latitude and broadcast to the road intelligent vehicles through the V2X communication subsystem.

2. The fusion perception and positioning system based on vehicle-road collaboration according to claim 1 is characterized by: The communication between the on-board communication module and the roadside communication module adopts vehicle network wireless communication technology, including two technical routes: LTE-V2X and DSRC.

Images