Automatic driving vehicle driving real-time road edge fusion method

Abstract

The invention discloses a real-time road edge fusion method for driving of an automatic driving vehicle. The method comprises the following steps: step 1, initializing a data source; the data source comprises a visual perception data source S, a millimeter wave radar data source L and a vehicle driving information data source Z; 2, updating the road edge line based on the visual perception data source S to obtain a visual road edge line yS (x); updating the road edge line based on the millimeter wave radar data source L to obtain a radar road edge line yL (x); the driving state and the conversion matrix RT are updated based on the vehicle driving information data source Z, and a turning road edge line yZ (x) under the vehicle turning state is obtained; 3, performing fusion processing on the visual road edge line yS (x), the radar road edge line yL (x) and the turning road edge line yZ (x) to obtain a fused road edge line yR (x) after fusion; and 4, taking the fused road edge line yR (x) after fusion as one input value of the road edge line at the next moment, and updating the road edge line y (x) at the next moment. The parking lot autonomous valet parking system and the parking lot autonomous valet parking method have the advantages of obtaining a more robust road edge through fusion of multi-source data, improving safety and reliability of automatic driving and the like.

Description

Technical Field

[0001] The present invention relates to a data fusion technology, and in particular to a real-time road edge fusion method for an autonomous driving vehicle. Background Art

[0002] The edge line of the road is used to indicate the edge of the motor vehicle lane or to divide the boundary between the motor vehicle and non-motor vehicle lanes. The edge of the road plays a vital role in the planning and control process of autonomous driving vehicles. The edge of the road can define the boundary information of the road to indicate the edge line of the roadway. By defining the road edge features, a reference for the safe driving range can be provided to the vehicle, and local planning and vehicle control optimization can be performed on the vehicle, thereby promoting the realization of vehicle autonomous driving. Therefore, road edge computing is a key technology in the field of autonomous driving. Road edge computing plays a vital role in the field of intelligent transportation. It is the core computing part of the roadside infrastructure in the vehicle-road-cloud integrated system. By pushing computing power to edge devices such as street lights, traffic lights, cameras, etc., edge computing realizes real-time processing and analysis of data, with the characteristics of low latency, high reliability and high throughput, and is particularly suitable for application scenarios with high real-time requirements. This technology not only reduces network latency and improves data processing efficiency, but also realizes data security protection. It can be applied to multiple fields such as intelligent transportation, smart cities, and the Internet of Things.

[0003] Road edge computing provides a solid data foundation for the decision-making and control of intelligent transportation systems by providing real-time and efficient data processing and analysis capabilities. At present, the generation source of road edges is mostly a single data source, which is limited in robustness and difficult to integrate and improve, and cannot provide a good reference guarantee for the safe driving area of ​​vehicles. (1) The calculation method based on laser radar and other perception road edge has the advantage of simple calculation, but the detection cost is high and it is easily affected by reflection intensity, etc., and cannot predict the real-time road edge well. (2) When the method based on vehicle driving information (vehicle speed and yaw angular velocity) is used to obtain vehicle trajectory and predict road edges, although it can well predict the current state of the vehicle, it is difficult to predict the changing road ahead, which poses a safety hazard. (3) The road edge calculation method based on visual perception information has a good prediction ability for the road ahead, but is limited by the stability of perception (such as wear and tear of edge lines, etc.), and its robustness is limited. In summary, most of these road edge calculation processes based on traditional algorithms are concentrated on scenes such as straight driving and lane change, and have poor adaptability to turning scenes. In addition, due to the inconsistency of the information obtained from the source data, it is difficult to fuse multiple source data to generate road edges. Summary of the invention

[0004] The present invention aims to avoid the deficiencies in the above-mentioned prior art and to provide a real-time road edge fusion method for autonomous driving vehicles, so as to obtain more robust road edges through fusion of multi-source data and improve the safety and reliability of autonomous driving.

[0005] The present invention adopts the following technical solutions to solve the technical problems.

[0006] A real-time road edge fusion method for an autonomous driving vehicle comprises the following steps:

[0007] Step 1: Initialize the data source; the data source includes a visual perception data source S, a millimeter wave radar data source L, and a vehicle driving information data source Z;

[0008] Step 2: Update the road edge line based on the visual perception data source S to obtain the visual road edge line y S (x); Update the road edge line based on the millimeter wave radar data source L to obtain the radar road edge line y L (x); Update the driving state and conversion matrix RT based on the vehicle driving information data source Z to obtain the turning road edge line y under the vehicle turning state Z (x);

[0009] Step 3: Visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) Perform fusion processing to obtain the fused road edge line y R (x);

[0010] Step 4: The fused road edge line y R (X) is used as one of the input values ​​of the road edge line at the next moment, and the road edge line y(x) at the next moment is updated.

[0011] The real-time road edge fusion method for an autonomous driving vehicle of the present invention is also characterized in that:

[0012] Furthermore, the road edge line y(x) is expressed as follows:

[0013] y(X)=CX=C0+C1X+C2X 2 +C3X 3 (1)

[0014] In formula (1), C is the row vector [C0, C1, C2, C3], and X is the column vector [1, x, x 2 , x 3 ] TThe intercept of the road edge line y(x) is C0, in meters; the slope is atan(C1); the curvature is 2C2, in meters. The curvature change rate is 6C3, unit

[0015] Furthermore, in step 2, the process of updating the road edge line based on the visual perception data source S includes the following steps:

[0016] Step 211: Determine the validity of the road edge line in the visual perception data source S to obtain a valid road edge line;

[0017] Step 212: converting the effective road edge line into a point cloud;

[0018] Step 213: Perform ICP registration based on the effective road edge line after point cloudification, and update the visual road edge line y in the visual perception data source S S (x).

[0019] Furthermore, the formula of the ICP registration is as follows (2):

[0020]

[0021] In formula (2), the original point cloud is p′ i =[x′ i , y′ i , z′ i ], the target point cloud is p i =[x i ,y i , z i ], R and T are the rotation matrix and displacement vector based on the vehicle's initial position, respectively; ‖·‖ represents the norm. The smaller the residual value J is, the better the matching is. i represents the coordinates of the i-th point cloud, and n represents the total number of point clouds.

[0022] Furthermore, in step 2, the process of updating the road edge line based on the millimeter wave radar data source L includes the following steps:

[0023] Step 221: determining the dynamic and static state of the target sensed by the millimeter wave radar, and retaining the data of the static target;

[0024] Step 222: converting the coordinates of the static target at multiple different times to the vehicle body coordinate system CSt at the current time t, and performing point set distinction on the points on the left edge line and the right edge line of the road;

[0025] Step 223: Fit the clustering results of the points on the left edge line of the road to obtain the virtual road left edge line y 左虚拟 , the clustering results of the points on the right edge line of the road are fitted to obtain the virtual road right edge line y右虚拟 ;

[0026] Step 224: According to the virtual road left edge line y 左虚拟 and the virtual road right edge line y 右虚拟 , calculate the virtual road left edge line y 左虚拟 and the virtual road right edge line y 右虚拟 The road edge line parameter information is used to update the radar road edge line y in the millimeter wave radar data source L. L (x).

[0027] Furthermore, in step 2, the process of updating the driving state and the conversion matrix RT based on the vehicle driving information data source Z includes the following steps:

[0028] Step 231: Obtain the vehicle speed speed and yaw rate yawrate from the vehicle driving information data source Z, and calculate the yaw angle yaw and coordinates (x t ,y t ) and the transformation matrix Rt;

[0029] Step 232: Convert the coordinates of the vehicle at multiple different times into the vehicle body coordinate system CSt at the current time t, and perform curve fitting based on the converted coordinates to obtain the virtual center line y of the lane of the vehicle driving road 虚中 .

[0030] Step 233: Based on the lane virtual center line y at the current time t 虚中 and yaw angle yaw to determine whether the vehicle is moving straight or turning at the current moment t.

[0031] Furthermore, in step 3, the visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) The fusion process includes the following steps:

[0032] Step 31: Visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) performing prior score calculation;

[0033] Step 32: Remove invalid road edge lines:

[0034] Step 33: For the removed road edge lines, fusion is performed according to the point cloud score results.

[0035] The present invention also discloses an electronic device, characterized in that it comprises at least one processor and a memory in communication connection with the at least one processor;

[0036] Among them, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time road edge fusion method for autonomous driving vehicle driving.

[0037] The present invention also discloses a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the real-time road edge fusion method for an autonomous driving vehicle is implemented.

[0038] The present invention also discloses a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the real-time road edge fusion method for an autonomous driving vehicle is implemented.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] The present invention discloses a real-time road edge fusion method for autonomous driving vehicle driving, comprising: step 1: initializing a data source; the data source comprises a visual perception data source S, a millimeter wave radar data source L and a vehicle driving information data source Z; step 2: updating the road edge line based on the visual perception data source S to obtain a visual road edge line y S (x); Update the road edge line based on the millimeter wave radar data source L to obtain the radar road edge line y L (x); Update the driving state and conversion matrix RT based on the vehicle driving information data source Z to obtain the turning road edge line y under the vehicle turning state Z (x); Step 3: Visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) Perform fusion processing to obtain the fused road edge line y R (x); Step 4: The fused road edge line y after fusion R (x) is used as one of the input values ​​of the road edge line at the next moment, and the road edge line y(x) at the next moment is updated.

[0041] The technical features of the real-time road edge fusion method for an autonomous driving vehicle of the present invention include the following aspects.

[0042] (1) Providing a lightweight real-time online method for generating road edges, which only uses low-cost cameras, millimeter-wave radars, and vehicle driving data. This can effectively solve the problems of high cost and real-time prediction of road edge generation using lidar and other sensors.

[0043] (2) Providing a road edge alignment method for different sources can effectively solve the problem that road edges of multi-source data cannot be well integrated.

[0044] (3) Providing a method for generating road edges by fusing multi-source data can effectively solve the problem of poor robustness of road edges generated by a single data source and improve the safety of autonomous driving.

[0045] (4) Providing a solution to the failure of the visual edge line when turning at an intersection can effectively solve the problem of missing road edges and safety issues when turning, and provide a continuous and comprehensive reference basis for autonomous driving.

[0046] The real-time road edge fusion method for autonomous driving vehicle driving of the present invention has the advantages of obtaining more robust road edges through fusing multi-source data and improving the safety and reliability of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of a real-time road edge fusion method for an autonomous driving vehicle according to the present invention.

[0048] Figure 2 It is a schematic diagram of the grid area of ​​the present invention.

[0049] Figure 3 A distance marker map for calculating the prior score of the present invention.

[0050] The present invention will be further described below through specific implementation modes in conjunction with the accompanying drawings. DETAILED DESCRIPTION

[0051] See also Figures 1 to 3 , a real-time road edge fusion method for an autonomous driving vehicle of the present invention comprises the following steps:

[0052] Step 1: Initialize the data source; the data source includes a visual perception data source S, a millimeter wave radar data source L, and a vehicle driving information data source Z;

[0053] In the present invention, the autonomous driving vehicle includes a visual perception system, a millimeter wave radar perception system and an electronic control unit ECU.

[0054] The result set perceived by the visual perception system is the visual perception data source S. The visual perception data source S includes road edge line parameter information; the road edge line includes a left edge line and a right edge line; the road edge line parameter information includes the attribute id of the left edge line and the right edge line and the parameters such as C0, C1, C2, and C3 of the corresponding curves of the left edge line and the right edge line; the attribute id of the left edge line and the right edge line in the visual perception data source S; the attribute id is the identification mark of the edge line; the attribute id of the left edge line marks that it is located on the left side of the road, and the attribute id of the right edge line marks that it is located on the right side of the road, and the left and right attributes of the edge line can be identified through the attribute id.

[0055] The result set perceived by the millimeter wave radar perception system is the millimeter wave radar data source L. The millimeter wave radar data source L includes the longitudinal absolute velocity v.

[0056] The memory of the electronic control unit ECU stores a vehicle driving information data source Z. The vehicle driving information data source Z includes the vehicle speed speed and the yaw rate yawrate; wherein the vehicle speed at the current time t is speed t .

[0057] Step 2: Update the road edge line based on the visual perception data source S to obtain the visual road edge line y S (x); Update the road edge line based on the millimeter wave radar data source L to obtain the radar road edge line y L (x); Update the driving state and conversion matrix RT based on the vehicle driving information data source Z to obtain the turning road edge line y under the vehicle turning state Z (x);

[0058] Step 3: Visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) Perform fusion processing to obtain the fused road edge line y R (x);

[0059] Step 4: The fused road edge line y R (x) is used as one of the input values ​​of the road edge line at the next moment, and the road edge line y(x) at the next moment is updated.

[0060] A real-time road edge fusion method for driving an autonomous vehicle of the present invention, after initializing the data source, updates the road edge based on the visual perception data source S of visual perception and the millimeter wave radar data source L of millimeter wave radar perception, and updates the driving state and conversion matrix based on the vehicle driving information data source Z; obtains the road edge line parameter information of each data source based on the visual perception data source S and the millimeter wave radar data source L, aligns the road edge line parameter information of the two different data sources, determines the input source of the current road edge fusion based on the state of different perception types and the driving state, obtains the fused road edge line, and the fusion result can be used as one of the input values ​​of the road edge at the next moment. Repeating the perception and driving information input, the latest road edge can be continuously updated. A real-time road edge fusion method for driving an autonomous vehicle of the present invention obtains a more robust road edge by fusing multi-source data, which will provide a comprehensive driving safety reference guarantee for autonomous driving.

[0061] In the present invention, the road edge is mainly defined as an objectively existing or fitted road edge line estimate under the current state of the vehicle. The expression formula of the road edge line y(x) is shown in the following formula (1).

[0062] y(X)=CX=C0+C1X+C2X 2 +C3X 3 (1)

[0063] In formula (1), C is the row vector [C0, C1, C2, C3], and X is the column vector [1, x, x 2 , x 3 ] T The intercept of the road edge line y(x) is C0, in meters; the slope is atan(C1); the curvature is 2C2, in meters. The curvature change rate is 6C3, unit

[0064] In specific implementation, in step 2, the process of updating the road edge line based on the visual perception data source S includes the following steps:

[0065] Step 211: Determine the validity of the road edge line in the visual perception data source S to obtain a valid road edge line;

[0066] Determine the validity of the road edge line parameter information in the visual perception data source S. The validity determination process mainly includes whether the curvature and the intercept meet the requirements. When determining the validity of the road edge line in the visual perception data source S, the road edge lines whose curvature 2C2 and the intercept C0 are both zero or abnormal values ​​(in specific implementation, in one embodiment, the C0 value range is limited to ±10 meters, the C2 value range is limited to ±0.032, and those exceeding the above value range are determined as abnormal values) are removed as invalid results, and the valid road edge lines are retained;

[0067] Step 212: converting the effective road edge line into a point cloud;

[0068] First, define the road grid size (the default grid side length is 0.5 meters), and generate the effective point cloud coordinates of the road edge line according to formula (1) (in the specific implementation, when generating point cloud coordinates, in the vehicle body coordinate system CSt at the current time t, the distance from the rear axle of the vehicle is divided from 0 to 200 meters according to the grid distance of 0.5 meters, and assigned to x in sequence; the value of x is input into formula 1 to obtain the horizontal coordinate y corresponding to x, and the point cloud coordinate z is assigned to 0 by default). Assuming that the axis line of the rear axle is regarded as a straight line, then this is 0-200 meters, with the axis line of the rear axle as the baseline (starting line), and the rectangular area of ​​the road surface 200 meters from the baseline to the direction of the vehicle's advance is the grid area, such as Figure 2 shown.

[0069] Step 213: Perform ICP (Iterative Closest Point) registration based on the effective road edge line after point cloudification, and update the visual road edge line y in the visual perception data source S S (x).

[0070] The formula for ICP registration is as follows (2).

[0071]

[0072] In formula (2), the original point cloud is p′ i =[x′ i , y′ i , z′ i ], the target point cloud is p i =[x i ,y i , z i ], R and T are the rotation matrix and displacement vector based on the vehicle's initial position, respectively; ||·|| represents the norm. The smaller the residual value J is, the better the matching is. i represents the coordinates of the i-th point cloud, and n represents the total number of point clouds.

[0073] In specific implementation, at the current time t when the vehicle is traveling, the visual device perceives the left edge line y of the road左t and the right edge line y 右t , at the previous moment t-1, the visual device perceives the left edge line y of the road 左t-1 and the right edge line y 右t-1 The parameters of the left edge line and the right edge line at the current time t and the previous time t-1 include the attribute id of the edge line and the parameters C0, C1, C2, C3, etc. of the curve corresponding to the edge line.

[0074] The specific process of ICP registration is as follows:

[0075] (1) The left edge line y at time t-1 is calculated by formula (1) 左t-1 Convert to the road left edge line point cloud y 左云t-1 , the right edge line y at the previous moment t-1 右t-1 Convert to the right edge line point cloud y of the road 右云t-1 ; The transformation matrix Rt transforms the road left edge line point cloud y through formula (8) 左云t-1 The coordinates of the right edge line of the road and the point cloud y 右云t-1 The coordinates are converted to the vehicle body coordinate system CSt at the current time t;

[0076] The left edge line y at the current time t 左t Convert to the road left edge line point cloud y 左云t , right edge line y 右t Convert to the right edge line point cloud y of the road 右云t ;

[0077] (2) In the vehicle body coordinate system CSt at the current time t, the point cloud y of the left edge line of the road 左云t-1 , point cloud y of the right edge of the road 右云t-1 , point cloud y of the left edge of the road 左云t And the point cloud y of the right edge of the road 右云t ICP registration is performed using formula (2) to obtain the left superimposed road edge line y 左叠 and right superimposed road edge line y 右叠 ;

[0078] If the left superimposed road edge line y at different times 左叠 If the attribute ids of the left superimposed road edge lines y at different times are the same, no processing will be done; 左叠 If the attribute id is different, the id refers to the id at the latest moment for the left superimposed road edge line y 左叠 The attribute id of the road edge line at different times on the right is assigned by referring to the processing method on the left.

[0079] (3) For each superimposed road point cloud of attribute id, perform polynomial fitting according to formula (1). After polynomial fitting, the road edge line y is regenerated.s The attributes id and y s Corresponding to the parameters of the curve C0, C1, C2, C3, etc., the road edge line y s The expression can be used S (x) = C S X=C 0S +C 1S x+C 2S x 2 +C 3S x 3 Indicates; C S =[C 0S , C 1S , C 2S , C 3S ], X is the column vector [1, x, x 2 , x 3 ] T .

[0080] In specific implementation, in step 2, the process of updating the road edge line based on the millimeter wave radar data source L includes the following steps:

[0081] Step 221: determining the dynamic and static state of the target sensed by the millimeter wave radar, and retaining the data of the static target;

[0082] The longitudinal absolute speed v of the target sensed by the millimeter-wave radar (mainly dynamic and static objects above the ground and of a certain size) is used to determine whether the sensed target is a dynamic target or a static target, and the data of the dynamic target is removed, and only the data of the static target (including the coordinates of the static target) is retained. In specific implementation, considering the accuracy of the millimeter-wave radar, the speed threshold is relaxed to 2m / s (i.e. 7.2km / h) by default, and all targets with a longitudinal absolute speed lower than 2m / s are determined as static targets.

[0083] Step 222: converting the coordinates of the static target at multiple different times to the vehicle body coordinate system CSt at the current time t, and performing point set distinction on the points on the left edge line and the right edge line of the road;

[0084] The process of point set differentiation comprises the following steps:

[0085] Step 2221: The conversion matrix RT converts the coordinates of the static target at multiple different times into the coordinates of the vehicle body coordinate system CSt at the current time t through formula (8). In specific implementation, the multiple different times are assumed to be 16 sets of input data up to the current time t, that is, the coordinates of the static target at 16 times from time t-15, time t-14...time t-1 to the current time t;

[0086] Step 2222: Obtain the yaw rate yawrate (of each different static target) from the memory of the electronic control unit ECU, and calculate the average yaw angle yaw of each static target in the different static targets at multiple different times by formula (3): a According to the calculated average yaw angle (of each different static target) a , each different static target is divided into two groups of point sets: the static target point set on the left side of the vehicle and the static target point set on the right side of the vehicle.

[0087] Step 2223: When the vehicle is moving straight, the average horizontal coordinate value Ha of the horizontal coordinates of the static target point set on the left side of the vehicle and the horizontal coordinates of the static target point set on the right side of the vehicle is calculated respectively. L and Ha R ; Keep the horizontal coordinates of the static target point set on the left side of the vehicle between 0 and Ha L The horizontal coordinates of the points and the static target points on the right side of the vehicle are between 0 and Ha R All points (excluding points with abscissas exceeding the average abscissa value Ha L and Ha R Points), cluster all the retained points, and assign each point a left attribute or a right attribute. All points with left attributes constitute the left edge line point set, and all points with right attributes constitute the right edge line point set.

[0088] When the vehicle is turning, based on the vehicle's driving trajectory, the static target point set on the left side of the vehicle and the static target point set on the right side of the vehicle are respectively iterated for adjacent points, and the vehicle's driving trajectory is fitted based on formula (1), and the kth predicted trajectory point (x k ,y k ,0), extract the lateral coordinate x of the static target point set on the left side of the vehicle at x k-1 to x k Calculate x between k-1 to x k The Euclidean distance between all points Keep the point with the smallest calculated Euclidean distance dis, that is, the point to the kth driving trajectory point (x k ,y k ,0) the nearest point (x, y), and limit the default range of the Euclidean distance dis to 0-10 meters. If it exceeds the range, the processing point will be eliminated. The processing method of the static point set on the right is the same as that on the left. All the retained points are clustered, and each point is assigned a left attribute or a right attribute. All the points with the left attribute constitute the left edge line point set, and all the points with the right attribute constitute the right edge line point set.

[0089] Step 223: Fit the clustering results of the points on the left edge line of the road to obtain the virtual road left edge line y左虚拟 , the clustering results of the points on the right edge line of the road are fitted to obtain the virtual road right edge line y 右虚拟 ;

[0090] The clustering results of the points on the two edge lines of the road are used for curve fitting, and the expression of the fitting curve of the two edge lines is: L (x) = C L X=C 0L +C 1L x+C 2L x 2 +C 3L x 3 Indicates; C L =[C 0L , C 1L , C 2L , C 3L ], X is the column vector [1, x, x 2 , x 3 ] T .

[0091] Step 224: According to the virtual road left edge line y 左虚拟 and the virtual road right edge line y 右虚拟 , calculate the virtual road left edge line y 左虚拟 and the virtual road right edge line y 右虚拟 The road edge line parameter information (including the attribute ID of the left edge line and the right edge line and the parameters C0, C1, C2, C3 and so on of the curves corresponding to the left edge line and the right edge line) is updated to update the radar road edge line y in the millimeter wave radar data source L. L (x).

[0092] In specific implementation, in step 2, the process of updating the driving state and the conversion matrix RT based on the vehicle driving information data source Z includes the following steps:

[0093] Step 231: Obtain the vehicle speed speed and yaw rate yawrate from the vehicle driving information data source Z, and calculate the yaw angle yaw and coordinates (x t ,y t ) and the transformation matrix Rt;

[0094] The calculation formula of the yaw angle yaw is shown in the following formula (3);

[0095]

[0096] In formula (3), t is the current time, the default starting time of the vehicle is set to t=0, and the coordinates of the vehicle's starting position are (0, 0).

[0097] The distance dis between the vehicle position at the current time t and the starting position t The calculation formula of is shown in the following formula (4);

[0098]

[0099] In formula (4), speed t is the speed at the current time t, speed t-1 is the speed at time t-1, and Δt is the time difference between time t and time t-1.

[0100] Coordinate (x t ,y t ) is calculated as follows:

[0101]

[0102] In formula (5), (x t ,y t ) is the vehicle coordinate at the current time t. In the present invention, the road plane is assumed to be a horizontal plane, so the height coordinate z of the vehicle is not updated.

[0103] The calculation formula of the transformation matrix RTt is shown in the following formula (6);

[0104]

[0105] In formula (6), Tt is the position of the vehicle in the first frame coordinate system CSs at the current time t [x t ,y t , z t ] T； Rt is the rotation matrix based on the yaw angle yaw, and the expression of Rt is shown in formula (7). In the first frame coordinate system CSs, the time of the first frame is the time when the vehicle starts, and it is also the time when the sensor receives the first frame of data; that is, the time t = 0 is the first frame, and the platform obtains sensor information at a period of 50ms. After each acquisition, the t+1 operation is performed at the time.

[0106]

[0107] Step 232: Convert the coordinates of the vehicle at multiple different times into the vehicle body coordinate system CSt at the current time t, and perform curve fitting based on the converted coordinates to obtain the virtual center line y of the lane of the vehicle driving road 虚中 .

[0108] Formula (8) is the coordinate transformation formula.

[0109]

[0110] In formula (8), Pw is the coordinate of the first frame coordinate system CSs, P c is the coordinate of the vehicle body coordinate system CSc at the current time t. Curve fitting is performed using formula (1), and formula (1) is used to express the lane virtual center line y 虚中 .

[0111] Step 233: Based on the lane virtual center line y at the current time t 虚中 and yaw angle yaw to determine whether the vehicle is moving straight or turning at the current moment t.

[0112] When estimating the turning state, the lane virtual center line y 虚中 C1 and C2 are selected from the parameters of t, and the yaw angle yaw is selected from the 16 consecutive input values ​​from time t-15 to the current time t. When the parameter values ​​C1 and C2 change by more than 10% or 0.001, the yaw angle yaw changes by more than 0.02 radians and continuously changes by more than twice the fitting order (the fitting order of formula (1) in the present invention is 3, so the value defaults to 6), it is considered that the vehicle has entered a turning state.

[0113] In the specific implementation, in step 3, the visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) The fusion process includes the following steps:

[0114] Step 31: Visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) performing prior score calculation;

[0115] The road edge is converted into a point cloud to achieve unified fusion input; the points in the point cloud are assigned score attributes, the initial score of the point cloud is 0, and the upper limit of the score is 1. The scoring results of the point cloud in each source road edge are as follows:

[0116] (1) Visual road edge line y S (x) The point cloud score at the longitudinal distance of 1.1 times the vehicle speed from the center of the rear axle of the vehicle is 1; the speed unit is 60 km / h and the distance unit is m. Taking the vehicle speed of 60 km / h as an example, Figure 3 The score for the section from the center line of the center rear axle to the front of the vehicle is 66m, and the score for the section from 66m to 200m is 0.5.

[0117] (2) Radar road edge line y L(x) The score is 0.5 when the distance from the center of the rear axle to the center of the rear axle is 1.1 times the vehicle speed, and the score is 1 when the distance from the center of the rear axle to the maximum distance (in specific implementation, the maximum distance is set to 200m from the center of the rear axle in the longitudinal direction). Taking the vehicle speed of 60km / h as an example, if Figure 3 The score for the section from the center line of the center rear axle to the front of the vehicle is 66m, and the score for the section from 66m to 200m is 1.

[0118] (3) Turning road edge line y Z The score of (x) is set to 0.25. The weight of 0.25 is to prevent the special conditions of turning. In turning conditions, the default radar result is not good. If vision exists, it is mainly based on vision, as shown below Figure 3 The visual score in the range of 66 to 200 is 0.5. At this time, you need to set a score of 0.25, which is lower than the visual score. The range of 0 to 66m is also set at the same time. When the vision is invalid under turning conditions, that is, the vision has no score and is the default value of 0, the radar result is used.

[0119] The validity of the visual perception result is related to the vehicle speed. The result is considered valid within a distance of 1.1 times the vehicle speed. For example, when the vehicle speed is 60km / h, the effective visual perception distance is 66m from the center of the rear axle of the vehicle. The visual road edge line within this range has the highest default priority, which is set to 1. Taking the vehicle speed of 60km / h as an example, if Figure 3 The score for the section from the center line of the rear axle to the front of the vehicle is 66m, and the score for the section from 66m to 200m is 0.5.

[0120] In the present invention, only the data in the range of 0-200 meters is processed, and the rest of the range is not processed. The scoring value setting is based on a pre-set four levels: 0, 0.25, 0.5 and 1. The default initial state is set to 0, and is updated after the scoring step;

[0121] Step 32: Remove invalid road edge lines:

[0122] During the visual road edge update process, if there is less than one valid road edge line to update, setting the visual road edge is invalid; during the radar road edge update process, if there is less than one valid road edge line, setting the radar road edge is invalid.

[0123] Step 33: For the removed road edge lines, fusion is performed according to the point cloud score results.

[0124] Finally, the fusion is performed according to the score results calculated in step 31.

[0125] For different data sources, the result of the data source with the highest score is used as a reference, and the results of the remaining data sources are registered by iteratively registering the adjacent points of the point cloud. The registration formula refers to the above ICP formula (2), and the registration reference point cloud is the source data with the highest score after eliminating invalid data sources in step 32. The selection formula of the fused point cloud after registration is as follows (9):

[0126]

[0127] In formula (9), score y* (x) is the score result of different data sources based on the x coordinate under the vehicle body in step 31; the different data sources include visual perception data source S, millimeter wave radar data source L and vehicle driving information data source Z, and the corresponding edge line is the visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x), the fusion process is to iteratively input the x coordinate of the vehicle body, obtain the y value with the highest score, and generate a fused point cloud. For the fused point cloud, curve fitting parameterization and road attribute output are performed, and the parameters of the road edge of the comprehensive information are output (including the edge line id and the curve parameters c0, c1, c2 and c3 corresponding to the id, etc.).

[0128] The present invention also discloses an electronic device, characterized in that it comprises at least one processor and a memory in communication connection with the at least one processor;

[0129] Among them, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time road edge fusion method for autonomous driving vehicle driving.

[0130] The present invention also discloses a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the real-time road edge fusion method for an autonomous driving vehicle is implemented.

[0131] The present invention also discloses a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the real-time road edge fusion method for an autonomous driving vehicle is implemented.

[0132] The present invention provides a real-time road edge fusion method for an autonomous driving vehicle, which has the following technical features.

[0133] (1) Provide a lightweight real-time online method for generating road edges, solving the problems of high cost and real-time prediction of road edges generated by using lidar and other sensors;

[0134] (2) Providing a fusion method for aligning road edges from different data sources to solve the problem that road edges from multiple sources cannot be well fused;

[0135] (3) Providing a method for generating road edges by fusing multi-source data to solve the problem of poor robustness of road edges generated by a single data source and improve the safety of autonomous driving;

[0136] (4) Provide a solution to the failure of the visual edge line when turning at an intersection, solve the problem of missing road edges and safety during turns, and provide a continuous and comprehensive reference for autonomous driving.

[0137] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0138] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A real-time road edge fusion method for an autonomous driving vehicle, characterized in that: The steps include: Step 1: Initialize the data source; the data source includes a visual perception data source S, a millimeter wave radar data source L, and a vehicle driving information data source Z; Step 2: Update the road edge line based on the visual perception data source S to obtain the visual road edge line y S (x); Update the road edge line based on the millimeter wave radar data source L to obtain the radar road edge line y L (x); Update the driving state and conversion matrix RT based on the vehicle driving information data source Z to obtain the turning road edge line y under the vehicle turning state Z (x); Step 3: Visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) Perform fusion processing to obtain the fused road edge line y R (x); Step 4: The fused road edge line y R (x) is used as one of the input values ​​of the road edge line at the next moment, and the road edge line y(x) at the next moment is updated.

2. The method for real-time road edge fusion of an autonomous driving vehicle according to claim 1, characterized in that: The expression formula of the road edge line y(x) is shown in the following formula (1); y(X)=CX=C0+C1X+C2X 2 +C3X 3 (1) In formula (1), C is the row vector [C0, C1, C2, C3], and X is the column vector [1, x, x 2 , x 3 ] T , the intercept of the road edge line y(x) is C0, unit m; the slope is atan(C1); the curvature is 2C2, unit The curvature change rate is 6C3, unit 3. The real-time road edge fusion method for an autonomous driving vehicle according to claim 1, characterized in that: In step 2, the process of updating the road edge line based on the visual perception data source S includes the following steps: Step 211: Determine the validity of the road edge line in the visual perception data source S to obtain a valid road edge line; Step 212: converting the effective road edge line into a point cloud; Step 213: Perform ICP registration based on the effective road edge line after point cloudification, and update the visual road edge line y in the visual perception data source S S (x).

4. The method for real-time road edge fusion of an autonomous driving vehicle according to claim 3, characterized in that: The formula of ICP registration is as follows (2): In formula (2), the original point cloud is p′ i =[x′ i , y′ i , z′ i ], the target point cloud is p i =[x i ,y i , z i ], R and T are the rotation matrix and displacement vector based on the vehicle’s initial posture, respectively; ‖·‖ represents the norm, the smaller the residual value J is, the better the matching is, i represents the coordinates of the i-th point cloud, and n represents the total number of point clouds.

5. The real-time road edge fusion method for an autonomous driving vehicle according to claim 1, characterized in that: In step 2, the process of updating the road edge line based on the millimeter wave radar data source L includes the following steps: Step 221: determining the dynamic and static state of the target sensed by the millimeter wave radar, and retaining the data of the static target; Step 222: converting the coordinates of the static target at multiple different times to the vehicle body coordinate system CSt at the current time t, and performing point set distinction on the points on the left edge line and the right edge line of the road; Step 223: Fit the clustering results of the points on the left edge line of the road to obtain the virtual road left edge line y 左虚拟 , the clustering results of the points on the right edge line of the road are fitted to obtain the virtual road right edge line y 右虚拟 ; Step 224: According to the virtual road left edge line y 左虚拟 and the virtual road right edge line y 右虚拟 , calculate the virtual road left edge line y 左虚拟 and the virtual road right edge line y 右虚拟 The road edge line parameter information is used to update the radar road edge line y in the millimeter wave radar data source L. L (x).

6. The real-time road edge fusion method for an autonomous driving vehicle according to claim 1, characterized in that: In step 2, the process of updating the driving state and the conversion matrix RT based on the vehicle driving information data source Z includes the following steps: Step 231: Obtain the vehicle speed speed and yaw rate yawrate from the vehicle driving information data source Z, and calculate the yaw angle yaw and coordinates (x t ,y t ) and the transformation matrix Rt; Step 232: Convert the coordinates of the vehicle at multiple different times into the vehicle body coordinate system CSt at the current time t, and perform curve fitting based on the converted coordinates to obtain the virtual center line y of the lane of the vehicle driving road 虚中 ; Step 233: Based on the lane virtual center line y at the current time t 虚中 and yaw angle yaw to determine whether the vehicle is moving straight or turning at the current moment t.

7. The real-time road edge fusion method for an autonomous driving vehicle according to claim 1, characterized in that: In step 3, the visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) The fusion process includes the following steps: Step 31: Visual road edge line y S (x), radar road edge line y L (x) and the turning road edge line y Z (x) performing prior score calculation; Step 32: Remove invalid road edge lines: Step 33: For the removed road edge lines, fusion is performed according to the point cloud score results.

8. An electronic device, characterized in that: comprising at least one processor and a memory communicatively connected to the at least one processor; In which, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time road edge fusion method for autonomous driving vehicle driving as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the real-time road edge fusion method for an autonomous driving vehicle according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the real-time road edge fusion method for an autonomous driving vehicle according to any one of claims 1 to 7 is implemented.

Images