A road center keeping method and device based on vehicle driving track prediction

Abstract

The present application relates to the technical field of unmanned driving, in particular to a road center keeping method and device based on vehicle trajectory prediction, which is used for lane center keeping of large-curvature turning. The present application determines the curvature of the lane by image recognition of lane lines, and switches to the road center keeping method based on trajectory prediction in the present application if the curvature is large. When performing center keeping, the position at the next moment is predicted by acquiring the driving state data of the vehicle in combination with the kinematic model of the vehicle. Since the time interval is small, the prediction result is relatively accurate. Then the drivable area of the lane and the center line of the drivable area are determined by combining image recognition and millimeter wave radar, and the reference position is intercepted from the center line. If the deviation is large, the driving state data of the vehicle is adjusted by comparing the predicted position and the reference position, the vehicle approaches the center line of the drivable area, and the process is executed in a loop to continuously correct the position of the vehicle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned driving, and particularly relates to a road centering keeping method and device based on vehicle driving track prediction. BACKGROUND

[0002] Lane Centering Assist (LCA) is part of an Advanced Driver Assistance Systems (ADAS) that uses sensors and algorithms to help a vehicle stay in the center of its lane. The existing road centering keeping function may be affected by the response time and accuracy of the system on continuous curves, making it difficult to maintain the correct lane position. SUMMARY

[0003] Therefore, the present application aims to provide a road centering keeping method and device based on vehicle driving track prediction to solve the problems in the background art.

[0004] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0005] The road centering keeping method based on vehicle driving track prediction comprises the following steps:

[0006] Obtaining road image and laser point cloud data in front of the vehicle, and obtaining current driving state data of the vehicle, wherein the driving state data comprises current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration and vehicle heading angle;

[0007] Extracting road line features in the road image, and determining the curvature of the lane based on the road line features;

[0008] When the curvature of the road is greater than a preset curvature threshold, calculating the predicted position of the vehicle at the next moment based on the driving state data and a pre-constructed vehicle kinematics model;

[0009] Determining the feasible region and the feasible region center line of the front lane based on the laser point cloud data and the road line features, and determining the reference position of the vehicle at the next moment based on the feasible region center line;

[0010] Calculating the effective deviation distance of the predicted position and the reference position, and adjusting the current driving state of the vehicle based on the predicted position and the reference position when the effective deviation distance is greater than a preset distance threshold, so that the vehicle is closer to the reference position at the next moment.

[0011] In an embodiment of the present application, the road line features in the road image are extracted, comprising:

[0012] performing first preprocessing on the road image to obtain a preprocessed image, wherein the first preprocessing manner comprises high-pass filtering and color space transformation;

[0013] performing threshold segmentation on the preprocessed image to obtain a region of interest, wherein the region of interest is a region where a white road line or a yellow road line is located;

[0014] performing perspective transformation on the region of interest to obtain a bird's-eye view road line plan;

[0015] extracting an outer contour feature and an inner contour feature of the road line plan, and performing processing on the outer contour feature and the inner contour feature, and taking the outer contour feature and the inner contour feature as road line features.

[0016] In an embodiment of the present application, determining the curvature of the lane based on the road line features comprises:

[0017] performing curve fitting on the outer contour feature and the inner contour feature respectively to obtain an outer curve and an inner curve, extracting the curvatures of the outer curve and the inner curve respectively, and averaging the curvatures of the outer curve and the inner curve to obtain the curvature of the lane.

[0018] In an embodiment of the present application, calculating the predicted position of the vehicle at the next time based on the driving state data and a pre-constructed vehicle kinematics model comprises:

[0019] calculating a heading angle change amount Δθ, wherein the mathematical expression of the heading angle change amount Δθ is:

[0020]

[0021] wherein a is a vehicle steering angle acceleration, Δt is a time interval between the current time and the next time, and ω is an angular velocity;

[0022] calculating a heading angle θ' at the next time based on the heading angle change amount Δθ and the vehicle heading angle θ of the vehicle, wherein θ' = θ + Δθ;

[0023] calculating a predicted position (x', y') of the vehicle at the next time based on the current position of the vehicle, the vehicle speed, and the heading angle θ' at the next time, wherein the mathematical expression of the predicted position (x', y') of the vehicle at the next time is:

[0024] x' = x + vcos(θ') Δt

[0025] y' = y + vsin(θ') Δt

[0026] In the formula, (x, y) is the current position of the vehicle.

[0027] In an embodiment of the present application, the feasible region and the feasible region center line of the front lane are determined based on the laser point cloud data and the road line feature, including:

[0028] The laser point cloud data is secondly preprocessed, wherein the second preprocessing includes removing outliers and noise;

[0029] The ground filtering algorithm is used to separate the ground point cloud and the non-ground point cloud;

[0030] The non-ground point cloud is clustered to obtain a plurality of point cloud clusters;

[0031] The plurality of point cloud clusters are detected based on a pre-constructed recognition model to obtain identifiable targets and / or non-identifiable targets, wherein the recognition model is used to identify identifiable targets in a road environment;

[0032] When the non-identifiable targets located in the road line feature exist in the plurality of point cloud clusters, the non-identifiable targets are judged as obstacles based on a preset height threshold;

[0033] The drivable region of the lane is determined based on the obstacle judgment result and the road line feature;

[0034] A plurality of normal calibration lines are drawn in the driving region based on the curvature of the road, wherein two ends of the normal calibration line are respectively intersected with the road line feature or the obstacle boundary;

[0035] The center points of the plurality of normal calibration lines are determined, and the coordinates of the center points of the plurality of normal calibration lines are determined;

[0036] The coordinates of the center points of the plurality of normal calibration lines are curve-fitted to obtain a feasible region center line.

[0037] In an embodiment of the present application, the coordinates of the center points of the plurality of normal calibration lines are determined, including:

[0038] Each point in the laser point cloud data is converted to a camera coordinate system to obtain the coordinates of a plurality of points in the camera coordinate system;

[0039] A depth map is constructed based on the coordinates of the plurality of points in the camera coordinate system, wherein the depth map includes the depth values of a plurality of pixel points;

[0040] The depth values of the center points of the plurality of normal calibration lines are determined from the depth map, and the coordinates of the center points of the plurality of normal calibration lines are calculated based on the depth values of the center points of the plurality of normal calibration lines.

[0041] In an embodiment of the present application, the reference position of the vehicle at the next moment is determined based on the center line of the feasible region, comprising:

[0042] calculating the driving distance S of the vehicle at the next moment, S = vΔt;

[0043] taking the current position as the starting point, determining a target point with a driving distance S in the center line of the feasible region, and taking the coordinates of the target point as the reference position of the vehicle at the next moment.

[0044] In an embodiment of the present application, the effective deviation distance between the predicted position and the reference position is calculated, comprising:

[0045] calculating the distance S between the predicted position and the reference position d , and determining the included angle γ between the line connecting the predicted position and the reference position and the center line of the feasible region;

[0046] calculating the effective deviation distance S d between the predicted position and the reference position based on the distance S between the predicted position and the reference position o , the included angle γ between the line connecting the predicted position and the reference position and the center line of the feasible region, S o = S d × tan γ.

[0047] In an embodiment of the present application, the current driving state of the vehicle is adjusted based on the predicted position and the reference position, comprising:

[0048] calculating the heading adjustment angle Δθ' of the vehicle based on the effective deviation distance and the vehicle speed, wherein the mathematical expression of the heading adjustment angle Δθ' is:

[0049]

[0050] adjusting the angular acceleration of the vehicle so that the heading angle adjustment amount of the vehicle at the next moment is Δθ'.

[0051] The present application also provides a road centering and maintaining device based on vehicle driving trajectory prediction, comprising:

[0052] an acquisition module for acquiring road images and laser point cloud data in front of the vehicle, and acquiring current driving state data of the vehicle, wherein the driving state data comprises the current position, the vehicle speed, the vehicle steering angle, the vehicle steering angle acceleration and the vehicle heading angle;

[0053] a feature extraction module for extracting road line features in the road images, and determining the curvature of the lane based on the road line features;

[0054] a trajectory prediction module, configured to calculate a predicted position of the vehicle at a next moment based on the driving state data and a pre-constructed vehicle kinematics model when the curvature of the road is greater than a preset curvature threshold;

[0055] a reference module, configured to determine a drivable area and a drivable area center line of a front lane based on the laser point cloud data and the road line feature, and determine a reference position of the vehicle at the next moment based on the drivable area center line;

[0056] a centering adjustment module, configured to calculate an effective deviation distance of the predicted position and the reference position, and adjust the current driving state of the vehicle based on the predicted position and the reference position when the effective deviation distance is greater than a preset distance threshold, so that the vehicle is closer to the reference position at the next moment.

[0057] The present application has the following advantages: the present application is a lane centering keeping method and device based on vehicle driving trajectory prediction, and is used for lane centering keeping in a large-curvature curve. The present application determines the curvature of the lane by image recognition of lane lines, and switches to the lane centering keeping method based on trajectory prediction in the present application if the curvature is large. When centering keeping is performed, the position at the next moment is predicted by obtaining the driving state data of the vehicle in combination with the vehicle kinematics model, and the prediction result is relatively accurate due to the small time interval. The drivable area of the lane and the center line of the drivable area are determined by image recognition and millimeter wave radar combination, and the reference position is intercepted from the center line. If the deviation between the predicted position and the reference position is large, the driving state data of the vehicle is adjusted to make the vehicle close to the center line of the drivable area, and the process is repeatedly executed to continuously correct the position of the vehicle. The present application has good keeping effect in curve centering keeping. BRIEF DESCRIPTION OF DRAWINGS

[0058] The present application will be further described below in combination with the drawings and embodiments:

[0059] Figure 1 is a flowchart of a lane centering keeping method based on vehicle driving trajectory prediction in an embodiment of the present application;

[0060] Figure 2 is a principle diagram of position prediction in an embodiment of the present application;

[0061] Figure 3 is a schematic diagram of a drivable area center line in an embodiment of the present application;

[0062] Figure 4 is a structure diagram of a lane centering keeping device based on vehicle driving trajectory prediction in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The present application is described in greater detail by way of specific examples as follows. Other advantages and benefits of the present application will become apparent to those skilled in the art upon consideration of the disclosure herein. The present application can be implemented or applied in other different embodiments and various modifications or changes can be made to the details of the application without departing from the spirit of the application. It should be noted that the following examples and features in the examples can be combined with each other in the case of no conflict.

[0064] It should be noted that the drawings provided in the following examples only schematically illustrate the basic concept of the present application, and only the layers related to the present application are shown in the drawings, rather than being drawn according to the number of layers, shapes and sizes in actual implementation. The actual implementation of each layer can be a random change in shape, number and proportion, and the layer layout pattern can also be more complex.

[0065] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details.

[0066] Figure 1 is a flowchart of a road center keeping method based on vehicle trajectory prediction shown in an embodiment of the present application, as shown in Figure 1 The road center keeping method based on vehicle trajectory prediction of the present embodiment can include steps S110 to S150:

[0067] S110, acquiring road images and laser point cloud data in front of the vehicle, and acquiring current driving state data of the vehicle, wherein the driving state data includes current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration and vehicle heading angle;

[0068] In the present application, laser point cloud data is continuously collected by a laser radar on the top of the vehicle, and road images are collected by a camera in front of the vehicle. Driving state data is continuously collected by a speed sensor, a gyroscope and an acceleration sensor inside the vehicle. The above data needs to be aligned in the time dimension, and single collection is all the road images, laser point cloud data and driving state data at the same time.

[0069] S120, extracting road line features in the road images, and determining the curvature of the lane based on the road line features;

[0070] The existing road center keeping function is realized based on a video imaging system and a lane boundary detection system. These systems monitor the road condition through sensors such as cameras, and measure the distance and orientation of the vehicle from the left and right lane lines in real time. When the system identifies the lane boundary, it enters the working state. The above-mentioned method takes the lane line as the correction reference, and the vehicle does not advance along the lane center line. It can be effectively kept in a straight line or a small-curvature curve, but in a large-curvature curve, the vehicle is far away from the front lane line, and the system has a large error in judgment. Therefore, when the vehicle follows the lane line to turn, it is easy to deviate from the lane line in a short time. The vehicle constantly corrects itself, and even cannot keep in the center.

[0071] The present application is used in the center keeping of a large-curvature curve. Therefore, the curvature of the road line is first extracted from the road image, which specifically includes:

[0072] S121, first preprocessing the road image to obtain a preprocessed image, wherein the first preprocessing method includes high-pass filtering and color space conversion;

[0073] Specifically, the road image is converted from the RGB space to the gray image, and then the road image is high-pass filtered to retain the high-frequency information, which is convenient for subsequent feature extraction.

[0074] S122, threshold segmentation of the preprocessed image to obtain a region of interest, wherein the region of interest is the region where the white road line or the yellow road line is located;

[0075] In the present application, the threshold range is set in advance to segment the preprocessed image. The main purpose is to retain the white road line region and the yellow road line region.

[0076] S123, perspective transformation of the region of interest to obtain a bird's-eye view road line plan;

[0077] In the present application, since the road image is taken from the front view, the bird's-eye view road plan is obtained by perspective transformation. The principle is that the distance from the vehicle is enlarged by using the ratio of the front view that is large near and small far. The front view of the image of interest is divided into a plurality of horizontal strip sub-images, and then the scaling ratio P r Each strip sub-image is enlarged, i.e. i ' = P i × P r , which can be converted into a bird's-eye view road line plan.

[0078] The scaling ratio P rThe pre-test is obtained by erecting a camera with the same perspective, shooting a rectangular test frame on the ground, the test frame is provided with a center axis, a plurality of equidistant points are arranged on the center axis, and then the plurality of equidistant points are divided into a plurality of horizontal strip sub-images, the bottom of the image is taken as a zero point, and then the length and width of the test frame region in the strip sub-image are determined by pixel point counting, the reference length and width dimensions of the strip sub-image of the zero point position and the measured length and width dimensions of the plurality of strip sub-images are obtained, and the proportions of the plurality of equidistant points are calculated respectively.

[0079] Since the fineness of the strip sub-image division directly determines the continuity of the transformed contour, when the image of interest is enlarged, each row of pixels needs to be scaled. Since there are only a few proportions of equidistant points in the test process, a linear interpolation algorithm is also needed to calculate the scaling proportion of each row of pixels.

[0080] In addition, various image processing libraries (such as OpenCV) can also be used to realize the perspective transformation. However, the image quality processed by using such a tool is lost, which is not conducive to subsequent processing. Therefore, the above process is adopted for processing.

[0081] S124, extracting the outer contour feature and the inner contour feature of the road line plan, and performing the outer contour feature and the inner contour feature, and taking the outer contour feature and the inner contour feature as road line features.

[0082] Since the road line has an inner contour feature and an outer contour feature, the inner and outer contours are retained as road line features in the present application. The road line processed by the above scaling process can better maintain the continuity of the contour line and the extraction quality of the contour line.

[0083] S125, respectively performing curve fitting on the outer contour feature and the inner contour feature to obtain an outer curve and an inner curve; extracting the curvatures of the outer curve and the inner curve, and averaging the curvatures of the outer curve and the inner curve to obtain the curvature of the lane.

[0084] Since the contour line extracted by the above process is a point set composed of a plurality of points, it cannot be directly used for curvature extraction, therefore, the present application adopts a curve fitting method to transform it into a continuous curve. Then the curvature is extracted, and finally the curvatures of the inner and outer contours are averaged to obtain the curvature of the road line.

[0085] S130, when the curvature of the road is greater than a preset curvature threshold, calculating the predicted position of the vehicle at the next moment based on the driving state data and a pre-constructed vehicle kinematics model;

[0086] If the road line curvature is large through calculation, the road center keeping can be performed by the vehicle trajectory prediction and lane virtual center line based method in the present application.

[0087] The present application adopts time segmentation to keep road centering, that is, the vehicle position at the end of each short time period is predicted at the beginning of the short time period, and then adjusted if the predicted position is not on the road center line. Since the time period is short, fast response can be kept, and the reference basis is the road center line, so better centering effect can be kept in large curvature curves.

[0088] Figure 2 For the principle of position prediction in an embodiment of the present application, as shown in Figure 2 , the predicted position of the vehicle at the next time is calculated based on the driving state data and a pre-constructed vehicle kinematics model, including:

[0089] The heading angle change amount Δθ at the next time is calculated, and the mathematical expression of the heading angle change amount Δθ is:

[0090]

[0091] In the formula, a is the vehicle steering angle acceleration, Δt is the time interval between the current time and the next time, and ω is the angular velocity;

[0092] The heading angle θ' at the next time is calculated based on the heading angle change amount Δθ and the vehicle heading angle θ of the vehicle, and θ' = θ + Δθ;

[0093] The predicted position (x', y') of the vehicle at the next time is calculated based on the current position of the vehicle, the vehicle speed, and the heading angle θ' at the next time, and the mathematical expression of the predicted position (x', y') of the vehicle at the next time is:

[0094] x' = x + vcos(θ') Δt

[0095] y' = y + vsin(θ') Δt

[0096] In the formula, (x, y) is the current position of the vehicle.

[0097] The above process can be understood in combination with Figure 2 The present application predicts the position of the vehicle based on the kinematics model, and it is worth noting that the above process omits factors such as vehicle roll and tire slip. However, it has good prediction effect in short time prediction.

[0098] S140, determine a feasible region and a feasible region center line of a front lane based on the laser point cloud data and the road line feature, and determine a reference position of the vehicle at a next time based on the feasible region center line;

[0099] Since performing road centering based on the reference line may cause a larger error when the curvature of the road is large, the application constructs a virtual feasible region center line as a reference for vehicle centering, specifically including:

[0100] S141, second preprocessing based on the laser point cloud data, wherein the second preprocessing includes removing outliers and noise; remove outliers or noise to ensure the accuracy of subsequent processing. In addition, by downsampling, the amount of point cloud data can be reduced, the processing speed can be improved, and at the same time sufficient resolution can be maintained to capture environmental features;

[0101] S142, ground filtering algorithm based on region ground point cloud and non-ground point cloud; the ground provides a basic plane on which the vehicle can travel. Common algorithms include the random sample consensus (RANSAC) algorithm, the height threshold-based method, etc. For example, guardrails in point cloud are usually long strip-shaped or continuous point sets, so a height threshold can be used to filter out guardrail point cloud.

[0102] S143, clustering the non-ground point cloud to obtain a plurality of point cloud clusters; the application uses Euclidean clustering to cluster point cloud data that may belong to the same object in a cluster.

[0103] S144, detecting a plurality of point cloud clusters based on a pre-constructed recognition model to obtain identifiable targets and / or non-identifiable targets, wherein the recognition model is used to identify identifiable targets in the road environment;

[0104] Use deep learning or machine learning methods to perform semantic segmentation on point cloud data, directly marking which points belong to guardrails, which points belong to road signs, and which points belong to road piers, etc. This method requires a pre-trained model and can provide higher recognition accuracy.

[0105] S145, when there are non-identifiable targets within the road line feature in the plurality of point cloud clusters, performing obstacle judgment on the non-identifiable targets based on a pre-set height threshold;

[0106] Since only common road facilities are trained during training, for non-identifiable targets, they may be obstacles in the road. If the non-identifiable target in the road line has a height greater than the pre-set height threshold, it is likely to be an obstacle on the lane.

[0107] S146, determining a drivable region of the lane based on the obstacle judgment result and the road line feature;

[0108] For the target that may be an obstacle, it is not necessary to know why it is a specific object in this application, as long as the boundary of this point cloud cluster is demarcated. And the center position is demarcated based on the obstacle boundary and the lane line.

[0109] S147, demarcate a plurality of normal calibration lines in the driving area based on the curvature of the road, wherein the two ends of the normal calibration line respectively intersect the road line feature or intersect the obstacle boundary;

[0110] Since the curvature of the road ahead is known, the road profile can be sampled to obtain a plurality of sampling points. The normal line of each sampling point is determined, and then the normal line is intercepted by the lane line or the obstacle boundary to obtain the normal calibration line. The center of the normal calibration line is the center point of the vehicle driving.

[0111] S148, determine the center point of the plurality of normal calibration lines, and determine the coordinates of the center point of the plurality of normal calibration lines;

[0112] Since the road curvature and the normal calibration line are features in the lane image, the vehicle cannot obtain the coordinate information thereof. In order to find the coordinates of the center point of the plurality of normal calibration lines, each point in the laser point cloud data is converted to the camera coordinate system to obtain the coordinates of the plurality of points in the camera coordinate system;

[0113] Specifically, for each laser radar point P=(X, Y, Z), the extrinsic parameter between the laser radar and the camera is used to convert it to the camera coordinate system to obtain P'=(X', Y', Z'). Then, the intrinsic matrix K of the camera is used to project it to the image plane, and the corresponding pixel coordinates (u pixel ,v pixel ) are calculated.

[0114] Construct a depth map based on the coordinates of the plurality of points in the camera coordinate system, wherein the depth map includes the depth values of a plurality of pixel points;

[0115] Wherein, the application creates a depth map with the same resolution as the image, which is initialized to infinity or the maximum possible depth value. For each projected laser radar point, if the depth Z' of the point is less than the depth value of the corresponding pixel in the current depth map, the depth value of the pixel is updated to Z'.

[0116] Determine the depth value of the center point of the plurality of normal calibration lines from the depth map, and calculate the coordinates of the center point of the plurality of normal calibration lines based on the depth value of the center point of the plurality of normal calibration lines.

[0117] Specifically, the coordinate calculation method is as follows:

[0118] For each pixel (upixel v pixel ) its corresponding depth value is Z.

[0119] The pixel point is inversely solved using the intrinsic matrix K of the camera to obtain the coordinates P of the pixel point under the camera coordinates C =(X C ,Y C ,Z C ), Z C =Z; and X C ,Y C are calculated by the following formula:

[0120]

[0121] In the formula, c x and c y are the coordinates of the image center point, f x and f y are focal lengths;

[0122] The pixel point is converted to the standard coordinates using the extrinsic matrix, and the point coordinates under the standard coordinate system are P w =(X w ,Y w ,Z w )

[0123] P w =R -1 (P C -t)

[0124] Wherein, R -1 is the inverse matrix of the rotation matrix R of the laser radar and the camera, and t is the translation vector of the laser radar and the camera.

[0125] S149, the coordinates of the center points of the plurality of normal calibration lines are curve fitted to obtain a feasible region center line.

[0126] Finally, the coordinates of the center points of the plurality of normal calibration lines are curve fitted to obtain a reference trajectory of the vehicle moving forward, i.e. a feasible region center line. Figure 3 is a schematic diagram of the feasible region center line in an embodiment of the present application, and the fitted feasible region center line is as shown in Figure 3 .

[0127] After obtaining the feasible region center line, the reference position of the vehicle at the next time can be quickly obtained, i.e. the position of the vehicle after moving at the current speed for Δt, i.e. the vehicle driving distance S at the next time is calculated, S=vΔt; and the target point with the vehicle driving distance S is determined in the feasible region center line with the current position as the starting point, and the coordinates of the target point are taken as the reference position of the vehicle at the next time.

[0128] S150, calculating an effective deviation distance of the predicted position and the reference position, and when the effective deviation distance is greater than a preset distance threshold, adjusting a current driving state of the vehicle based on the predicted position and the reference position, so that the vehicle is closer to the reference position at a next time.

[0129] In the present application, only the vertical distance between the predicted position and the center line of the feasible region is calculated, and thus the effective deviation distance of the predicted position and the reference position is calculated. The vertical distance between the predicted position and the center line of the feasible region can also be directly calculated. However, by calculating the deviation distance, the prediction process in the foregoing can be fed back, and the feedback amount can be increased so that the prediction result is more and more accurate.

[0130] In an embodiment of the present application, the effective deviation distance of the predicted position and the reference position is calculated, including:

[0131] S1501, calculating a distance S between the predicted position and the reference position d , and determining an included angle γ between a line connecting the predicted position and the reference position and the center line of the feasible region;

[0132] S1502, calculating the effective deviation distance S of the predicted position and the reference position based on the distance S between the predicted position and the reference position d , and the included angle γ between the line connecting the predicted position and the reference position and the center line of the feasible region o , S o = S d × tan γ.

[0133] The effective deviation distance is the vertical distance between the predicted point and the center line. It is considered that the effective deviation distance is greatly different from the actual distance of the vehicle at the time Δt, and thus the effective deviation distance is approximately considered to satisfy the tangent function.

[0134] S1503, calculating a heading adjustment angle Δθ' of the vehicle based on the effective deviation distance and the vehicle speed, wherein a mathematical expression of the heading adjustment angle Δθ' is:

[0135]

[0136] S1504, adjusting an angular acceleration of the vehicle, so that a heading angle adjustment amount of the vehicle at a next time is Δθ'.

[0137] Finally, the heading angle adjustment amount Δθ' is brought into the formula , and an angular acceleration a is calculated by inverse solution. After the angular acceleration is adjusted, the position of the vehicle can be corrected at the next time.

[0138] Finally, the next moment is taken as the current moment, and then the loop is executed in S110. That is, the vehicle position can be continuously corrected, and since the road center line is taken as the reference and the time interval At is small, the reaction speed is fast and the following effect is better. The phenomenon of vehicle left-right drawing and even exiting the centering function is reduced.

[0139] The road centering keeping method based on vehicle trajectory prediction of the application is used for lane centering keeping of large-curvature over-bending. The application determines the curvature of the lane by image recognition of the lane line, and if the curvature is large, the road centering keeping method based on trajectory prediction in the application is switched to. When centering keeping is performed, the position at the next moment is predicted by acquiring the driving state data of the vehicle in combination with the vehicle kinematics model, and since the time interval is small, the prediction result is more accurate. The drivable area of the lane and the center line of the drivable area are determined by image recognition and millimeter wave radar combination, and the reference position is intercepted from the center line. If the deviation is large, the driving state data of the vehicle is adjusted to make the vehicle close to the center line of the drivable area by comparing the predicted position and the reference position, and the process is executed in a loop to continuously correct the vehicle position. The application has a good keeping effect in the centering keeping of the bending lane.

[0140] As shown in Figure 4 The application further provides a road centering keeping device based on vehicle driving trajectory prediction, comprising:

[0141] An acquisition module is configured to acquire road images and laser point cloud data in front of the vehicle, and acquire current driving state data of the vehicle, wherein the driving state data comprises a current position, a vehicle speed, a vehicle steering angle, a vehicle steering angle acceleration, and a vehicle heading angle;

[0142] A feature extraction module is configured to extract road line features in the road images, and determine the curvature of the lane based on the road line features;

[0143] A trajectory prediction module is configured to calculate a predicted position of the vehicle at the next moment based on the driving state data and a pre-constructed vehicle kinematics model when the curvature of the road is greater than a preset curvature threshold;

[0144] A reference module is configured to determine a drivable area and a drivable area center line of the lane in front based on the laser point cloud data and the road line features, and determine a reference position of the vehicle at the next moment based on the drivable area center line;

[0145] The centering adjustment module is configured to calculate an effective deviation distance of the predicted position and the reference position, and when the effective deviation distance is greater than a preset distance threshold, adjust a current driving state of the vehicle based on the predicted position and the reference position, so that the vehicle is closer to the reference position at a next time.

[0146] The road centering keeping device based on vehicle driving track prediction provided by the present application is used for lane centering keeping of a large-curvature over-bend. The present application determines the curvature of the lane by image recognition of lane lines, and switches to the road centering keeping method based on track prediction in the present application if the curvature is large. When the centering keeping is performed, the position at the next time is predicted by acquiring the driving state data of the vehicle in combination with the kinematic model of the vehicle, and the prediction result is relatively accurate due to the small time interval. Then, the drivable area of the lane and the center line of the drivable area are determined by image recognition and millimeter wave radar combination, and the reference position is intercepted from the center line. If the deviation between the predicted position and the reference position is large, the driving state data of the vehicle is adjusted to make the vehicle close to the center line of the drivable area, and the process is repeatedly executed, so that the position of the vehicle is continuously corrected. The present application has good keeping effect in the centering keeping of the bend.

[0147] The embodiment also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement any of the methods in the embodiment.

[0148] The embodiment also provides an electronic terminal, which comprises a processor and a memory.

[0149] The memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, so that the terminal executes any of the methods in the embodiment.

[0150] The computer readable storage medium in the embodiment can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by the hardware of the computer program. The computer program mentioned above can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the storage medium mentioned above includes ROM, RAM, magnetic disc or optical disc and various media that can store program codes.

[0151] The electronic terminal provided by the embodiment comprises a processor, a memory, a transceiver and a communication interface, the memory and the communication interface are connected with the processor and the transceiver and complete communication between each other, the memory is configured to store a computer program, the communication interface is configured to communicate, and the processor and the transceiver are configured to run the computer program, so that the electronic terminal executes each step of the method.

[0152] In the present embodiment, the memory can include a random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory.

[0153] The processor described above can be a general processor including a central processing unit (CPU), a network processor (NP), etc., and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component.

[0154] In the above-described embodiments, although the present application has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. Embodiments of this application are intended to embrace all such alternatives, modifications and variations as can fall within the scope of the appended claims.

[0155] The above-described embodiments are merely illustrative for the principles and effects of the present application, and are not intended to limit the present application. Any modification or change made by any person skilled in the art without departing from the spirit and scope of the present application shall be covered by the appended claims.

Claims

1. A road centering preservation method based on vehicle trajectory prediction, characterized in that, Including the following steps: Acquire road images and laser point cloud data in front of the vehicle; and acquire the vehicle's current driving status data, including current position, vehicle speed, vehicle steering angle, vehicle steering angular acceleration, and vehicle heading angle; Extract road line features from the road image, and determine the curvature of the lanes based on the road line features; When the curvature of the road is greater than a preset curvature threshold, the predicted position of the vehicle at the next moment is calculated based on the driving state data and the pre-built vehicle kinematics model. Based on the laser point cloud data and the road line features, the feasible area and the center line of the feasible area of ​​the lane ahead are determined, and the reference position of the vehicle at the next moment is determined based on the center line of the feasible area. Calculate the effective deviation distance between the predicted position and the reference position, and when the effective deviation distance is greater than a preset distance threshold, adjust the current driving state of the vehicle based on the predicted position and the reference position so that the vehicle is closer to the reference position in the next moment.

2. The road centering method based on vehicle trajectory prediction according to claim 1, characterized in that, Extracting road line features from the road image includes: The road image is subjected to a first preprocessing to obtain a preprocessed image, wherein the first preprocessing method includes high-pass filtering and color space transformation; The preprocessed image is segmented using a threshold to obtain the region of interest, wherein the region of interest is the area containing either a white road line or a yellow road line; A perspective transformation is performed on the region of interest to obtain a bird's-eye view of the road lines. Extract the outer contour features and inner contour features of the road line plan view, and use the outer contour features and inner contour features as road line features.

3. The road centering method based on vehicle trajectory prediction according to claim 2, characterized in that, Determining the lane curvature based on the road alignment features includes: Curve fitting is performed on the outer contour feature and the inner contour feature respectively to obtain the outer curve and the inner curve; the curvature of the outer curve and the curvature of the inner curve are extracted respectively, and the average value of the curvature of the outer curve and the curvature of the inner curve is calculated to obtain the curvature of the lane.

4. The road centering method based on vehicle trajectory prediction according to claim 2, characterized in that, The predicted position of the vehicle at the next moment is calculated based on the driving state data and the pre-built vehicle kinematic model, including: Calculate the change in heading angle at the next moment. Wherein, the change in heading angle The mathematical expression is: ； In the formula, For the vehicle's steering angle acceleration, The time interval between the current moment and the next moment. Angular velocity; Based on the change in heading angle and the vehicle heading angle Calculate the heading angle at the next moment. ,in, ; Based on the vehicle's current position, vehicle speed, and heading angle at the next moment. Calculate the vehicle's predicted position at the next moment. Among them, the predicted position of the vehicle at the next moment. The mathematical expression is: ； In the formula, This is the vehicle's current location.

5. The road centering method based on vehicle trajectory prediction according to claim 1, characterized in that, Determining the feasible area and centerline of the feasible area of ​​the lane ahead based on the laser point cloud data and the road line features includes: The laser point cloud data is subjected to a second preprocessing, wherein the second preprocessing includes removing outliers and noise; A ground filtering algorithm is used to distinguish between ground point clouds and non-ground point clouds; Clustering the non-ground point clouds yields multiple point cloud clusters; Multiple point cloud clusters are detected based on a pre-built recognition model to obtain identifiable targets and / or unidentifiable targets. The recognition module is used to identify identifiable targets in the road environment. When there is an unidentifiable target located within the road line feature in multiple point cloud clusters, the unidentifiable target is judged as an obstacle based on a preset height threshold. The drivable area of ​​the lane is determined based on the obstacle assessment results and the road line characteristics. Multiple normal calibration lines are drawn within the driving area based on the curvature of the road, wherein the two ends of the normal calibration lines intersect the road line features or the boundary of the obstacle, respectively. Determine the center point of the multiple normal calibration lines, and determine the coordinates of the center point of the multiple normal calibration lines; Curve fitting is performed on the coordinates of the center points of multiple normal calibration lines to obtain the center line of the feasible region.

6. The road centering method based on vehicle trajectory prediction according to claim 5, characterized in that, Determine the coordinates of the center points of multiple normal calibration lines, including: Transform each point in the laser point cloud data to the camera coordinate system to obtain the coordinates of multiple points in the camera coordinate system; A depth map is constructed based on the coordinates of multiple points in the camera coordinate system, wherein the depth map includes the depth values ​​of multiple pixels. The depth values ​​of the center points of multiple normal calibration lines are determined from the depth map, and the coordinates of the center points of the root normal calibration lines are calculated based on the depth values ​​of the center points of the multiple normal calibration lines.

7. The road centering method based on vehicle trajectory prediction according to claim 1, characterized in that, Determining the vehicle's reference position at the next moment based on the centerline of the feasible region includes: Calculate the vehicle's travel distance at the next moment. ; Starting from the current location, determine the vehicle's travel distance within the centerline of the feasible area as follows: The target point is determined, and the coordinates of the target point are used as the reference position of the vehicle at the next moment.

8. The road centering method based on vehicle trajectory prediction according to claim 1, characterized in that, Calculating the effective deviation distance between the predicted position and the reference position includes: Calculate the distance between the predicted position and the reference position. And determine the angle between the line connecting the predicted location and the reference location and the centerline of the feasible region. ; Based on the distance between the predicted location and the reference location The angle between the line connecting the predicted location and the reference location and the centerline of the feasible region. Calculate the effective deviation distance between the predicted position and the reference position. , .

9. A road centering method based on vehicle trajectory prediction according to claim 2, characterized in that, Adjusting the vehicle's current driving state based on the predicted location and the reference location includes: The vehicle's heading adjustment angle is calculated based on the effective deviation distance and vehicle speed. The heading adjustment angle The mathematical expression is: ； Adjust the vehicle's angular acceleration so that the vehicle's heading angle adjustment in the next moment is... .

10. A road centering keeping device based on vehicle trajectory prediction, characterized in that, include: The acquisition module is used to acquire road images and laser point cloud data in front of the vehicle; And to acquire the vehicle's current driving status data, which includes the current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration, and vehicle heading angle; The feature extraction module is used to extract road line features from the road image and determine the curvature of the lanes based on the road line features; The trajectory prediction module is used to calculate the predicted position of the vehicle at the next moment based on the driving state data and the pre-built vehicle kinematic model when the curvature of the road is greater than a preset curvature threshold. The reference module is used to determine the feasible area and the center line of the feasible area of ​​the lane ahead based on the laser point cloud data and the road line features, and to determine the reference position of the vehicle at the next moment based on the center line of the feasible area. The centering adjustment module is used to calculate the effective deviation distance between the predicted position and the reference position, and when the effective deviation distance is greater than a preset distance threshold, adjust the current driving state of the vehicle based on the predicted position and the reference position so that the vehicle is closer to the reference position in the next moment.

Images