Road centering keeping method and device based on vehicle driving track prediction

Abstract

The invention relates to the technical field of unmanned driving, in particular to a road centering keeping method and device based on vehicle driving track prediction, and is used for lane centering keeping of large-curvature turning. According to the method, the curvature of the lane is determined by using an image to recognize the lane line, and if the curvature is large, the method is switched to the road centering keeping method based on trajectory prediction. According to the method, the position of the next moment is predicted by obtaining the driving state data of the vehicle and combining the vehicle kinematics model during centering keeping, and due to the fact that the time interval is small, the prediction result is accurate. Then determining a drivable area of a lane and a center line of the drivable area by combining image recognition and millimeter wave radar, and intercepting a reference position from the center line; comparing the predicted position with the reference position, if the deviation is large, adjusting the driving state data of the vehicle, and circularly executing the process to continuously correct the position of the vehicle when the vehicle is close to the center line of the drivable area.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to a road centering keeping method and device based on vehicle driving trajectory prediction. Background Art

[0002] Lane Centering Assist (LCA) is part of the Advanced Driver Assistance Systems (ADAS), which uses sensors and algorithms to help the vehicle stay in the center of its lane. The existing lane centering function may affect the response time and accuracy of the system on continuous curves, making it difficult to maintain the correct lane position. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a road centering keeping method and device based on vehicle driving trajectory prediction to solve the problems in the background technology.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A road centering keeping method based on vehicle driving trajectory prediction of the present invention comprises the steps of:

[0006] Acquire the road image and laser point cloud data in front of the vehicle; and acquire the current driving state data of the vehicle, wherein the driving state data includes the current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration and vehicle heading angle;

[0007] Extracting road line features in the road image, and determining the curvature of the lane based on the road line features;

[0008] When the curvature of the road is greater than a preset curvature threshold, calculating a predicted position of the vehicle at the next moment based on the driving state data and a pre-built vehicle kinematic model;

[0009] Determine a feasible area and a center line of the feasible area of ​​the front lane based on the laser point cloud data and the road line features, and determine a reference position of the vehicle at a next moment based on the center line of the feasible area;

[0010] The effective deviation distance between the predicted position and the reference position is calculated, and when the effective deviation distance is greater than a preset distance threshold, the current driving state of the vehicle is adjusted based on the predicted position and the reference position so that the vehicle is closer to the reference position at the next moment.

[0011] In one embodiment of the present application, extracting road line features in the road image includes:

[0012] Performing a first preprocessing on the road image to obtain a preprocessed image, wherein the first preprocessing method includes high-pass filtering and color space transformation;

[0013] Performing threshold segmentation on the preprocessed image to obtain a region of interest, wherein the region of interest is a region where a white road line or a yellow road line is located;

[0014] Performing perspective transformation on the region of interest to obtain a road line plane map from a bird's eye view;

[0015] The outer contour features and the inner contour features of the road line plan are extracted, and the outer contour features and the inner contour features are compared and used as road line features.

[0016] In an embodiment of the present application, determining the curvature of the lane based on the road line feature includes:

[0017] Curve fitting is performed on the outer contour feature and the inner contour feature respectively to obtain an outer curve and an inner curve; the curvature of the outer curve and the curvature of the inner curve are extracted respectively, and the curvature of the outer curve and the curvature of the inner curve are averaged to obtain the curvature of the lane.

[0018] In one embodiment of the present application, the predicted position of the vehicle at the next moment is calculated based on the driving state data and a pre-built vehicle kinematic model, including:

[0019] Calculate the heading angle change Δθ at the next moment, wherein the mathematical expression of the heading angle change Δθ is:

[0020]

[0021] Where a is the vehicle steering angular acceleration, Δt is the time interval between the current moment and the next moment, and ω is the angular velocity;

[0022] Calculate the heading angle θ' at the next moment based on the heading angle change Δθ and the vehicle heading angle θ, wherein θ'=θ+Δθ;

[0023] The predicted position (x', y') of the vehicle at the next moment is calculated based on the current position of the vehicle, the vehicle speed and the heading angle θ' at the next moment, wherein the mathematical expression of the predicted position (x', y') of the vehicle at the next moment is:

[0024] x'=x+vcos(θ')Δt

[0025] y'=y+vsin(θ')Δt

[0026] Where (x, y) is the current position of the vehicle.

[0027] In one embodiment of the present application, determining a feasible area and a center line of the feasible area of ​​the front lane based on the laser point cloud data and the road line feature includes:

[0028] Performing a second preprocessing on the laser point cloud data, wherein the second preprocessing includes removing outliers and noise;

[0029] Use ground filtering algorithm to region ground point cloud and non-ground point cloud;

[0030] Clustering the non-ground point cloud to obtain multiple point cloud clusters;

[0031] Roughly detecting multiple point clouds based on a pre-built recognition model to obtain recognizable targets and / or unrecognizable targets, wherein the recognition module is used to identify recognizable targets in a road environment;

[0032] When there is an unrecognizable target located within the road line feature in the plurality of point cloud clusters, performing obstacle determination on the unrecognizable target based on a preset height threshold;

[0033] Determining a drivable area of ​​a lane based on the obstacle judgment result and the road line characteristics;

[0034] Delineating a plurality of normal calibration lines in the driving area based on the curvature of the road, wherein two ends of the normal calibration lines intersect with the road line features or with the obstacle boundaries respectively;

[0035] Determine the center points of the multiple normal calibration lines, and determine the coordinates of the center points of the multiple normal calibration lines;

[0036] The coordinates of the center points of multiple normal calibration lines are curve fitted to obtain the center line of the feasible area.

[0037] In one embodiment of the present application, determining the coordinates of the center points of the plurality of normal calibration lines includes:

[0038] Convert each point in the laser point cloud data to the camera coordinate system to obtain the coordinates of multiple points in the camera coordinate system;

[0039] Constructing a depth map based on the coordinates of the plurality of points in the camera coordinate system, wherein the depth map includes depth values ​​of the plurality of pixel points;

[0040] Depth values ​​of center points of the plurality of normal calibration lines are determined from the depth map, and coordinates of the center points of the root normal calibration lines are calculated based on the depth values ​​of the center points of the plurality of normal calibration lines.

[0041] In one embodiment of the present application, determining a reference position of the vehicle at the next moment based on the center line of the feasible area includes:

[0042] Calculate the vehicle travel distance S at the next moment, S = vΔt;

[0043] Taking the current position as the starting point, a target point with a vehicle travel distance S is determined in the center line of the feasible area, and the coordinates of the target point are used as the reference position of the vehicle at the next moment.

[0044] In one embodiment of the present application, calculating the effective deviation distance between the predicted position and the reference position includes:

[0045] Calculate the distance S between the predicted position and the reference position d , and determining the angle γ between the line connecting the predicted position and the reference position and the center line of the feasible area;

[0046] Based on the distance S between the predicted position and the reference position d , the angle γ between the line connecting the predicted position and the reference position and the center line of the feasible area, and the effective deviation distance S between the predicted position and the reference position is calculated. o , S o =S d ×tanγ.

[0047] In an embodiment of the present application, adjusting the current driving state of the vehicle based on the predicted position and the reference position includes:

[0048] The heading adjustment angle Δθ' of the vehicle is calculated based on the effective deviation distance and the vehicle speed, wherein the mathematical expression of the heading adjustment angle Δθ' is:

[0049]

[0050] The angular acceleration of the vehicle is adjusted so that the heading angle adjustment amount of the vehicle at the next moment is Δθ'.

[0051] The present application also provides a road centering keeping device based on vehicle driving trajectory prediction, comprising:

[0052] An acquisition module is used to acquire the road image and laser point cloud data in front of the vehicle; and to acquire the current driving state data of the vehicle, wherein the driving state data includes the current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration and vehicle heading angle;

[0053] A feature extraction module, used to extract road line features in the road image, and determine the curvature of the lane based on the road line features;

[0054] A trajectory prediction module, used to calculate the predicted position of the vehicle at the next moment based on the driving state data and a pre-built vehicle kinematic model when the curvature of the road is greater than a preset curvature threshold;

[0055] A reference module, used to determine a feasible area and a center line of the feasible area of ​​the front lane based on the laser point cloud data and the road line features, and determine a reference position of the vehicle at a next moment based on the center line of the feasible area;

[0056] A centering adjustment module is used to calculate the effective deviation distance between the predicted position and the reference position, and when the effective deviation distance is greater than a preset distance threshold, adjust the current driving state of the vehicle based on the predicted position and the reference position so that the vehicle is closer to the reference position at the next moment.

[0057] The beneficial effects of the present invention are as follows: a method and device for maintaining the center of a road based on vehicle driving trajectory prediction of the present invention, and the present application is used for maintaining the center of a lane with a large curvature. The present application determines the curvature of the lane by using an image to identify lane lines. If the curvature is large, it switches to the road centering method based on trajectory prediction in the present application. When performing centering maintenance, the position at the next moment is predicted by obtaining the driving state data of the vehicle in combination with the vehicle kinematic model. Since the time interval is small, the prediction result is more accurate. Then, image recognition and millimeter-wave radar are combined to determine the drivable area of ​​the lane and the center line of the drivable area, and a reference position is intercepted from the center line. Comparing the predicted position and the reference position, if the deviation is large, the driving state data of the vehicle is adjusted so that the vehicle is close to the center line of the drivable area, and the process is executed repeatedly to continuously correct the vehicle position. The present application has a good maintenance effect in centering maintenance on a curve. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0059] Figure 1 is a flow chart of a road centering keeping method based on vehicle driving trajectory prediction shown in an embodiment of the present application;

[0060] Figure 2 A schematic diagram of position prediction in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of the center line of a feasible area in an embodiment of the present application;

[0062] Figure 4 It is a structural diagram of a road centering keeping device based on vehicle driving trajectory prediction shown in one embodiment of the present application. DETAILED DESCRIPTION

[0063] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0064] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show the layers related to the present invention rather than being drawn according to the number, shape and size of the layers in actual implementation. In actual implementation, the type, quantity and proportion of each layer may be changed arbitrarily, and the layer layout may also be more complicated.

[0065] In the following description, numerous details are discussed to provide a more thorough explanation of embodiments of the present invention; however, it is apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details.

[0066] Figure 1 is a flow chart of a road centering method based on vehicle driving trajectory prediction shown in an embodiment of the present application, such as Figure 1 As shown: A road centering keeping method based on vehicle driving trajectory prediction in this embodiment may include steps S110 to S150:

[0067] S110, acquiring a road image and laser point cloud data in front of the vehicle; and acquiring current driving state data of the vehicle, wherein the driving state data includes current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration, and vehicle heading angle;

[0068] In this application, the laser radar on the top of the vehicle continuously collects laser point cloud data, and the camera in front of the vehicle collects road images. The speed sensor, gyroscope and acceleration sensor inside the vehicle continuously collect driving status data. The above data needs to be aligned in the time dimension, and the road image, laser point cloud data and driving status data collected at the same time are collected at a single time.

[0069] S120, extracting road line features in the road image, and determining the curvature of the lane based on the road line features;

[0070] The existing road centering function is based on video imaging systems and lane edge detection systems. These systems monitor road conditions through sensors such as cameras, and measure the distance and direction of the vehicle to the lane lines on the left and right sides in real time. When the system recognizes the lane dividing line, it will enter the working state. The above method uses the lane line as a correction reference. The vehicle in this application does not move forward along the center line of the lane. It can be effectively maintained in a straight line or a small curvature curve. However, this method is used in a curve with a large curvature. Due to the large curvature of the lane line, the distance between the vehicle and the lane line in front is far, and the system judgment error is large. Therefore, when the vehicle follows the lane line to turn, it is easy to deviate from the following lane line in a short time. Cause the vehicle to constantly correct itself, or even fail to stay centered.

[0071] This application is used in the centering of a curve with a large curvature, so the curvature of the road line is first extracted from the road image, specifically including:

[0072] S121, performing a first preprocessing on the road image to obtain a preprocessed image, wherein the first preprocessing method includes high-pass filtering and color space transformation;

[0073] Specifically, the road image is converted from the RGB space to a grayscale image, and then the road image is high-pass filtered to retain high-frequency information for subsequent feature extraction.

[0074] S122, performing threshold segmentation on the preprocessed image to obtain a region of interest, wherein the region of interest is a region where a white road line or a yellow road line is located;

[0075] In this application, a threshold range is pre-set to segment the pre-processed image, the main purpose of which is to retain the white road line area and the yellow road line area.

[0076] S123, performing perspective transformation on the region of interest to obtain a road line plane map from a bird's eye view;

[0077] In this application, since the road image is taken from the front, the application uses perspective transformation to obtain the road plan from a bird's-eye view. The principle is: use the ratio of the front view that the near is larger and the far is smaller to enlarge the image part far away from the vehicle. This application divides the image of interest from the front side view into multiple horizontal strip sub-images, and then based on the pre-tested scaling ratio P r , enlarge each strip sub-image, that is, P i '=P i ×P r , it can be converted into a road line plan from a bird's eye view.

[0078] Scaling ratio P rThrough preliminary testing, it is obtained that by setting up cameras with the same viewing angle, a rectangular test frame on the ground is photographed, a central axis is set in the test frame, and multiple equally divided points are set on the central axis, and then it is divided into multiple horizontal strip sub-images based on the multiple equally divided points, and the bottom of the image is taken as the zero point. Then, the length and width of the test frame area in the strip sub-image are determined by counting pixels, and the reference length and width dimensions of the strip sub-image at the zero point position and the measured length and width dimensions of multiple strip sub-images are obtained, and the proportions of multiple equally divided points are calculated respectively.

[0079] Since the fineness of the strip sub-image division directly determines the continuity of the transformed contour, each row of pixels needs to be scaled when the image of interest is enlarged. Since there are only a few equally divided points in the test process, the scaling ratio of each row of pixels needs to be calculated through a linear interpolation algorithm.

[0080] Alternatively, various image processing libraries (such as OpenCV) may be used to implement perspective transformation. However, the image quality after processing using such tools is lost, which is not conducive to subsequent processing. Therefore, this application adopts the above process for processing.

[0081] S124, extracting outer contour features and inner contour features of the road line plan, and performing comparison between the outer contour features and the inner contour features, and using the outer contour features and the inner contour features as road line features.

[0082] Since the road line has inner contour features and outer contour features, the present application retains the inner and outer contours as road line features. The road line after the above scaling process can better maintain the continuity of the contour line and the extraction quality of the contour line.

[0083] S125, performing curve fitting on the outer contour feature and the inner contour feature respectively to obtain an outer curve and an inner curve; extracting the curvature of the outer curve and the curvature of the inner curve respectively, and averaging the curvature of the outer curve and the curvature of the inner curve to obtain the curvature of the lane.

[0084] Since the contour line extracted by the above process is a point set composed of multiple points and cannot be directly used for curvature extraction, this application uses curve fitting to transform it into a continuous curve, then performs curvature extraction, and finally calculates the average of the curvatures of the inner and outer contours to obtain the curvature of the road line.

[0085] S130, when the curvature of the road is greater than a preset curvature threshold, calculating a predicted position of the vehicle at the next moment based on the driving state data and a pre-built vehicle kinematic model;

[0086] If the curvature of the road line is large as calculated, the method based on vehicle trajectory prediction and lane virtual center line in this application can be used to perform road centering.

[0087] This application uses a time segmentation approach to keep the road centered, that is, at the beginning of each short time period, the vehicle position at the end of the short time period is predicted, and then if the predicted position is not on the road centerline, an adjustment is made. Since the time period is short, a fast response can be maintained, and the reference is the road centerline, so a good centering effect can be maintained in large curvature curves.

[0088] Figure 2 This is a schematic diagram of the position prediction in one embodiment of the present application, such as Figure 2 , calculating the predicted position of the vehicle at the next moment based on the driving state data and a pre-built vehicle kinematic model, including:

[0089] Calculate the heading angle change Δθ at the next moment, wherein the mathematical expression of the heading angle change Δθ is:

[0090]

[0091] Where a is the vehicle steering angular acceleration, Δt is the time interval between the current moment and the next moment, and ω is the angular velocity;

[0092] Calculate the heading angle θ' at the next moment based on the heading angle change Δθ and the vehicle heading angle θ, wherein θ'=θ+Δθ;

[0093] The predicted position (x', y') of the vehicle at the next moment is calculated based on the current position of the vehicle, the vehicle speed and the heading angle θ' at the next moment, wherein the mathematical expression of the predicted position (x', y') of the vehicle at the next moment is:

[0094] x'=x+vcos(θ')Δt

[0095] y'=y+vsin(θ')Δt

[0096] Where (x, y) is the current position of the vehicle.

[0097] The above process can be combined with Figure 2 It is understood that this application predicts the position of the vehicle based on the kinematic model. It is worth noting that the above process omits factors such as vehicle roll and tire slip. However, it has a good prediction effect in short-term prediction.

[0098] S140, determining a feasible area and a center line of the feasible area of ​​the front lane based on the laser point cloud data and the road line feature, and determining a reference position of the vehicle at a next moment based on the center line of the feasible area;

[0099] Since executing road centering based on the reference line will produce a large error on a curve with large curvature, the present application constructs a virtual feasible area center line as a reference for vehicle centering, specifically including:

[0100] S141, performing a second preprocessing on the laser point cloud data, wherein the second preprocessing includes removing outliers and noise; removing outliers or noise to ensure the accuracy of subsequent processing. In addition, downsampling can be used to reduce the amount of point cloud data and increase processing speed while maintaining sufficient resolution to capture environmental features;

[0101] S142, ground point cloud and non-ground point cloud based on ground filtering algorithm area; the ground provides the basic plane on which the vehicle can drive. Commonly used algorithms include random sampling consensus (RANSAC) algorithm, height threshold-based method, etc. For example, guardrails are usually represented in point clouds as long strips or continuous point sets, so height thresholds can be used to filter out guardrail point clouds.

[0102] S143, clustering the non-ground point cloud to obtain multiple point cloud clusters; the present application adopts Euclidean clustering to cluster point cloud data that may belong to the same object into one cluster.

[0103] S144, roughly detecting the plurality of point clouds based on a pre-built recognition model to obtain recognizable targets and / or unrecognizable targets, wherein the recognition module is used to identify recognizable targets in the road environment;

[0104] Use deep learning or machine learning methods to perform semantic segmentation on point cloud data, directly marking which points belong to guardrails, which points belong to road signs, which points belong to road piers, etc. This method requires a pre-trained model and can provide higher recognition accuracy.

[0105] S145, when there is an unrecognizable target located within the road line feature in the plurality of point cloud clusters, performing obstacle determination on the unrecognizable target based on a preset height threshold;

[0106] Since only common road facilities are trained during training, unrecognizable targets may be obstacles on the road. If the unrecognizable target is on the road line and its height is greater than the preset height threshold, it is most likely an obstacle on the lane.

[0107] S146, determining a drivable area of ​​the lane based on the obstacle judgment result and the road line feature;

[0108] For targets that may be obstacles, this application does not need to know what specific objects they are, but only needs to define the boundaries of this point cloud cluster. The center position is defined based on the obstacle boundaries and lane lines.

[0109] S147, demarcating a plurality of normal calibration lines in the driving area based on the curvature of the road, wherein two ends of the normal calibration lines intersect with the road line feature or with the obstacle boundary respectively;

[0110] Since the curvature of the road ahead is known, the road profile can be sampled to obtain multiple sampling points. The normal of each sampling point is determined, and then the normal is intercepted through the lane line or obstacle boundary to obtain the normal calibration line. The center of the normal calibration line is the center point of the vehicle.

[0111] S148, determining the center points of the multiple normal calibration lines, and determining the coordinates of the center points of the multiple normal calibration lines;

[0112] Since the road curvature and the normal calibration lines are features in the lane image, the vehicle cannot obtain the coordinate information therein. In order to find the coordinates of the center points of multiple normal calibration lines, this application converts each point in the laser point cloud data to the camera coordinate system to obtain the coordinates of multiple points in the camera coordinate system;

[0113] Specifically, for each lidar point P = (X, Y, Z), use the external parameters between the lidar and the camera to transform it into the camera coordinate system, and get P' = (X', Y', Z'). Then use the camera's internal parameter matrix K to project it onto the image plane and calculate the corresponding pixel coordinates (u pixel ,v pixel ).

[0114] Constructing a depth map based on the coordinates of the plurality of points in the camera coordinate system, wherein the depth map includes depth values ​​of the plurality of pixel points;

[0115] The application creates a depth map with the same resolution as the image, initialized to infinity or the maximum possible depth value. For each projected LiDAR point, if the depth Z' of the point is less than the depth value of the corresponding pixel in the current depth map, the depth value of the pixel is updated to Z'.

[0116] Depth values ​​of center points of the plurality of normal calibration lines are determined from the depth map, and coordinates of the center points of the root normal calibration lines are calculated based on the depth values ​​of the center points of the plurality of normal calibration lines.

[0117] Specifically, the coordinate calculation method is as follows:

[0118] For each pixel in the depth map (upixel ,v pixel )The corresponding depth value is Z.

[0119] Use the camera's intrinsic parameter matrix K to inversely solve the pixel's coordinates P under the camera coordinates C =(X C ,Y C ,Z C ), Z C = Z; and X C ,Y C Calculated by the following formula:

[0120]

[0121] In the formula, c x and c y is the coordinate of the center point of the image, f x and f y is the focal length;

[0122] Use the external parameter matrix to transform it to the standard coordinate system. The point coordinates in the standard coordinate system are P w =(X w ,Y w ,Z w )

[0123] P w =R -1 (P C -t)

[0124] Among them, R -1 is the inverse matrix of the rotation matrix R of the laser radar and the camera, and t is the translation vector of the laser radar and the camera.

[0125] S149, performing curve fitting on the coordinates of the center points of the plurality of normal calibration lines to obtain the center line of the feasible area.

[0126] Finally, by performing curve fitting on the coordinates of the center points of the multiple normal calibration lines, the reference trajectory of the vehicle, that is, the center line of the feasible area, can be obtained. Figure 3 is a schematic diagram of the center line of the feasible area in an embodiment of the present application. The center line of the feasible area obtained by fitting is as follows: Figure 3 shown.

[0127] After obtaining the center line of the feasible area, the reference position of the vehicle at the next moment can be quickly calculated, that is, the position of the vehicle after moving at the current speed Δt, that is, the vehicle travel distance S at the next moment is calculated, S = vΔt; and taking the current position as the starting point, determine the target point with a vehicle travel distance of S in the center line of the feasible area, and use the coordinates of the target point as the reference position of the vehicle at the next moment.

[0128] S150, calculating the effective deviation distance between the predicted position and the reference position, and when the effective deviation distance is greater than a preset distance threshold, adjusting the current driving state of the vehicle based on the predicted position and the reference position so that the vehicle is closer to the reference position at the next moment.

[0129] In this application, only the vertical distance between the predicted position and the center line of the feasible area is calculated. Therefore, the effective deviation distance between the predicted position and the reference position is calculated. The vertical distance between the predicted position and the center line of the feasible area can also be directly calculated. However, by calculating the deviation distance, the prediction process in the previous text can also be fed back, and the feedback amount can be increased to make the prediction result more and more accurate.

[0130] In one embodiment of the present application, calculating the effective deviation distance between the predicted position and the reference position includes:

[0131] S1501, calculating the distance S between the predicted position and the reference position d , and determining the angle γ between the line connecting the predicted position and the reference position and the center line of the feasible area;

[0132] S1502, based on the distance S between the predicted position and the reference position d , the angle γ between the line connecting the predicted position and the reference position and the center line of the feasible area, and the effective deviation distance S between the predicted position and the reference position is calculated. o , S o =S d ×tanγ.

[0133] Among them, the effective deviation distance is the vertical distance between the predicted point and the center line. Considering that the effective deviation distance is quite different from the formal distance of the vehicle at time Δt, it is approximately regarded as satisfying the tangent function.

[0134] S1503, calculating a heading adjustment angle Δθ' of the vehicle based on the effective deviation distance and the vehicle speed, wherein the mathematical expression of the heading adjustment angle Δθ' is:

[0135]

[0136] S1504: Adjust the angular acceleration of the vehicle so that the heading angle adjustment amount of the vehicle at the next moment is Δθ′.

[0137] Finally, the heading angle adjustment Δθ' is substituted into In the formula, the angular acceleration a is calculated by inverse solution. After adjusting the angular acceleration, the vehicle position can be corrected at the next moment.

[0138] Finally, the next moment is taken as the current moment, and then the process returns to S110 to execute the loop. The vehicle position can be continuously corrected, and because the road centerline is used as a reference and the time interval Δt is small, the vehicle has a faster response speed and a better following effect. The phenomenon of the vehicle moving left and right or even exiting the centering function is reduced.

[0139] The road centering keeping method based on vehicle driving trajectory prediction of the present invention is used for lane centering keeping in large curvature bends. The present application uses image recognition of lane lines to determine the curvature of the lane. If the curvature is large, it switches to the road centering keeping method based on trajectory prediction in the present application. When performing centering keeping, the position at the next moment is predicted by obtaining the vehicle's driving state data and combining it with the vehicle's kinematic model. Since the time interval is small, the prediction result is more accurate. Then, image recognition and millimeter-wave radar are combined to determine the drivable area of ​​the lane and the center line of the drivable area, and a reference position is intercepted from the center line. Compare the predicted position and the reference position. If the deviation is large, the vehicle's driving state data is adjusted to make the vehicle close to the center line of the drivable area. The process is executed repeatedly to continuously correct the vehicle position. The present application has a good keeping effect in centering keeping in curves.

[0140] like Figure 4 As shown, the present application also provides a road centering keeping device based on vehicle driving trajectory prediction, comprising:

[0141] An acquisition module is used to acquire the road image and laser point cloud data in front of the vehicle; and to acquire the current driving state data of the vehicle, wherein the driving state data includes the current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration and vehicle heading angle;

[0142] A feature extraction module, used to extract road line features in the road image, and determine the curvature of the lane based on the road line features;

[0143] A trajectory prediction module, used to calculate the predicted position of the vehicle at the next moment based on the driving state data and a pre-built vehicle kinematic model when the curvature of the road is greater than a preset curvature threshold;

[0144] A reference module, used to determine a feasible area and a center line of the feasible area of ​​the front lane based on the laser point cloud data and the road line features, and determine a reference position of the vehicle at a next moment based on the center line of the feasible area;

[0145] A centering adjustment module is used to calculate the effective deviation distance between the predicted position and the reference position, and when the effective deviation distance is greater than a preset distance threshold, adjust the current driving state of the vehicle based on the predicted position and the reference position so that the vehicle is closer to the reference position at the next moment.

[0146] The road centering device based on vehicle driving trajectory prediction of the present invention is used for lane centering in large curvature bends. The present application uses image recognition of lane lines to determine the curvature of the lane. If the curvature is large, it switches to the road centering method based on trajectory prediction in the present application. When performing centering, the position at the next moment is predicted by obtaining the vehicle's driving state data and combining it with the vehicle's kinematic model. Since the time interval is small, the prediction result is more accurate. Then, image recognition and millimeter-wave radar are combined to determine the drivable area of ​​the lane and the center line of the drivable area, and a reference position is intercepted from the center line. Compare the predicted position and the reference position. If the deviation is large, the vehicle's driving state data is adjusted to make the vehicle close to the center line of the drivable area. The process is executed repeatedly to continuously correct the vehicle position. The present application has a good maintenance effect in centering on a curve.

[0147] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented, wherein the method is the execution logic of this system.

[0148] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0149] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0150] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0151] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0152] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0153] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0154] In the above-mentioned embodiments, although the present invention has been described in conjunction with the specific embodiments of the present invention, many replacements, modifications and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. The embodiments of the present invention are intended to cover all such replacements, modifications and variations falling within the broad scope of the appended claims.

[0155] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A road centering method based on vehicle driving trajectory prediction, characterized in that: Includes steps: Acquire the road image and laser point cloud data in front of the vehicle; and acquire the current driving state data of the vehicle, wherein the driving state data includes the current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration and vehicle heading angle; Extracting road line features in the road image, and determining the curvature of the lane based on the road line features; When the curvature of the road is greater than a preset curvature threshold, calculating a predicted position of the vehicle at the next moment based on the driving state data and a pre-built vehicle kinematic model; Determine a feasible area and a center line of the feasible area of ​​the front lane based on the laser point cloud data and the road line features, and determine a reference position of the vehicle at a next moment based on the center line of the feasible area; The effective deviation distance between the predicted position and the reference position is calculated, and when the effective deviation distance is greater than a preset distance threshold, the current driving state of the vehicle is adjusted based on the predicted position and the reference position so that the vehicle is closer to the reference position at the next moment.

2. A road centering keeping method based on vehicle driving trajectory prediction according to claim 1, characterized in that: Extracting road line features in the road image includes: Performing a first preprocessing on the road image to obtain a preprocessed image, wherein the first preprocessing method includes high-pass filtering and color space transformation; Performing threshold segmentation on the preprocessed image to obtain a region of interest, wherein the region of interest is a region where a white road line or a yellow road line is located; Performing perspective transformation on the region of interest to obtain a road line plane map from a bird's eye view; The outer contour features and the inner contour features of the road line plan are extracted, and the outer contour features and the inner contour features are compared and used as road line features.

3. A road centering keeping method based on vehicle driving trajectory prediction according to claim 2, characterized in that: Determining the curvature of the lane based on the road line feature includes: Curve fitting is performed on the outer contour feature and the inner contour feature respectively to obtain an outer curve and an inner curve; the curvature of the outer curve and the curvature of the inner curve are extracted respectively, and the curvature of the outer curve and the curvature of the inner curve are averaged to obtain the curvature of the lane.

4. A road centering keeping method based on vehicle driving trajectory prediction according to claim 2, characterized in that: Calculating the predicted position of the vehicle at the next moment based on the driving state data and a pre-built vehicle kinematics model, including: Calculate the heading angle change Δθ at the next moment, wherein the mathematical expression of the heading angle change Δθ is: Where a is the vehicle steering angular acceleration, Δt is the time interval between the current moment and the next moment, and ω is the angular velocity; Calculate the heading angle θ' at the next moment based on the heading angle change Δθ and the vehicle heading angle θ, wherein θ'=θ+Δθ; The predicted position (x', y') of the vehicle at the next moment is calculated based on the current position of the vehicle, the vehicle speed and the heading angle θ' at the next moment, wherein the mathematical expression of the predicted position (x', y') of the vehicle at the next moment is: x'=x+vcos(θ')Δt y'=y+vsin(θ')Δt Where (x, y) is the current position of the vehicle.

5. The method for keeping the center of a road based on vehicle trajectory prediction according to claim 1, characterized in that: Determining a feasible area and a center line of the feasible area of ​​the front lane based on the laser point cloud data and the road line features includes: Performing a second preprocessing on the laser point cloud data, wherein the second preprocessing includes removing outliers and noise; Use ground filtering algorithm to region ground point cloud and non-ground point cloud; Clustering the non-ground point cloud to obtain multiple point cloud clusters; Roughly detecting multiple point clouds based on a pre-built recognition model to obtain recognizable targets and / or unrecognizable targets, wherein the recognition module is used to identify recognizable targets in a road environment; When there is an unrecognizable target located within the road line feature in the plurality of point cloud clusters, performing obstacle determination on the unrecognizable target based on a preset height threshold; Determining a drivable area of ​​a lane based on the obstacle judgment result and the road line characteristics; Delineating a plurality of normal calibration lines in the driving area based on the curvature of the road, wherein two ends of the normal calibration lines intersect with the road line features or with the obstacle boundaries respectively; Determine the center points of the multiple normal calibration lines, and determine the coordinates of the center points of the multiple normal calibration lines; The coordinates of the center points of multiple normal calibration lines are curve fitted to obtain the center line of the feasible area.

6. A road centering keeping method based on vehicle driving trajectory prediction according to claim 5, characterized in that: Determine the coordinates of the center points of multiple normal calibration lines, including: Convert each point in the laser point cloud data to the camera coordinate system to obtain the coordinates of multiple points in the camera coordinate system; Constructing a depth map based on the coordinates of the plurality of points in the camera coordinate system, wherein the depth map includes depth values ​​of the plurality of pixel points; Depth values ​​of center points of the plurality of normal calibration lines are determined from the depth map, and coordinates of the center points of the root normal calibration lines are calculated based on the depth values ​​of the center points of the plurality of normal calibration lines.

7. A road centering keeping method based on vehicle driving trajectory prediction according to claim 1, characterized in that: Determining a reference position of the vehicle at a next moment based on the center line of the feasible area includes: Calculate the vehicle travel distance S at the next moment, S = vΔt; Taking the current position as the starting point, a target point with a vehicle travel distance S is determined in the center line of the feasible area, and the coordinates of the target point are used as the reference position of the vehicle at the next moment.

8. The method for keeping the center of a road based on vehicle driving trajectory prediction according to claim 1, characterized in that: Calculating the effective deviation distance between the predicted position and the reference position includes: Calculate the distance S between the predicted position and the reference position d , and determining the angle γ between the line connecting the predicted position and the reference position and the center line of the feasible area; Based on the distance S between the predicted position and the reference position d , the angle γ between the line connecting the predicted position and the reference position and the center line of the feasible area, and the effective deviation distance S between the predicted position and the reference position is calculated. o , S o =S d ×tanγ.

9. A road centering keeping method based on vehicle driving trajectory prediction according to claim 2, characterized in that: Adjusting the current driving state of the vehicle based on the predicted position and the reference position includes: The heading adjustment angle Δθ' of the vehicle is calculated based on the effective deviation distance and the vehicle speed, wherein the mathematical expression of the heading adjustment angle Δθ' is: The angular acceleration of the vehicle is adjusted so that the heading angle adjustment amount of the vehicle at the next moment is Δθ'.

10. A road centering device based on vehicle driving trajectory prediction, characterized in that: include: An acquisition module is used to acquire the road image and laser point cloud data in front of the vehicle; and obtaining current driving state data of the vehicle, wherein the driving state data includes current position, vehicle speed, vehicle steering angle, vehicle steering angle acceleration and vehicle heading angle; A feature extraction module, used to extract road line features in the road image, and determine the curvature of the lane based on the road line features; A trajectory prediction module, used to calculate the predicted position of the vehicle at the next moment based on the driving state data and a pre-built vehicle kinematic model when the curvature of the road is greater than a preset curvature threshold; A reference module, used to determine a feasible area and a center line of the feasible area of ​​the front lane based on the laser point cloud data and the road line features, and determine a reference position of the vehicle at a next moment based on the center line of the feasible area; A centering adjustment module is used to calculate the effective deviation distance between the predicted position and the reference position, and when the effective deviation distance is greater than a preset distance threshold, adjust the current driving state of the vehicle based on the predicted position and the reference position so that the vehicle is closer to the reference position at the next moment.

Images