Device and method for controlling a vehicle

Abstract

An apparatus and a method for manipulating a vehicle are provided. The method comprises the following steps: obtaining point cloud data of a vehicle through a light beam, obtaining an RGB image of the vehicle through a camera, and obtaining a current speed of the vehicle through a wheel speed sensor; obtaining a steering angle using the current speed and a local path associated with the point cloud data; obtaining a steering compensation angle by using the current speed and a lane center line deviation error associated with the RGB image; and obtaining a control command of the vehicle by using the steering angle and the steering compensation angle, and controlling the vehicle to run on the lane according to the control command.

Description

Technical Field

[0001] The invention relates to a device and a method for controlling a vehicle. Background Art

[0002] Currently, the technology for controlling vehicles (such as unmanned vehicles) has been continuously developing. However, the existing technology is usually easily affected by weather and complex lane environments, resulting in poor control effects of the vehicle. Summary of the invention

[0003] The invention provides a device and method for controlling a vehicle, which can control the vehicle so as to keep the vehicle stably in the center of a lane.

[0004] The device for controlling a vehicle of the present invention includes a LiDAR, a camera, a wheel speed sensor, and a processor. The processor is coupled to the LiDAR, the camera, and the wheel speed sensor, wherein the processor obtains point cloud data of the vehicle through the LiDAR, obtains an RGB (Red, Green, Blue) image of the vehicle through the camera, and obtains the current speed of the vehicle through the wheel speed sensor; the processor obtains a steering angle (target angle) using the current speed and local path way points associated with the point cloud data; the processor obtains a steering compensation angle (compensator angle) using the current speed and a central lane distance error (central lane distance error) associated with the RGB image; the processor obtains a steering command (steering command) of the vehicle using the steering angle and the steering compensation angle, and controls the vehicle to travel in the lane according to the steering command.

[0005] The method for controlling a vehicle of the present invention comprises the following steps: obtaining point cloud data of the vehicle through a lidar, obtaining an RGB image of the vehicle through a camera, and obtaining the current speed of the vehicle through a wheel speed sensor; obtaining a steering angle using the current speed and a local path associated with the point cloud data; obtaining a steering compensation angle using the current speed and a lane centerline deviation error associated with the RGB image; and obtaining a vehicle control command using the steering angle and the steering compensation angle, and controlling the vehicle to travel in the lane according to the control command.

[0006] Based on the above, the device and method for controlling a vehicle of the present invention can use the information obtained by the lidar, the camera and the wheel speed sensor to obtain the steering angle and the steering compensation angle, and then obtain the control command to control the vehicle to drive in the lane. In this way, the vehicle can be controlled to keep the vehicle stably in the center of the lane, thereby reducing the risk of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a schematic diagram of a device for controlling a vehicle according to an embodiment of the present invention.

[0008] Figure 2 is a flow chart of a method for controlling a vehicle according to an embodiment of the present invention.

[0009] Figure 3 yes Figure 2 Further explanation of .

[0010] Figure 4 is a schematic diagram of normal distribution transformation according to an embodiment of the present invention.

[0011] Figure 5 FIG. 4 is a schematic diagram of global path planning and local path planning according to an embodiment of the present invention.

[0012] Figure 6 is a schematic diagram of a pure tracking algorithm according to an embodiment of the present invention.

[0013] Figure 7 FIG. 4 is a schematic diagram of a lane detection operation according to an embodiment of the present invention.

[0014] Figure 8 FIG. 4 is a schematic diagram of a steering compensation angle calculation operation according to an embodiment of the present invention.

[0015] Fig. 9 FIG. 4 is a schematic diagram of obtaining a vehicle control command according to an embodiment of the present invention.

[0016] Fig.10 FIG. 4 is a schematic diagram of an execution accuracy evaluation operation according to an embodiment of the present invention.

[0017] Description of Reference Numerals

[0018] 100: Device for controlling a vehicle;

[0019] 110: Guangda;

[0020] 120: Camera;

[0021] 130: wheel speed sensor;

[0022] 140: processor;

[0023] 150: storage medium;

[0024] S210, S220, S230, S240, S221, S222, S223, S224, S231, S232, S210-1: steps;

[0025] Id: Look_ahead distance;

[0026] Δd: Lane centerline deviation error;

[0027] Reduce the coefficient. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0029] Figure 1 FIG. 1 is a schematic diagram of a device 100 for controlling a vehicle according to an embodiment of the present invention. Figure 1 The device 100 may include a LiDAR 110, a camera 120, a wheel speed sensor 130, and a processor 140. The processor 140 is coupled to the LiDAR 110, the camera 120, and the wheel speed sensor 130. In other embodiments, the device 100 may further include a storage medium 150 coupled to the processor 140. The device 100 may be disposed in a vehicle (not shown).

[0030] The processor 140 may include a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar components or combinations thereof. The processor 140 may access and execute multiple modules and various applications stored in the storage medium 150.

[0031] The storage medium 150 may include any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD) or similar components or a combination of the above components, and is used to store multiple modules or various applications that can be executed by the processor 140.

[0032] Figure 2 is a flow chart of a method for controlling a vehicle according to an embodiment of the present invention. Figure 3 yes Figure 2 Please also refer to Figure 1 , Figure 2 and Figure 3 .

[0033] In step S210 , the processor 140 may obtain point cloud data of the vehicle through the lidar 110 , obtain an RGB image of the vehicle through the camera 120 , and obtain the current speed of the vehicle through the wheel speed sensor 130 .

[0034] In step S220, the processor 140 may use the current speed and the local path way points associated with the point cloud data to obtain the steering angle (target angle). In one embodiment, the storage medium 150 may store a high definition map (HD map), a vector map, a starting point of the vehicle, and a target point of the vehicle. The HD map is, for example, a point cloud map of a scene. Further, the HD map may be in PCD format. On the other hand, the vector map may include definitions of road types, lane locations, intersections, and traffic signs / traffic signs. It should be noted here that Figure 2 Step S220 may include Figure 3 The steps S221, S222, S223 and S224 will be described below.

[0035] Please continue to refer to Figure 3 In step S221, the processor 140 may perform pose estimation on the point cloud data, the HD map, and the starting point to obtain the current pose of the vehicle. The current pose may include the x-coordinate, y-coordinate, and yaw angle of the vehicle, but the present invention is not limited thereto. In one embodiment, the pose estimation may include a Normal Distributions Transform (NDT).

[0036] Figure 4 is a schematic diagram of a normal distribution transformation according to an embodiment of the present invention. Figure 1 , Figure 2 , Figure 3 and Figure 4 In this embodiment, the processor 140 can divide the target point cloud into 3D grids (Three Dimensions Grid) according to a fixed resolution, and can calculate the mean and covariance matrix (Covariance Matrix) of the target point cloud in each 3D grid. Then, the processor 140 can calculate the probability of the source point cloud (Source Point Cloud) being distributed in the target point cloud as a matching relationship. By matching the features between the target point cloud and the source point cloud, the processor 140 can locate the position of the vehicle and detect the movement of the vehicle.

[0037] In other embodiments, the above-mentioned posture estimation may also include an iterative closest point algorithm (Iterative Closest Point, ICP), a point cloud matching based on deep learning, a LiDAR Odometry and Mapping algorithm (LOAM) and a fast point feature histogram (FPFH), but the present invention is not limited thereto. Specifically, the Iterative Closest Point can find the best match between the target point cloud and the source point cloud by minimizing the average distance between the target point cloud and the source point cloud. On the other hand, the point cloud matching based on deep learning can learn the features between point clouds through a deep learning model, thereby achieving more accurate matching.

[0038] Please go back Figure 3. In step S222, the processor 140 may perform global path planning on the vector map, the current posture and the target point to obtain the global path (global path way points) of the vehicle. In one embodiment, the global path planning may include trajectory planning (Trajectory Planning). Specifically, Trajectory Planning may use the current posture and the vector map (including the route and speed limit, etc.) to map the current posture to the corresponding route, thereby finding the shortest global path. Then, in step S223, the processor 140 may perform local path planning on the current posture and the global path to obtain the local path of the vehicle. In one embodiment, the local path planning may include a roll-out generation algorithm (Roll-Out Generation). Specifically, Roll-Out Generation may move a fixed distance vertically from the global path, and perform a fixed number of steps of trajectory sampling at each position, and may select the most representative trajectory through a Bayesian optimization strategy, thereby improving the effectiveness and accuracy of the local path. It is worth noting that Trajectory Planning and Roll-Out Generation are both methods of the Open Planner motion planning algorithm.

[0039] Figure 5 is a schematic diagram of global path planning and local path planning according to an embodiment of the present invention. Figure 1 , Figure 2 , Figure 3 and Figure 5 In this embodiment, when the processor 140 performs global path planning, the processor 140 may not consider traffic signs, and may only consider the starting point of the vehicle and the destination point of the vehicle. On the other hand, when the processor 140 performs local path planning, the processor 140 may consider the current environmental state of the vehicle, including but not limited to the lane where the vehicle is currently located, whether there are other vehicles in the next lane, and / or whether the traffic sign is currently red.

[0040] In other embodiments, the global path planning may include a mission planning algorithm (Mission Planner, Route Planner), an A* algorithm, a D* algorithm, machine learning, and a neural network. On the other hand, the local path planning may include a motion trajectory planning algorithm (Motion Planner), an A* algorithm, a dynamic window algorithm (Dynamic Window Approach, DWA), a rapidly exploring random tree algorithm (Rapidly-exploring Random Trees, RRT), a linear quadratic regulation algorithm (Linear Quadratic Regulation, LQR) and a model predictive control algorithm (Model Predictive Control, MPC), but the present invention is not limited thereto.

[0041] Please go back Figure 3 In step S224, the processor 140 may perform a steering angle calculation operation on the current speed and the local path to obtain the steering angle, wherein the steering angle calculation operation may include a pure pursuit algorithm (Pure Pursuit). Specifically, Pure Pursuit is a path pursuit algorithm based on the distance and angle difference between the vehicle and the target point, thereby calculating the direction and speed of the vehicle.

[0042] Figure 6 is a schematic diagram of a pure tracking algorithm according to an embodiment of the present invention. Figure 1 , Figure 2 , Figure 3 and Figure 6 In this embodiment, if the value of the Look_ahead distance (Id) parameter is set to a too large value, the vehicle is likely to encounter a problem of taking a shortcut (Cut Corner). On the other hand, if the Look_ahead distance (Id) parameter is set to a too small value, the vehicle is likely to encounter a problem of oscillation. After setting the Look_ahead distance to an appropriate value, the processor 140 may Figure 6 The steering angle (θ t ).

[0043] In other embodiments, the steering angle calculation operation may further include model predictive control (MPC), fuzzy logic control (FLC), proportional integral derivative control (PID control) and artificial neural network (ANN), but the present invention is not limited thereto.

[0044] Please go back Figure 2In step S230, the processor 140 may use the current speed and the central lane distance error associated with the RGB image to obtain a steering compensation angle. Figure 2 Step S230 may include Figure 3 Step S231 and step S232.

[0045] Please continue to refer to Figure 3 In step S231, the processor 140 may perform a lane detection operation on the RGB image to obtain a lane centerline deviation error, wherein the lane detection operation may include a YOLO target detection algorithm (You Only Look Once, YOLO).

[0046] Figure 7 is a schematic diagram of a lane line detection operation according to an embodiment of the present invention. Figure 1 , Figure 2 , Figure 3 and Figure 7 In this embodiment, the processor 140 may perform YOLO on the RGB image to detect the object in the RGB image. Then, the processor 140 may frame the object and mark the neural network of the specific object category, and may transform the neural network into a neural network capable of achieving multi-task applications. Based on this, the processor 140 may perform feature matching on the RGB image to simultaneously perform lane detection and object detection to calculate Figure 7 Lane centerline deviation error (Δd) is shown.

[0047] In other embodiments, the lane detection operation may include edge detection and semantic segmentation, but the present invention is not limited thereto. Specifically, edge detection may be a machine vision algorithm. For example, the processor 140 may use an edge detection algorithm (Canny algorithm) to detect lane lines in an RGB image, and then use a Hough Transform to convert the detected edges into straight lines, thereby identifying the lane lines in the RGB image. On the other hand, semantic segmentation may divide the RGB image into different regions to implement lane detection operations. Semantic segmentation is, for example, Mask R-CNN and U-Net.

[0048] Please go back Figure 3 In step S232, the processor 140 may perform a steering compensation angle calculation operation on the current speed and the lane centerline deviation error to obtain a steering compensation angle, wherein the steering compensation angle calculation operation may include proportional integral derivative control (PID control).

[0049] Figure 8 is a schematic diagram of a steering compensation angle calculation operation according to an embodiment of the present invention. Figure 1 , Figure 2 , Figure 3 and Figure 8 In this embodiment, the processor 140 can obtain a steering compensation angle for resisting deviation from the path through the lane centerline deviation error (Δd) and the current speed of the vehicle, thereby correcting the inaccurate positioning caused by factors such as environmental changes. Specifically, the angle integrator in the PID controller can be used to adjust the steering compensation angle of the vehicle. Furthermore, since the vehicle is prone to oscillation when driving at high speed, the processor 140 can generate a reduction coefficient according to the current speed of the vehicle. To prevent the vehicle from vibrating when turning. Figure 8 As shown, when the processor 140 obtains the lane centerline deviation error (Δd) and the current speed of the vehicle (VehicleSpeed) is greater than the speed threshold value (MinSpeedLimit), the processor 140 can use the lane centerline deviation error (Δd) to update the angle integrator value (IntegralVal). On the other hand, when the processor 140 does not obtain the lane centerline deviation error (Δd) or the current speed of the vehicle is less than the speed threshold value, the processor 140 can use the reduction coefficient To adjust the angle integrator value. Specifically, reduce the coefficient It can be a value less than 1 to slowly return the angle integrator value to zero. Finally, the processor 140 can use the angle integrator value to obtain the steering compensation angle (θ c ).

[0050] In other embodiments, the steering compensation angle calculation operation may include a model predictive control algorithm (MPC), a fuzzy logic control (FLC), and an artificial neural network (ANN), but the present invention is not limited thereto.

[0051] Please go back Figure 2 In step S240, the processor 140 may use the steering angle and the steering compensation angle to obtain a steering command of the vehicle, and may control the vehicle to travel in the lane according to the steering command.

[0052] Fig. 9 FIG. 1 is a schematic diagram of obtaining a vehicle control command according to an embodiment of the present invention. Figure 1 , Figure 2 , Figure 3 and Fig. 9. In the present embodiment, the processor 140 may fuse the steering angle and the steering compensation angle to improve the accuracy of lane keeping and the accuracy of motion control. Specifically, the pure tracking algorithm controller may control the instantaneous deviation of the vehicle but cannot solve the steady-state error of the vehicle. Therefore, the processor 140 may use the angle integrator in the PID controller to correct the steady-state error of the vehicle. Based on this, the device 100 of the present invention may solve the problems of instantaneous deviation and steady-state error at the same time. Furthermore, in the present embodiment, the steering command may include a speed steering command and an angular steering command.

[0053] In other embodiments, in order to evaluate the accuracy of the device 100 in manipulating the vehicle, the processor 140 may perform an accuracy evaluation operation offline, as will be described below.

[0054] Fig.10 is a schematic diagram of an execution accuracy evaluation operation according to an embodiment of the present invention. Figure 1 , Figure 3 and Fig.10 . In step S210-1, the processor 140 may obtain the vehicle's AVM (Around View Monitor) RGB image through the camera 120. Then, the processor 140 may use the surrounding RGB image to perform an accuracy evaluation operation. In one embodiment, the accuracy evaluation operation may include an accuracy root mean square error (RMSE) evaluation operation. Specifically, the processor 140 may perform image space scale correction, edge detection, and Hough transform on the surrounding RGB image to extract lane lines in the surrounding RGB image. Based on this, the processor 140 may obtain the accuracy of controlling the vehicle (i.e., the distance between the center of the vehicle and the center of the lane). Further, the processor 140 may obtain the accuracy of each point on the path of the vehicle to calculate the overall accuracy of controlling the vehicle using the root mean square error. Formula (1) is an example.

[0055]

[0056] where e rms is the overall accuracy of controlling the vehicle, e1 is the accuracy of the first point on the vehicle's path, e2 is the accuracy of the second point on the vehicle's path, and so on until e N is the accuracy of the Nth point on the vehicle’s path.

[0057] In summary, the device and method for controlling a vehicle of the present invention can use the information obtained by the lidar, the camera and the wheel speed sensor to obtain the steering angle and the steering compensation angle, and then obtain the control command to control the vehicle to drive in the lane. In this way, the vehicle can be controlled to keep the vehicle stably in the center of the lane, thereby reducing the risk of accidents.

[0058] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A device for controlling a vehicle, characterized in that: include: Guangda; camera; Wheel speed sensor; as well as A processor is coupled to the lidar, the camera and the wheel speed sensor, wherein The processor obtains point cloud data of the vehicle through the lidar, obtains an RGB image of the vehicle through the camera, and obtains a current speed of the vehicle through the wheel speed sensor; The processor obtains a steering angle using the current speed and a local path associated with the point cloud data; The processor obtains a steering compensation angle using the current speed and a lane centerline deviation error associated with the RGB image; The processor obtains a steering command of the vehicle using the steering angle and the steering compensation angle, and controls the vehicle to travel in a lane according to the steering command.

2. The device according to claim 1, characterized in that Also included is a storage medium coupled to the processor, wherein the storage medium stores a high-definition map, a vector map, a starting point of the vehicle, and a target point of the vehicle, wherein The processor performs posture estimation on the point cloud data, the high-definition map, and the starting point to obtain a current posture of the vehicle; The processor performs global path planning on the vector map, the current posture, and the target point to obtain a global path of the vehicle; The processor performs local path planning on the current posture and the global path to obtain the local path of the vehicle.

3. The device according to claim 2, characterized in that The pose estimation includes a normal distribution transformation.

4. The device according to claim 2, characterized in that The global path planning includes trajectory planning.

5. The device according to claim 2, characterized in that The local path planning includes a roll-out generation algorithm.

6. The device according to claim 1, characterized in that The processor performs a steering angle calculation operation on the current speed and the local path to obtain the steering angle, wherein the steering angle calculation operation includes a pure tracking algorithm.

7. The device according to claim 1, characterized in that The processor performs a lane line detection operation on the RGB image to obtain the lane center line deviation error, wherein the lane line detection operation includes a YOLO target detection algorithm.

8. The device according to claim 1, characterized in that The processor performs a steering compensation angle calculation operation on the current speed and the lane centerline deviation error to obtain the steering compensation angle, wherein the steering compensation angle calculation operation includes proportional integral derivative control.

9. The device according to claim 1, characterized in that The processor obtains a surrounding RGB image of the vehicle through the camera; The processor uses the surrounding RGB image to perform an accuracy evaluation operation.

10. The device according to claim 9, characterized in that The accuracy evaluation operation includes an accuracy root mean square error evaluation operation.

11. A method for controlling a vehicle, suitable for being executed by a device comprising a lidar, a camera, and a wheel speed sensor, characterized in that: The method comprises the following steps: Obtaining point cloud data of the vehicle through the lidar, obtaining an RGB image of the vehicle through the camera, and obtaining a current speed of the vehicle through the wheel speed sensor; Obtaining a steering angle using the current speed and a local path associated with the point cloud data; Obtaining a steering compensation angle using the current speed and a lane centerline deviation error associated with the RGB image; and The steering angle and the steering compensation angle are used to obtain a steering command of the vehicle, and the vehicle is controlled to travel in a lane according to the steering command.

12. The method according to claim 11, characterized in that The step of obtaining a steering angle using the current speed and a local path associated with the point cloud data comprises: Performing posture estimation on the point cloud data, the high-definition map, and the starting point of the vehicle to obtain a current posture of the vehicle; Performing global path planning on the vector map, the current posture, and the target point of the vehicle to obtain a global path of the vehicle; and Local path planning is performed on the current posture and the global path to obtain the local path of the vehicle.

13. The method according to claim 12, characterized in that The pose estimation includes a normal distribution transformation.

14. The method according to claim 12, characterized in that The global path planning includes trajectory planning.

15. The method according to claim 12, characterized in that The local path planning includes a roll-out generation algorithm.

16. The method according to claim 11, characterized in that The step of obtaining a steering angle using the current speed and a local path associated with the point cloud data comprises: A steering angle calculation operation is performed on the current speed and the local path to obtain the steering angle, wherein the steering angle calculation operation includes a pure tracking algorithm.

17. The method according to claim 11, characterized in that The step of obtaining the steering compensation angle by using the current speed and the lane centerline deviation error associated with the RGB image comprises: A lane line detection operation is performed on the RGB image to obtain the lane center line deviation error, wherein the lane line detection operation includes a YOLO target detection algorithm.

18. The method according to claim 11, characterized in that The step of obtaining the steering compensation angle by using the current speed and the lane centerline deviation error associated with the RGB image comprises: A steering compensation angle calculation operation is performed on the current speed and the lane centerline deviation error to obtain the steering compensation angle, wherein the steering compensation angle calculation operation includes proportional integral derivative control.

19. The method according to claim 11, characterized in that Also includes: Obtaining a surround RGB image of the vehicle through the camera; and The surrounding RGB image is used to perform an accuracy evaluation operation.

20. The method according to claim 19, characterized in that The accuracy evaluation operation includes an accuracy root mean square error evaluation operation.

Images