Apparatus and method for steering a vehicle

Abstract

To provide a device and method for steering a vehicle.SOLUTION: A method includes the following steps: acquiring point group data on a vehicle by LiDAR; acquiring an RGB image of the vehicle by a camera; and acquiring the current speed of the vehicle by a wheel speed sensor. Local course waypoints related to the current speed of the vehicle and the point group data are used so as to acquire a target angle. The current speed of the vehicle and a roadway center line distance error are used to acquire a compensation angle. The target angle and the compensation angle are used to acquire a steering command for the vehicle, which is steered to travel within the roadway according to the steering command.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an apparatus and method for steering a vehicle. [Background technology]

[0002] Currently, technologies for controlling vehicles (e.g., unmanned vehicles) are constantly evolving. However, existing technologies are susceptible to weather and complex lane conditions, resulting in poor vehicle control. Summary of the Invention [Problem to be solved by the invention]

[0003] SUMMARY OF THE INVENTION The present invention is directed to an apparatus and method for steering a vehicle that can steer the vehicle to keep it stable and centered on the roadway. [Means for solving the problem]

[0004] The present invention provides an apparatus for steering a vehicle, including a LiDAR, a camera, a wheel speed sensor, and a processor. The processor is coupled to the LiDAR, the camera, and the wheel speed sensor, and the processor acquires point cloud data of the vehicle using the LiDAR, acquires an RGB image of the vehicle using the camera, and acquires a current speed of the vehicle using the wheel speed sensor. The processor uses the current speed and local path waypoints associated with the point cloud data to obtain a target angle. The processor uses the current speed and a roadway centerline distance error associated with the RGB image to obtain a compensation angle. The processor uses the target angle and the compensation angle to obtain a steering command for the vehicle, and steers the vehicle to travel within the roadway in accordance with the steering command.

[0005] The present invention provides a method for steering a vehicle, including: acquiring point cloud data of the vehicle by LiDAR; acquiring an RGB image of the vehicle by a camera; acquiring a current vehicle speed by wheel speed sensors; using the current speed of the vehicle and local path waypoints associated with the point cloud data to obtain a target angle; using the current speed of the vehicle and a roadway centerline distance error to obtain a compensation angle; the target angle and the compensation angle are used to obtain a steering command for the vehicle; and the vehicle is steered to travel within the roadway according to the steering command. [Effects of the Invention]

[0006] Based on the above, the device and method for steering a vehicle of the present invention can use information obtained by LiDAR, camera, and wheel speed sensors to obtain a target angle and a compensation angle, and then obtain a steering command to steer the vehicle to travel within the roadway. In this way, the vehicle can be stably steered and maintained in the center of the roadway, thereby reducing the risk of an accident.

[0007] To explain the present invention in more detail, several exemplary embodiments are described in detail below in conjunction with the drawings. [Brief explanation of the drawings]

[0008] The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, the drawings illustrating embodiments of the invention and together with the description serve to explain the principles of the invention.

[0009] [Figure 1] 1 is a schematic diagram of an apparatus for steering a vehicle according to one embodiment of the present invention; [Figure 2] FIG. 1 is a flow diagram of a method for steering a vehicle according to one embodiment of the present invention. [Figure 3] Further explanation of FIG. [Figure 4] FIG. 1 is a schematic diagram of normal distribution according to one embodiment of the present invention. [Figure 5]FIG. 1 is a schematic diagram of global and local path planning according to one embodiment of the present invention. [Figure 6] FIG. 1 is a schematic diagram of Pure Pursuit according to one embodiment of the present invention. [Figure 7] FIG. 2 is a schematic diagram of a road marking detection operation according to one embodiment of the present invention. [Figure 8] FIG. 1 is a schematic diagram of a compensation angle calculation operation according to one embodiment of the present invention. [Figure 9] FIG. 1 is a schematic diagram of obtaining a steering command for a vehicle according to one embodiment of the present invention. [Figure 10] FIG. 1 is a schematic diagram illustrating the performance of accuracy assessment according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] 1 is a schematic diagram of an apparatus 100 for steering a vehicle according to one embodiment of the present invention. Referring to FIG. 1, the apparatus 100 includes a LiDAR 110, a camera 120, a wheel speed sensor 130, and a processor 140. The processor 140 is coupled to the LiDAR 110, the camera 120, and the wheel speed sensor 130. In other embodiments, the apparatus 100 may further include a storage medium 150 coupled to the processor 140. The apparatus 100 may be mounted on a vehicle (not shown).

[0011] The processor 140 may include a central processing unit (CPU) or other programmable general-purpose or application-specific microcontroller unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image acquisition and processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPID), field programmable gate array (FPGA), or other similar component, or a combination of the above components. The processor 140 may access and execute multiple modules and various applications stored in the storage medium 150.

[0012] The storage medium 150 may include any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar component, or a combination of the above components, and is used to store multiple modules and various applications that can be executed by the processor 140.

[0013] Figure 2 is a flow diagram of a method for steering a vehicle according to one embodiment of the present invention, and Figure 3 is a further explanation of Figure 2. Please refer to Figures 1, 2 and 3 simultaneously.

[0014] In step S210, the processor 140 may acquire point cloud data of the vehicle using the LiDAR 110, may acquire an RGB image of the vehicle using the camera 120, and may acquire the current speed of the vehicle using the wheel speed sensor 130.

[0015] In step S220, processor 140 may obtain a target angle by using the current speed and local route waypoints associated with point cloud data. In one embodiment, storage medium 150 may store a high-definition (HD) map, a vector map, a vehicle start point, and a vehicle target point. The HD map is, for example, a point cloud map of a scene. Furthermore, the HD map may be in PCD format. Meanwhile, the vector map includes definitions of road shapes, roadway positions, intersections, and road markings / signals. Note that step S220 in FIG. 2 here includes steps S221, S222, S223, and S224 in FIG. 3, which will be further described below.

[0016] 3, in step S221, the processor 140 may perform pose estimation on the point cloud data, the HD map, and the starting point to obtain a current pose of the vehicle. The current pose may include the x-axis, y-axis, and yaw angle of the vehicle, but the present invention is not limited thereto. In one embodiment, the pose estimation may include normal distribution fitting (NDT).

[0017] FIG. 4 is a schematic diagram of normalization according to one embodiment of the present invention. Referring simultaneously to FIGS. 1, 2, 3, and 4, in this embodiment, the processor 140 may divide the target point cloud into 3D grids according to a fixed resolution and calculate the mean value and covariance matrix of the target point cloud in each 3D grid. The processor 140 may then calculate the probability of the distribution of the source point cloud in the target point cloud as a matching relationship. In other words, by matching features between the target point cloud and the source point cloud, the processor 140 can locate the vehicle and detect vehicle movement.

[0018] In other embodiments, the pose estimation may include Iterative Closest Point (ICP), deep learning-based point cloud matching, LiDAR Odometry and Mapping (LOAM), and Fast Point Feature Histograms (FPFH). However, the present invention is not limited thereto. Specifically, ICP can find the best match between the target point cloud and the source point cloud by minimizing the average distance between the target point cloud and the source point cloud. Meanwhile, deep learning-based point cloud matching can learn features between point clouds using a deep learning model to achieve more accurate matching.

[0019] Returning to FIG. 3 , in step S222, processor 140 may perform global path planning on the vector map, current pose, and target points to obtain global path waypoints for the vehicle. In one embodiment, global path planning may include trajectory planning. Specifically, trajectory planning may find the shortest global path waypoint by using the current pose and the vector map (including route, speed limits, etc.) to map the current pose to a corresponding path. Then, in step S223, processor 140 may perform local path planning on the current pose and global path waypoints to obtain local path waypoints for the vehicle. In one embodiment, local path planning may include rollout generation. Specifically, rollout generation may involve moving a fixed distance vertically from the global path and performing a fixed number of steps of trajectory sampling at each position, and selecting the most representative trajectory using a Bayesian optimization method to improve the validity and accuracy of the local path waypoints. Note that both trajectory planning and rollout generation are Open Planner methods.

[0020] 5 is a schematic diagram of global path planning and local path planning according to one embodiment of the present invention. Referring simultaneously to FIGS. 1, 2, 3, and 5, in this embodiment, when processor 140 performs global path planning, processor 140 may not consider traffic signals and may only consider the vehicle's starting point and destination point. On the other hand, when processor 140 performs local path planning, processor 140 may consider the vehicle's current environmental conditions, including, but not limited to, the roadway on which the vehicle is currently located, whether there are other vehicles on adjacent roadways, and / or whether the traffic signal is currently red.

[0021] In other embodiments, the global path planning may include Mission Planner, Route Planner, A*, D*, machine learning, and neural networks, while the local path planning may include Motion Planner, A*, Dynamic Window Approach (DWA), Rapidly-exploring Random Trees (RRT), Linear Quadratic Regulation (LQR), and Model Predictive Control (MPC), but the present invention is not limited thereto.

[0022] 3, in step S224, processor 140 may perform a target angle calculation operation on the current speed and the local route waypoint to obtain a target angle, which may include Pure Pursuit. Specifically, Pure Pursuit is a path tracking algorithm based on the distance and angle difference between the vehicle and the target point, thereby calculating the vehicle's moving direction and speed.

[0023] 6 is a schematic diagram of Pure Pursuit according to one embodiment of the present invention. Referring to FIGS. 1, 2, 3, and 6 simultaneously, in this embodiment, the parameter Look_ahead distance (I dIf the value of Look_ahead distance(I) is set to an excessively large value, it is likely to encounter corner cutting problems. d If Look_ahead distance is set to an excessively large value, the vehicle is likely to encounter vibration problems. After setting Look_ahead distance to an appropriate value, processor 140 calculates the target angle (θ t ) can be obtained.

[0024] In other embodiments, the target angle calculation operations described above may also include model predictive control (MPC), fuzzy logic control (FLC), proportional-integral-derivative control (PID control), and artificial intelligence networks (ANN), although the present invention is not limited thereto.

[0025] 2, in step S230, processor 140 may obtain a compensation angle by using the current speed and the roadway centerline distance error associated with the RGB image. Note that step S230 in FIG. 2 may include steps S231 and S232 in FIG. 3.

[0026] Referring to FIG. 3, in step S231, the processor 140 may perform a road marking detection operation on the RGB image to obtain a roadway centerline distance error, and the road marking detection operation may include You Only Look Once (YOLO).

[0027] FIG. 7 is a schematic diagram of a road marking detection operation according to one embodiment of the present invention. Referring simultaneously to FIGS. 1, 2, 3, and 7, processor 140 may perform YOLO on an RGB image to detect objects in the RGB image. Processor 140 may then frame and label the objects with a neural network of a specific object category and convert the neural network into a neural network capable of achieving multitasking applications. In this manner, processor 140 may perform feature mapping on the RGB image to simultaneously perform road marking detection and object detection to calculate the roadway centerline distance error (Δd), as shown in FIG. 7.

[0028] In another embodiment, the road marking detection operation described above may include edge detection and semantic segmentation, but the present invention is not limited thereto. In particular, edge detection may be a machine vision algorithm. For example, the processor 140 may use an edge detection algorithm (Canny algorithm) to detect road markings in the RGB image, and then use a Hough transform to convert the detected edges into straight lines, thereby identifying the road markings in the RGB image. Semantic segmentation may divide the RGB image into different regions to perform the road marking detection operation. Examples of semantic segmentation include Mask R-CNN and U-Net.

[0029] Referring again to FIG. 3, in step S232, processor 140 may perform a compensation angle calculation operation on the current speed and roadway centerline distance error to obtain a compensation angle, which may include proportional-integral-derivative control (PID control).

[0030] FIG. 8 is a schematic diagram of a compensation angle calculation operation according to one embodiment of the present invention. Referring simultaneously to FIGS. 1, 2, 3, and 8, in this embodiment, processor 140 may obtain a compensation angle used to resist deviation from the path based on the roadway centerline distance error (Δd) and the vehicle's current speed, thereby compensating for position accuracy variations due to environmental changes or other factors. Specifically, an angle integrator in a PID controller may be used to adjust the vehicle's compensation angle. Furthermore, because a vehicle is likely to encounter vibration problems when traveling at high speeds, processor 140 may generate a reduction coefficient (∂) according to the vehicle's current speed to prevent the vehicle from encountering vibration problems when turning. As shown in FIG. 8, when processor 140 obtains the roadway centerline distance error (Δd) and the vehicle's current speed (VehicleSpeed) is greater than a speed threshold (MinSpeedLimit), processor 140 may use the roadway centerline distance error (Δd) to update the angle integral value (IntegralVal). On the other hand, when the processor 140 does not obtain the roadway centerline distance error (Δd) or the current speed of the vehicle is less than the speed threshold, the processor 140 may adjust the angle integral value by using a reduction factor (∂). In particular, the reduction factor (∂) may be a value less than 1 that slowly reduces the angle integral value to zero. Finally, the processor 140 calculates the compensation angle (θ c ) can be obtained by using the angular integral.

[0031] In other embodiments, the compensation angle calculation operation may include, but is not limited to, model predictive control (MPC), fuzzy logic control (FLC), and artificial intelligence network (ANN).

[0032] Referring again to FIG. 2, in step S240, processor 140 may use the target angle and compensation angle to obtain a steering command for the vehicle, and may control the vehicle to drive within the roadway according to the steering command.

[0033] FIG. 9 is a schematic diagram of obtaining a vehicle steering command according to one embodiment of the present invention. Referring simultaneously to FIGS. 1, 2, 3, and 9, in this embodiment, the processor 140 may combine the target angle and the compensation angle, thereby improving the accuracy of roadway keeping and movement control. Specifically, the Pure Pursuit controller can control the vehicle's instantaneous deviation but cannot resolve the vehicle's steady-state error. Therefore, the processor 140 may use an angle integrator in the PID controller to correct the vehicle's steady-state error. In this way, the device 100 of the present invention can simultaneously resolve the issues of instantaneous deviation and steady-state error. Furthermore, in this embodiment, the steering command may include a speed operation command and an angle steering command.

[0034] In other embodiments, to assess the accuracy of the device 100 in steering the vehicle, the processor 140 may perform the accuracy assessment operations described below offline.

[0035] FIG. 10 is a schematic diagram illustrating the execution of accuracy assessment according to one embodiment of the present invention. Referring simultaneously to FIGS. 1, 3, and 10, in step S210-1, processor 140 may acquire an around-view monitor (AVM) RGB image of the vehicle by camera 120. Processor 140 may then use the AVM RGB image to perform an accuracy assessment operation. In one embodiment, the accuracy assessment operation may include an accuracy root mean square error (RMSE) assessment operation. In particular, processor 140 may perform image space scale correction, edge detection, and a Hough transform on the AVM RGB image to extract road markings in the AVM RGB image. In this manner, processor 140 may obtain the vehicle steering accuracy (i.e., the distance between the vehicle center and the roadway center). Furthermore, processor 140 may obtain the accuracy of each point on the vehicle's path to calculate the overall vehicle steering accuracy by using the root mean square error. Equation 1 below is an example.

[0036]

number

[0037] e rms is the overall vehicle steering accuracy, e1 is the accuracy at the first point on the path, e2 is the accuracy at the second point on the path, and similarly e N is the accuracy at the Nth point on the vehicle's path.

[0038] In summary, the device and method for steering a vehicle of the present invention can use information obtained by LiDAR, cameras, and wheel speed sensors to obtain steering commands for steering the vehicle to travel within the roadway, and then obtain steering commands for steering the vehicle to travel within the roadway. In this way, the vehicle can be stably steered and maintained in the center of the roadway, thereby reducing the risk of an accident.

[0039] It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the disclosed embodiments without departing from the scope or spirit of the invention. In view of the foregoing, it is intended that the present invention cover modifications and variations of this invention provided they come within the scope of the following claims and their equivalents. [Industrial Applicability]

[0040] The device and method for steering a vehicle of the present invention can be applied to keep the vehicle stable in the center of the roadway, thereby reducing the risk of an accident. [Explanation of symbols]

[0041] 100: Device for steering a vehicle 110: LiDAR 120: Camera 130: Wheel speed sensor 140: Processor 150:Storage medium S210, S220, S230, S240, S221, S222, S223, S224, S231, S232, S210-1: Step I d :Look_ahead distance Δd: Roadway center line distance error ∂: Reduction coefficient

Claims

1. 1. A device for steering a vehicle, comprising: LiDAR and A camera and A wheel speed sensor; a processor coupled to the LiDAR, the camera, and the wheel speed sensor; Including, The processor acquires point cloud data of the vehicle using the LiDAR, acquires an RGB image of the vehicle using the camera, and acquires a current speed of the vehicle using the wheel speed sensor; the processor obtains a target angle using the current velocity and local path waypoints associated with the point cloud data; the processor obtains a compensation angle using the current speed and a roadway centerline distance error associated with the RGB image; the processor obtains a steering command for the vehicle using the target angle and the compensation angle, and steers the vehicle to travel within a roadway in accordance with the steering command; the processor performs a target angle calculation operation on the current velocity and the local path waypoint to obtain the target angle; The target angle calculation operation includes a Pure Pursuit. A device for steering a vehicle.

2. a storage medium coupled to the processor Further comprising: the storage medium stores an HD map, a vector map, a starting point of the vehicle, and a destination point of the vehicle; the processor performs pose estimation on the point cloud data, the HD map, and the starting point to obtain a current pose of the vehicle; the processor performs global path planning on the vector map, the current pose, and the target points to obtain global path waypoints for the vehicle; the processor performs local path planning on the current pose and the global path waypoints to obtain local path waypoints for the vehicle.

10. An apparatus for steering a vehicle according to claim 1.

3. The pose estimation includes normal distribution tuning (NDT).

3. An apparatus for steering a vehicle according to claim 2.

4. The global path planning includes a trajectory planning.

3. An apparatus for steering a vehicle according to claim 2.

5. The local path planning includes rollout generation.

3. An apparatus for steering a vehicle according to claim 2.

6. the processor performs a road marking detection operation on the RGB image to obtain a roadway centerline distance error; The road marking detection operation includes You Only Look Once (YOLO), 10. An apparatus for steering a vehicle according to claim 1.

7. the processor performs a compensation angle calculation operation on the current speed and the roadway centerline distance error to obtain the compensation angle; The compensation angle calculation operation includes proportional-integral-derivative control (PID control).

10. An apparatus for steering a vehicle according to claim 1.

8. the processor acquires an around view monitor (AVM) RGB image of the vehicle with the camera; the processor performs an accuracy assessment operation using the AVM RGB image; 10. An apparatus for steering a vehicle according to claim 1.

9. the accuracy assessment operation includes an accuracy root mean square error (RMSE) assessment operation; 9. An apparatus for steering a vehicle according to claim 8.

10. 1. A method for steering a vehicle fitted with an apparatus including a LiDAR, a camera, and wheel speed sensors, comprising: Acquiring point cloud data of the vehicle using the LiDAR, acquiring an RGB image of the vehicle using the camera, and acquiring a current speed of the vehicle using the wheel speed sensor; obtaining a target angle using the current velocity and local path waypoints associated with the point cloud data; obtaining a compensation angle using the current speed and a roadway centerline distance error associated with the RGB image; obtaining a steering command for the vehicle using the target angle and the compensation angle, and steering the vehicle so as to travel within a roadway in accordance with the steering command; Including, Obtaining the target angle using the current velocity and the local path waypoints associated with the point cloud data includes: performing a target angle calculation operation on the current velocity and the local path waypoint to obtain the target angle, wherein the target angle calculation operation includes a Pure Pursuit. Including, A method for steering a vehicle.

11. Obtaining the target angle using the current velocity and the local path waypoints associated with the point cloud data includes: performing pose estimation on the point cloud data, the HD map, and the starting point of the vehicle to obtain a current pose of the vehicle; performing global path planning on a vector map, the current pose, and a target point for the vehicle to obtain global path waypoints for the vehicle; performing local path planning on the current attitude and the global path waypoints to obtain the local path waypoints for the vehicle; Including, 11. A method for steering a vehicle according to claim 10.

12. The pose estimation includes normal distribution tuning (NDT).

12. A method for steering a vehicle according to claim 11.

13. The global path planning includes a trajectory planning.

12. A method for steering a vehicle according to claim 11.

14. The local path planning includes rollout generation.

12. A method for steering a vehicle according to claim 11.

15. Obtaining the compensation angle using the current speed and the roadway centerline distance error associated with the RGB image includes: performing a road marking line detection operation on the RGB image to obtain the roadway centerline distance error, the road marking line detection operation including YOLO; Including, 11. A method for steering a vehicle according to claim 10.

16. Obtaining the compensation angle using the current speed and the roadway centerline distance error associated with the RGB image includes: performing a compensation angle calculation operation on the current speed and the roadway centerline distance error to obtain the compensation angle, the compensation angle calculation operation including proportional-integral-derivative control (PID control); Including, 11. A method for steering a vehicle according to claim 10.

17. acquiring an around view monitor (AVM) RGB image of the vehicle by the camera; using the AVM RGB image to perform accuracy assessment operations; Further comprising:

11. A method for steering a vehicle according to claim 10.

18. the accuracy assessment operation includes an accuracy root mean square error (RMSE) assessment operation; 20. A method for steering a vehicle according to claim 17.

Images