Auxiliary driving method based on pure vision sensor
By using a vision-based driver assistance method and an MPC algorithm based on exhaustive search and kinematic relationships to achieve LKA and ACC control, the high cost and mass production difficulties caused by high-precision map dependence are solved, and low-cost, high-reliability vehicle driver assistance is achieved.
Patent Information
- Application Number
- CN202411172879.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-12
AI Technical Summary
Existing driver assistance solutions rely on high-precision maps, which leads to high costs and is not conducive to large-scale mass production. The high frequency of map updates and information lag affect the accuracy and safety of the system. The system is vulnerable when map data is missing or incorrect.
The system employs a vision-based driver assistance method. By acquiring lane and vehicle information, it uses an exhaustive MPC algorithm to calculate Lane Assist (LKA) and a kinematic MPC algorithm to calculate Adaptive Cruise Control (ACC). The system is then connected to the vehicle's CAN bus via an embedded connection to achieve vehicle control, thus eliminating the reliance on high-precision maps.
It achieves low-cost vehicle assisted driving, is suitable for mass production, improves system reliability and safety, and avoids the resource and time costs of map updates.
Smart Images

Figure CN118928439A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of vehicle engineering technology, and in particular to an assisted driving method based on a pure vision sensor. Background Art
[0002] In the past, algorithm suppliers in the field of assisted driving usually provided L4 autonomous driving solutions based on high-precision maps. This solution has high costs for manual mapping, is not conducive to large-scale mass production, and may cause risks due to factors such as road and bridge construction.
[0003] Specifically, the production and updating of high-precision maps requires a lot of resources and time. The collection of high-precision data requires professional equipment and technicians, and the high update frequency leads to high costs. Since the update frequency of high-precision maps is limited by cost and fleet size, map information may not be able to reflect the latest changes in the road in a timely manner. This lag will affect the accuracy and safety of the autonomous driving system. Finally, the autonomous driving system that relies on high-precision maps may show greater vulnerability when faced with missing or erroneous map data. Once the map data is inaccurate or missing, the reliability of the system will be greatly reduced. Summary of the invention
[0004] The purpose of the present invention is to provide an assisted driving method based on pure visual sensors, which performs assisted driving according to the input perception information of the pure visual sensors, thereby solving the problem that the existing assisted driving solutions rely on high-precision maps, which is costly and not conducive to large-scale mass production.
[0005] To achieve the above object, the present invention provides an assisted driving method based on pure visual sensors, comprising the following steps:
[0006] Step 1: Obtain lane information and vehicle information;
[0007] Step 2: Use the exhaustive MPC algorithm to implement LKA calculation horizontally;
[0008] Step 3: longitudinally implement ACC calculation using the MPC algorithm based on kinematic relationship;
[0009] Step 4: When there is no obstacle vehicle ahead longitudinally, use P control for speed;
[0010] Step 5: After the corresponding algorithm code is generated, it is integrated in an embedded manner and connected to the real vehicle CAN bus to realize real vehicle control.
[0011] Optionally, in step 1, lane information and vehicle information are calculated using CAN message information sensed and output by the camera, wherein the lane information is the cubic polynomial coefficients (A0, A1, A2, A3) of the lane line, and the vehicle information includes the vehicle speed, heading angle, and expected steering wheel angle.
[0012] Optionally, during the execution of step 2, the vehicle previews a certain distance under the current vehicle speed, heading angle and expected steering wheel angle, and controls the current steering wheel angle using the lateral deviation between the vehicle and the control line after reaching the preview distance as the cost function indicator.
[0013] Optionally, the lateral control cost function expression is as follows:
[0014] J = 1000*(dy-dy r ) 2
[0015] d=dy-dyr
[0016] Among them, J is the cost function, dy is the lateral distance of the lane line, dyr is the actual lateral distance of the vehicle, and d is the lateral deviation between the vehicle and the lane line.
[0017] Optionally, during the execution of step 3, the future relative motion state is predicted based on the motion state of the vehicle and the front vehicle targets in the self-lane, left lane and right lane, that is, the relative distance, speed and acceleration, a polynomial of the displacement, speed and acceleration changing with time is constructed, and a cost function with relative displacement, speed and acceleration as indicators is constructed to implement corresponding speed planning.
[0018] Optionally, the motion state equations of the ego vehicle and the front vehicle are as follows:
[0019]
[0020] Expected displacement of the vehicle: X = C5t 5 +C4t4+C3t3+C2t2+C1t+C0
[0021] Expected speed of the vehicle: V = 5C5t4 + 4C4t3 + 3C3t2 + 2C2t + C1
[0022] Expected acceleration of the vehicle: A = 20C5t3 + 12C4t2 + 6C3t + 2C2
[0023] Expected jerk of the vehicle: A'=60C5t2+24C4t+6C3
[0024] Among them, the initial motion state of the ego vehicle (longitudinal displacement 0, longitudinal velocity V1, longitudinal acceleration A1), the initial motion state of the obstacle vehicle in front (longitudinal displacement X1, longitudinal velocity V2, longitudinal acceleration A2); C0, C1, C2, C3, C4, C5 are the weight coefficients of the quintic polynomial, which are used to predict the expected displacement velocity and acceleration of the ego vehicle respectively;
[0025] The corresponding cost function is as follows:
[0026] J=w a A sum +w a 'A' sum +w d D sum +w d 'D' sum
[0027] A sum is the sum of squares of the accumulated acceleration errors, A' sum is the sum of squares of the cumulative errors of the acceleration derivatives, D sum is the sum of squares of the cumulative displacement error, D' sum is the square sum of the accumulated velocity errors; w a is the weight coefficient of acceleration, w a' is the weight coefficient of the acceleration derivative, w d is the weight coefficient of displacement, w d' is the speed weight coefficient.
[0028] Optionally, in step 4, the cruise control scheme adopts single P control, and the corresponding acceleration is obtained by multiplying the error between the current speed of the vehicle and the target speed by a P coefficient;
[0029] The P-controlled cruise acceleration formula is as follows:
[0030] a=(V-V1) / P
[0031] Wherein, a is the planned acceleration of cruise control, V is the target vehicle speed, V1 is the vehicle speed, P is the 1 coefficient of P control, P=5.
[0032] Optionally, in step 5, the open source CAN driver architecture and network architecture of the Linux kernel are used to write corresponding underlying software program codes according to the dbc files of the actual vehicle's wire-controlled chassis and wire-controlled steering.
[0033] The present invention provides an assisted driving method based on pure visual sensors, which acquires lane information and vehicle information based on pure visual camera perception, implements LKA lane keeping based on the MPC algorithm of exhaustive method in the lateral direction, obtains steering wheel angle information through the lateral control cost function, and calculates the speed information of ACC adaptive cruise based on the MPC algorithm of kinematic relationship in the longitudinal direction, and then integrates it into the industrial computer in the embedded Linux kernel open source mode, connects it with the CAN bus of the real vehicle, and realizes the real vehicle control. When cruising at a fixed speed, the lateral LKA technology is combined, and the speed is controlled by P control. The present invention can realize vehicle assisted driving by relying only on the perception scheme of the pure visual sensor camera, gets rid of the dependence on high-precision maps, has low deployment cost, and is conducive to large-scale mass production. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0035] Figure 1 It is a schematic flow chart of the steps of an assisted driving method based on pure visual sensors of the present invention.
[0036] Figure 2 It is a schematic diagram of the kinematic relationship of lateral speed planning of an assisted driving method based on a pure visual sensor of the present invention.
[0037] Figure 3 It is a schematic diagram of the kinematic relationship of longitudinal speed planning of an assisted driving method based on a pure visual sensor of the present invention.
[0038] Figure 4 It is a simulation comparison schematic diagram of the lateral deviation of the lane centerline in a specific embodiment of the present invention.
[0039] Figure 5 It is a schematic diagram of the simulation effect of the relative speed of longitudinal following vehicle in a specific embodiment of the present invention. DETAILED DESCRIPTION
[0040] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0041] See also Figure 1The present invention provides an assisted driving method based on a pure visual sensor, comprising the following steps:
[0042] S1: Obtain lane information and vehicle information;
[0043] S2: LKA calculation is realized horizontally using the MPC algorithm based on the exhaustive method;
[0044] S3: longitudinally, the MPC algorithm based on kinematic relationship is used to realize ACC calculation;
[0045] S4: When there is no obstacle vehicle ahead longitudinally, the speed is controlled by P;
[0046] S5: After the corresponding algorithm code is generated, it is integrated in an embedded manner and connected to the real vehicle CAN bus to realize real vehicle control.
[0047] The following describes the three main components of the present invention:
[0048] 1. LKA lane keeping
[0049] MPC algorithm based on exhaustive method. The cubic polynomial coefficients (A0, A1, A2, A3) of the lane lines of the self-lane and the adjacent lane are obtained by using the CAN message information sensed by the camera output. According to the effectiveness of the lane line, the corresponding lateral compensation value is given to obtain the lane center line of the self-lane and the adjacent lane as the lateral control line. According to the current state of the steering wheel angle, the corresponding angles are divided on the left and right (exhaustive method). Under the current state of the vehicle (vehicle speed, heading angle, expected steering wheel angle), the vehicle previews a certain distance, and uses the lateral deviation of the vehicle from the control line after reaching the preview distance as the cost function indicator to control the current steering wheel angle (such as Figure 2 As shown). In the figure, L is the preview distance length. After previewing a certain distance L, the lateral deviation length d from the center line of the lane. The green line is the steering wheel angle assumed by the exhaustive method. Select the steering wheel angle state with the smallest cost function, pass the low-pass filter with the current steering wheel angle, and increase it with a certain steering wheel angular velocity deltasteerangle (° / T). The value of deltasteerangle is adjusted according to the vehicle speed and the size of A0. The preview distance L is also positively correlated with the vehicle speed. The faster the vehicle speed, the larger the corresponding preview distance coefficient.
[0050] The steering wheel angle for lateral control of the vehicle will produce a certain yaw angular velocity, which will in turn have a certain impact on the steering wheel angle, and a certain compensation amount needs to be added to the corresponding steering wheel angle.
[0051] The yaw rate gain formula of the vehicle steady-state response is as follows:
[0052]
[0053] (1) In the formula, δ is the front wheel angle (rad), u is the vehicle speed (m / s), L is the wheelbase (m), k is the stability coefficient, and w r is the vehicle's yaw rate (angular velocity).
[0054] The calculation formula for the compensation amount of the front wheel turning angle generated by the vehicle's yaw rate is:
[0055] Δ=w*R*57.3*C (2)
[0056] In formula (2), Δ is the compensation value of the front wheel angle generated by the yaw rate, w is the yaw rate calculated in formula (1) + the yaw rate measured by EPS, R is the compensation value from the steering wheel angle to the front wheel angle, and C is the inverse of the yaw rate gain calculated in formula (1).
[0057] Lateral control cost function;
[0058] J = 1000*(dy-dy r ) 2 (3)
[0059] d=dy-dyr (4)
[0060] In formula (4), dy is the lateral distance between the vehicle and the lane line after the vehicle previews a certain distance L in the current state (vehicle speed, yaw angle), and dyr is the change in the lateral distance of the lane line after the vehicle previews the distance L. The lateral control cost function uses the lateral distance error as the only control variable.
[0061] 2. ACC adaptive cruise control
[0062] The longitudinal following speed planning adopts the MPC algorithm based on kinematic relationship. According to the motion state (relative distance, speed, acceleration) of the self-vehicle and the front vehicle (self-lane, left lane, right lane) targets, the future relative motion state is predicted, and the polynomial of displacement, speed and acceleration changing with time is constructed. The cost function with relative displacement, speed and acceleration as indicators is constructed to realize the corresponding speed planning.
[0063] Figure 3The longitudinal speed planning motion relationship is shown. The initial relative distance between the ego vehicle and the obstacle vehicle in front is X, the initial motion state of the ego vehicle (longitudinal speed V1, longitudinal acceleration A1), and the initial motion state of the obstacle vehicle in front (longitudinal speed V2, longitudinal acceleration A2). The MPC algorithm using kinematic relations is based on the existing motion relations and exhaustively lists the expected distance (Xfinal) with the vehicle in front, the expected speed of the ego vehicle (V2') and the acceleration of the ego vehicle (A2') at certain future subdivision moments. According to the time and the known state quantities of the ego vehicle at the beginning and end, a polynomial is constructed, and the cost function (with relative displacement) is used as the weight for speed planning. The kinematic equation and the cost function are shown in equations (5) and (6).
[0064]
[0065] Expected displacement of the vehicle: X = C5t 5 +C4t4+C3t3+C2t2+C1t+C0
[0066] Expected speed of the vehicle: V = 5C5t4 + 4C4t3 + 3C3t2 + 2C2t + C1
[0067] Expected acceleration of the vehicle: A = 20C5t3 + 12C4t2 + 6C3t + 2C2
[0068] Expected jerk of the vehicle: A'=60C5t2+24C4t+6C3
[0069] Among them, the initial motion state of the ego vehicle (longitudinal displacement 0, longitudinal velocity V1, longitudinal acceleration A1), the initial motion state of the obstacle vehicle in front (longitudinal displacement X1, longitudinal velocity V2, longitudinal acceleration A2);
[0070] C0, C1, C2, C3, C4, and C5 are the weight coefficients of the quintic polynomial, which are used to predict the expected displacement velocity and acceleration of the ego vehicle, respectively;
[0071] J=w a A sum +w a' A' sum +w d D sum +w d' D' sum (6)
[0072] A sum is the sum of squares of the accumulated acceleration errors, A' sum is the sum of squares of the cumulative errors of the acceleration derivatives, D sum is the sum of squares of the cumulative displacement error, D' sum is the sum of squares of the accumulated velocity errors; w ais the weight coefficient of acceleration, w a' is the weight coefficient of the acceleration derivative, w d is the weight coefficient of displacement, w d' is the speed weight coefficient.
[0073] Cruise control technology
[0074] If there is no obstacle vehicle in front longitudinally, P control is adopted for speed, and low-pass filtering is performed on the planned acceleration to make a smooth transition.
[0075] The P-controlled cruise acceleration formula is as follows:
[0076] a=(V-V1) / P
[0077] Wherein, a is the planned acceleration of cruise control, V is the target vehicle speed, V1 is the vehicle speed, P is the 1 coefficient of P control, P=5.
[0078] 3. Real vehicle deployment of assisted driving functions based on pure vision solutions
[0079] The present invention carries out the response work of LKA+ACC real vehicle integration, uses the CAN drive architecture and network architecture of the Linux kernel open source, and writes the corresponding underlying software program code according to the dbc files of the real vehicle's wire-controlled chassis and wire-controlled steering. The upper application layer algorithm code is generated by the code generation tool of matlab, and the application layer code and the underlying embedded code are integrated in the industrial computer and connected to the real vehicle CAN bus to realize real vehicle control.
[0080] Furthermore, in order to prove the effectiveness of the assisted driving method based on pure visual sensors, the present invention also proposes a specific embodiment:
[0081] Use matlab / simulink to simulate the lateral LKA and longitudinal ACC based on pure camera perception information, see Figure 4 and Figure 5 The embodiment reduces the cost of deploying assisted driving in real vehicles and adopts a single camera solution. Figure 4 This is a simulation comparison diagram of the lateral deviation of the lane centerline. The red line is the lane centerline and the blue line is the actual vehicle driving path. Figure 5 This is the simulation effect diagram of the longitudinal following vehicle relative speed.
[0082] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.
Claims
1. An assisted driving method based on pure visual sensors, characterized in that: The following steps are involved: Step 1: Obtain lane information and vehicle information; Step 2: Use the exhaustive MPC algorithm to implement LKA calculation horizontally; Step 3: longitudinally implement ACC calculation using the MPC algorithm based on kinematic relationship; Step 4: When there is no obstacle vehicle ahead longitudinally, use P control for speed; Step 5: After the corresponding algorithm code is generated, it is integrated in an embedded manner and connected to the real vehicle CAN bus to realize real vehicle control.
2. The assisted driving method based on pure visual sensor as claimed in claim 1, characterized in that: In step 1, the lane information and vehicle information are calculated using the CAN message information sensed and output by the camera. The lane information is the cubic polynomial coefficients (A0, A1, A2, A3) of the lane line, and the vehicle information includes the vehicle speed, heading angle, and expected steering wheel angle.
3. The assisted driving method based on pure visual sensor as claimed in claim 2, characterized in that: During the execution of step 2, the vehicle previews a certain distance under the current vehicle speed, heading angle and expected steering wheel angle, and controls the current steering wheel angle using the lateral deviation between the vehicle and the control line after reaching the preview distance as the cost function indicator.
4. The assisted driving method based on pure visual sensor as claimed in claim 3, characterized in that: The lateral control cost function expression is as follows: J=1000*(from r ) 2 d=dy-dyr Among them, J is the cost function, dy is the lateral distance of the lane line, dyr is the actual lateral distance of the vehicle, and d is the lateral deviation between the vehicle and the lane line.
5. The assisted driving method based on pure visual sensor as claimed in claim 4, characterized in that: During the execution of step 3, the future relative motion state is predicted based on the motion state of the vehicle and the front vehicle targets in the self-lane, left lane and right lane, i.e., relative distance, speed and acceleration, and a polynomial of displacement, speed and acceleration changing with time is constructed. A cost function with relative displacement, speed and acceleration as indicators is constructed to implement the corresponding speed planning.
6. The assisted driving method based on pure visual sensor as claimed in claim 5, characterized in that: The motion state equations of the ego vehicle and the front vehicle are as follows: Expected displacement of the vehicle: X = C5t 5 +C4t4+C3t3+C2t2+C1t+C0 Expected speed of the vehicle: V = 5C5t4 + 4C4t3 + 3C3t2 + 2C2t + C1 Expected acceleration of the vehicle: A = 20C5t3 + 12C4t2 + 6C3t + 2C2 Expected jerk of the vehicle: A'=60C5t2+24C4t+6C3 Among them, the initial motion state of the ego vehicle (longitudinal displacement 0, longitudinal velocity V1, longitudinal acceleration A1), the initial motion state of the obstacle vehicle in front (longitudinal displacement X1, longitudinal velocity V2, longitudinal acceleration A2); C0, C1, C2, C3, C4, C5 are the weight coefficients of the quintic polynomial, which are used to predict the expected displacement velocity and acceleration of the ego vehicle respectively; The corresponding cost function is as follows: J=w a A sum +w a' A’ sum +w d D sum +w d' D’ sum A sum is the sum of squares of the accumulated acceleration errors, A' sum is the sum of squares of the cumulative errors of the acceleration derivatives, D sum is the sum of squares of the cumulative displacement error, D' sum is the sum of squares of the accumulated velocity errors; w a is the weight coefficient of acceleration, w a' is the weight coefficient of the acceleration derivative, w d is the weight coefficient of displacement, w d' is the weight coefficient of speed.
7. The assisted driving method based on pure visual sensor as claimed in claim 1, characterized in that: In step 4, the cruise control scheme adopts single P control, and the corresponding acceleration is obtained by multiplying the error between the current speed of the vehicle and the target speed by a P coefficient; The P-controlled cruise acceleration formula is as follows: a=(V-V1) / P Wherein, a is the planned acceleration of cruise control, V is the target vehicle speed, V1 is the vehicle speed, P is the 1 coefficient of P control, P=5.
8. The assisted driving method based on pure visual sensor as claimed in claim 1, characterized in that: In step 5, the corresponding underlying software program code is written based on the dbc files of the actual vehicle's wire-controlled chassis and wire-controlled steering, using the open source CAN driver architecture and network architecture of the Linux kernel.
