Road line recognition device and driving assistance device
Through the Kalman filter, the reliability of road marking is predicted and calculated, the accuracy of lane dividing lines identification under complex conditions such as detours is solved, and the safety and reliability of the driving assistance system are improved.
Patent Information
- Application Number
- CN202310609159.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
AI Technical Summary
In the prior art, it is difficult to accurately identify the lane dividing line under complex road conditions such as detours, especially in severe weather or abnormal vehicle sensors, which may cause the vehicle to deviate from the lane.
The Kalman filter is used to predict the road marking position, and the reliability of the road marking is calculated by calculating the difference between the predicted value and the observed value and the Kalman gain to identify the road marking position.
It improves the accuracy of lane identification and the safety of the driving assistance system, and can promptly remind the driver and terminate the assistance function in uncertain conditions to prevent the vehicle from deviating from the lane.
Smart Images

Figure CN119027897A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a road marking recognition device and a driving assistance device for recognizing road markings on a road. Background Art
[0002] In recent years, efforts to provide access to sustainable transportation systems that take into account the disadvantaged among traffic participants have become increasingly active. To achieve this goal, efforts are being made to further improve the safety and convenience of traffic through research and development of driving assistance technologies. As a technology related to such driving assistance, the following device is known in the past: a line segment as a candidate for a lane dividing line is extracted from an image captured by an on-board camera, and a reliability indicating the accuracy of the extracted line segment as a lane dividing line is calculated based on the linearity of the extracted line segment (for example, see Patent Document 1).
[0003] However, the shape of the lane dividing line varies depending on the shape of the road. Therefore, if the reliability is calculated based on linearity as in the device described in Patent Document 1, the recognition result of the lane dividing line may not be accurately evaluated on a curved road or the like.
[0004] Prior art literature
[0005] Patent Literature
[0006] Patent Document 1: Japanese Patent Application Publication No. 2013-191040. Summary of the invention
[0007] A road marking recognition device according to a technical solution of the present invention comprises: a detection unit that detects the surrounding conditions of a vehicle; a position determination unit that determines the position of a road marking for defining a lane in which the vehicle is traveling based on the detection result of the detection unit; a prediction unit that uses a Kalman filter to predict the position of the road marking at the current time point based on the position of the road marking in the time series determined by the position determination unit before the current time point; a calculation unit that calculates the reliability of the position of the road marking determined by the position determination unit; and an identification unit that identifies the position of the road marking based on the reliability calculated by the calculation unit. The calculation unit calculates the reliability based on the difference between the position of the road marking at the current time point predicted by the prediction unit, i.e., the predicted value, and the position of the road marking at the current time point determined by the position determination unit, i.e., the observed value, and the Kalman gain of the Kalman filter. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The objects, features and advantages of the present invention will be further clarified through the following description of the embodiments in conjunction with the accompanying drawings.
[0009] Figure 1 FIG. 1 is a diagram schematically showing road markings recognized from an image captured by a vehicle-mounted camera.
[0010] Figure 2 This is a block diagram schematically showing an example of the main configuration of the driving assistance device according to the embodiment of the present invention.
[0011] Figure 3 2 is a diagram for explaining an angle θ of the host vehicle with respect to the driving lane.
[0012] Figure 4 It is a diagram for explaining the calculation of the curvature radius of a virtual road marking line.
[0013] Figure 5A It is a diagram showing the state of change of the Kalman gain.
[0014] Figure 5B It is a diagram showing the change of the deviation degree.
[0015] Figure 6 It is a diagram for explaining reliability.
[0016] Figure 7 It is shown by Figure 2 A flowchart of an example of a process performed by a controller. DETAILED DESCRIPTION
[0017] The following reference Figure 1 to Figure 7 The embodiment of the present invention is described. The driving assistance device of the embodiment of the present invention has a driving assistance function of controlling a driving actuator to assist a driver of a vehicle or to automatically drive a vehicle. Hereinafter, "driving assistance" in the present embodiment includes driving assistance that assists a driver's driving operation and automatic driving that automatically drives a vehicle without relying on the driver's driving operation. Sometimes, a vehicle to which the driving assistance device of the present embodiment is applied is referred to as the present vehicle, in distinction from other vehicles.
[0018] Driving assistance functions include, for example, a lane keeping assistance function that assists the driver's steering wheel operation to keep the vehicle in the driving lane, a constant speed / following driving assistance function that assists the driver's acceleration or braking operation to drive at a certain speed or a certain distance between vehicles, etc. In addition, it also includes a lane change assistance function that controls the driving actuator to change the driving lane without relying on the driver's steering wheel operation, acceleration operation, and braking operation.
[0019] However, when the lane keeping assist function is effective, the vehicle identifies the position of the road markings that define the driving lane of the vehicle based on the captured image in front of the vehicle obtained by the vehicle-mounted camera, and assists the steering wheel operation based on the position of the recognition result of the road markings (hereinafter referred to as virtual road markings) so that the vehicle drives near the center of the driving lane. However, when identifying the position of the road markings based on the captured image of the vehicle-mounted camera, sometimes the position of the road markings cannot be accurately determined due to abnormalities of the sensor of the vehicle-mounted camera, road surface reflection, rain and snow and other bad weather. In addition, sometimes it is difficult to determine the position of the road markings at a location where there are road markings other than road markings (such as crosswalks). Figure 1 FIG. 2 is a diagram schematically showing road markings recognized from an image captured by a vehicle-mounted camera. Figure 1 , which are the recognition results of the road marking lines OL and OR defining the driving lane LN of the host vehicle 101, that is, the virtual road marking lines PL and PR. Figure 1 As shown, sometimes the positions of the virtual road marking lines PL and PR deviate from the positions of the road marking lines OL and OR due to sensor abnormality of the vehicle-mounted camera, etc. In such a case, when the driving assistance by the lane keeping assist function is continued based on the virtual road marking lines PL and PR, the vehicle 101 may deviate from the driving lane LN. Therefore, in this embodiment, the reliability of the position of the road marking line determined based on the image captured by the vehicle-mounted camera is calculated, and the position of the road marking line is identified based on the reliability.
[0020] Figure 2 1 is a block diagram schematically showing an example of the main structure of the driving assistance device 100 according to the embodiment of the present invention. Figure 2 As shown, the driving assistance device 100 mainly includes an electronic control unit (ECU) 10, an IMU (Inertial Measurement Unit) 1, a camera 2, an operation input unit 3, an information reporting unit 4, and an actuator AC. In addition, the driving assistance device 100 has a road line recognition device 50 that constitutes a part of the driving assistance device 100. The controller 10 includes a computer including a computing unit 11 such as a CPU (central processing unit), a storage unit 12 such as a ROM (read only memory) and a RAM (random access memory), and other peripheral circuits. The IMU 1, the camera 2, the operation input unit 3, the information reporting unit 4, and the actuator AC are connected to the controller 10 respectively.
[0021] IMU1 detects physical quantities representing the motion state of the vehicle 101, more specifically, detects the angle or angular velocity and acceleration of rotation around three orthogonal axes. Camera 2 is installed at a predetermined position on the front of the vehicle, continuously photographs the space in front of the vehicle, and obtains image data of the object. Camera 2 is a single-lens camera with a photographing element (image sensor) such as CCD (charge coupled device) and CMOS (complementary metal oxide semiconductor). Camera 2 has an image processing ECU20. Image processing ECU20 processes the image data obtained by photographing (hereinafter referred to as photographed image data or simply photographed image), detects the road markings contained in the photographing range, generates information related to the detected road markings, i.e., road marking information (coefficients C0, C1, C2, C3 described later), and outputs it to the controller 10. It should be noted that camera 2 can also be a stereo camera. Objects include road surface markings, road markings, road signs set on and around the road, structures, etc.
[0022] The operation input unit 3 is composed of buttons that can be operated by the passenger, and is, for example, arranged on a touch panel of a display provided in front of the driver's seat. In addition, for example, it is arranged at a position on the steering wheel of the driver's seat where the passenger (driver) can operate the steering wheel. The signal from the operation input unit 3 is input to the controller 10. The passenger can start, end, and set each driving assistance function via the operation input unit 3.
[0023] The information notification unit 4 is composed of a display arranged in front of the driver's seat, a speaker arranged around the driver's seat, etc., and notifies the occupants of information related to the driving assistance function. It should be noted that the display can also be arranged in the instrument panel opposite the driver's seat, or on the instrument panel in the center of the vehicle width direction. Or it can also be configured as a head-up display. The information notification unit 4 operates according to the control signal from the controller 10. The driver can confirm the start, end and setting status of each driving assistance function through the information notification unit 7.
[0024] The actuator AC is a driving actuator for controlling the driving of the vehicle 101. When the driving source is an engine, the actuator AC includes a throttle actuator for adjusting the opening of the throttle of the engine (throttle opening). When the driving source is a driving motor, the driving motor is included in the actuator AC. The brake actuator for operating the brake device of the vehicle 101 and the steering actuator for driving the steering device are also included in the actuator AC.
[0025] Figure 2 The controller 10 includes a position determination unit 111, a prediction unit 112, a reliability calculation unit 113, a recognition unit 114, a notification unit 115, and a driving assistance unit 116 as a calculation unit 11 ( Figure 1) is a functional structure. In addition, the controller 10 has a storage unit 12. It should be noted that Figure 2 As shown, the position determination unit 111 , the prediction unit 112 , the reliability calculation unit 113 , the recognition unit 114 , the IMU 1 , and the camera 2 are included in the road marking recognition device 50 .
[0026] The position determination unit 111 determines the road marking lines ( LN ) defining the lane LN in which the host vehicle 101 is traveling, based on the detection result (captured image) of the camera 2. Figure 1 The detection result of the camera 2 includes the road line information (coefficients C0, C1, C2, C3) input from the image processing ECU 20 of the camera 2.
[0027] The prediction unit 112 uses a Kalman filter to predict the position of the road marking at the current time point based on the positions of the road marking in the time series determined by the position determination unit 111 before the current time point.
[0028] The reliability calculation unit 113 calculates the reliability of the position of the road marking determined by the position determination unit 111. More specifically, the reliability calculation unit 113 calculates the reliability based on the difference between the observed value and the predicted value, where the observed value is the position of the road marking at the current time point determined by the position determination unit 111, and the predicted value is the position of the road marking at the current time point predicted by the prediction unit 112, and the Kalman gain of the Kalman filter.
[0029] <Prediction Process>
[0030] Here, the prediction of the position of the road marking by the prediction unit 112 is described in detail. First, the prediction unit 112 estimates the state of the road marking defining the lane LN of the host vehicle 101 at the current time point (time k). At this time, the estimated state of the road marking at the current time point is recorded as X k (θ, ω). θ is the angle of the road line determined by the position determination unit 111 relative to the vehicle 101. ω is the angular velocity detected by the IMU 1, more specifically, the rotational angular velocity (yaw rate) of the center of gravity of the vehicle 101 around the vertical axis. The prediction unit 112 performs a state estimation based on the previous time (time k-1). k-1 (θ, ω) predicts the estimated state value X at the current time point k The following is the prediction process of (θ, ω). k-1 (θ, ω) predicts the state X at the current time point k (θ, ω) is called the prior estimated value or predicted value and is denoted by X k|k-1 (θ, ω).
[0031] Formula (i) is used to calculate the prior estimated value X k|k-1 (θ, ω), where u is the interference component contained in the sensor values of camera 2 and IMU 1, and the interference component caused by processing delay when calculating θ and ω based on those sensor values. Formula (ii) is the covariance conversion formula. In formula (ii), P k-1 represents the post-estimation value X relative to the previous time (time k-1) k|k The covariance matrix of the error of the true value of (θ, ω) (hereinafter referred to as the post-estimation error). k|k (θ, ω), which will be described later. k|k-1 is relative to the prior estimated value X k|k-1 The covariance matrix of the error of the true value of (θ, ω) (hereinafter referred to as the prior state estimation error).
[0032] X k|k-1 =F·X k-1 +u (i)
[0033] P k|k-1 =F·P k-1 ·F T +Q (ii)
[0034] X in formula (i) and formula (ii) k-1 , F, P, Q are respectively expressed by the following equations (iii) to (vi). The system matrix F is determined by the equation (viii) described later. The process disturbance Q is shown in equation (xviii), including the roll angle detected by IMU1 and the pitch angular velocity W k .
[0035]
[0036]
[0037]
[0038]
[0039] Here, the state equation of the Kalman filter (Formula (i)) is explained. The road markings determined by the position determination unit 111, that is, the virtual road markings PL, PR, are expressed by a cubic formula (the following Formula (vii)) shown by the road marking information (coefficients C0, C1, C2, C3) input from the image processing ECU20 of the camera 2. It should be noted that the coefficients contained in the road marking information are not limited to four. For example, when the image processing ECU20 outputs five coefficients corresponding to the quartic equation, the road marking information can also include five coefficients. The inclination angle of the straight line equation calculated by linear regression based on the point sequence of the road markings represented by Formula (vii) represents the angle θ between the vehicle 101 and the driving lane. It should be noted that the angle θ is not the angle between the vehicle 101 and the driving lane LN specified by the actual road markings OL, OR, but the angle between the vehicle 101 and the lane specified by the virtual road markings PL, PR. Figure 3 θ is a diagram for explaining the angle θ between the vehicle 101 and the driving lane. Figure 3 As shown, the coefficient C0 of the road marking information represents the distance from the center position of the vehicle 101 (in Figure 3 In the example shown, the point on the virtual road lines PL and PR closest to the center position VC in the vehicle length direction and the vehicle width direction is shown. The line segments TL and TR in the figure represent the tangent lines at the point C0 on the virtual road lines PL and PR, respectively. The line segment CL in the figure is a line segment passing through the center position VC and parallel to the tangent lines TL and TR. Figure 3 As shown, the angle θ is defined by the line segment CL and the vehicle length direction of the host vehicle 101. The point CT indicates the center position of the turn of the host vehicle 101.
[0040] The prediction unit 112 uses the following equation (viii) based on the detection value (yaw angular velocity) ω input from the IMU 1 k , moving distance D k (the value obtained by integrating the traveling speed v of the host vehicle 101 over a predetermined time Δt), the curvature 1 / r of the road marking line (virtual road marking line PL, PR) determined by the position determination unit 111, and the angle θ at the previous time (time k-1) k-1 To predict the angle θ at the current time point (time k) k The system matrix F of the state equation (Formula (i)) is determined according to this Formula (viii).
[0041] Y=C0+ClX+C2X 2 +C3X 3 (vii)
[0042] θ k =θ k-1 +ω k ·Δt-D k / r (viii)
[0043] Figure 4 1 is a diagram for explaining the calculation of the curvature radius r of the virtual road marking lines PL and PR. The x-axis and y-axis in the diagram represent the traveling direction and the vehicle width direction of the host vehicle 101 at time t0. C0(t0), C0(t1), ..., C0(t n ) represents a point cloud represented by a coefficient C0 input from the image processing ECU 20 every predetermined time Δt from time t0 (hereinafter referred to as a sample point cloud).
[0044] The prediction unit 112 stores the road line information input from the image processing ECU 20 in a time series in the storage unit 12. The prediction unit 112 reads a sample point cloud (coefficient C0 of the time series) from the road line information stored in the time series in the storage unit 12, and calculates the curvature radius r of the virtual road lines PL and PR based on the sample point cloud as follows.
[0045] First, the prediction unit 112 calculates the vehicle coordinate system at time t0 using the following equation (ix): Figure 4 The movement amount Mov of the position of the vehicle 101 from time t0 in the coordinate system composed of the x-axis and the y-axis.
[0046]
[0047] Mov is the movement vector of the host vehicle 101, which is expressed by the following equation (x). In equation (x), movex and movey are the movement vectors of the host vehicle 101 from time point t0 to time point t n The amount of movement in the x and y directions. vn is the velocity vector of the vehicle 101 at a predetermined time Δt, which is expressed by the following equation (xi). vnx and vny in equation (xi) are the velocity vectors of the vehicle 101 at a predetermined time Δt detected by the IMU 1 at the time point t n The moving speed in the x and y directions is represented by Rz(ψn) from the time point t0 to the time point t n The rotation matrix of the rotation about the z-axis (vertical axis of the center of gravity of the host vehicle) is expressed by the following equation (xii). ψn is the rotation matrix of the host vehicle 101 from time point t0 to time point t n The rotation angle around the z-axis is calculated by the following formula (xiii) based on the angular velocity ω detected by IMU1.
[0048]
[0049]
[0050]
[0051]
[0052] The prediction unit 112 calculates the sample point clouds C0(t0), C0(t1), ..., C0(t n )'s coordinates (x0, y0), (x1, y1), ..., (x n ,y n ).
[0053] The prediction unit 112 substitutes the calculated coordinates (absolute values) of the sample point cloud into the following formula (xiv) to calculate the parameters a, b, and c of the curve represented by the sample point cloud. The position determination unit 111 substitutes the calculated parameters a, b, and c into formula (xv) to calculate the curvature radius r of the curve. The position determination unit 111 calculates the curvature radius r of each of the virtual road marking lines PL and PR according to formula (xv).
[0054]
[0055]
[0056] <Update Process>
[0057] The prediction unit 112 uses the state update formula shown in formula (vii) to calculate the state update formula based on the pre-estimated value X k|k-1 (θ, ω) and the observed value Zk to update the estimated state value X at the current time point k Update process of (θ, ω). Observation value Z k is the angle θ calculated based on the road marking information input from the image processing ECU 20 of the camera 2 at the current time and the detection value (yaw angular velocity) ω input from the IMU 1 at the current time. k (θ, ω) is called the post-estimation value or update value, denoted by X k|k (θ, ω). K in equation (vii) is the Kalman gain. The Kalman gain K is expressed as equation (xvii), based on the covariance matrix P of the prior state estimation error at the current time point k calculated by equation (ii). k|k-1 The covariance matrix R of the observation error is calculated. In addition, the Kalman gain K is expressed by the matrix formula of formula (xviii). θ is the component corresponding to the angle θ, k ω is the component corresponding to the angular velocity ω.
[0058] X k|k =X k|k-1 +K(Z k -X k|k-1 ) (xvi)
[0059]
[0060]
[0061] Whenever the image processing ECU 20 of the camera 2 receives road line information including the coefficient C0, the prediction unit 112 performs prediction processing using the equation (i) based on the state X at the previous moment. k-1 (θ, ω) predicts the state X at the current time point k (θ, ω). That is, calculate the prior estimated value X k|k-1 (θ, ω). In addition, the position identification unit 111 performs an update process using equation (vi), based on the pre-estimated value X k|k-1 (θ, ω) and the observed value Z k Update the estimated state value X at the current time point k (θ, ω). That is, calculate the post-estimation value X k|k (θ, ω). The position identification unit 111 calculates the post-estimation value X k|k (θ, ω) is stored in the storage unit 12. The post-estimation value X stored in the storage unit 12 k|k (θ, ω) is used for the state X at the next moment (time t+1) k+1 The prediction of (θ, ω). Figure 5A and 5B : is a diagram showing the change of the Kalman gain K and the degree of deviation De when the prediction process and the update process are repeatedly executed. The degree of deviation De is the prior estimation value X k|k-1 and the ex post facto estimated value X k|k The Mahalanobis distance represents the degree of deviation between the predicted value and the observed value. The degree of deviation De is calculated according to the following formula (xix).
[0062]
[0063] like Figure 5A and 5B As shown in FIG. 1 , immediately after the prediction using the Kalman filter is started, the Kalman gain K changes greatly and the prediction is unstable. Therefore, the pre-estimated value X may k|k-1 (θ, ω) and the post-estimation value X k|k The deviation De between (θ, ω) becomes greater than a predetermined level (time point t1). After that, by repeatedly executing the prediction process and the update process, the change of the Kalman gain K gradually decreases and converges to a certain value (at Figure 5A 0.5 in the example above).
[0064] However, when the prediction by the Kalman filter is stable, that is, when the Kalman gain K converges to a certain value, the deviation De becomes small, so the prior estimation value X k|k-1(θ, ω) and the post-estimation value X k|k (θ, ω) usually does not have a large deviation. Therefore, when the deviation degree De becomes above a specified level in a predicted stable state, it can be determined that an abnormality has occurred in the determination of the position of the road marking by the position determination unit 111. That is, it can be determined that the reliability of the position of the road marking determined by the position determination unit 111 is low. Therefore, the reliability calculation unit 113 calculates the reliability Re of the position of the road marking determined by the position determination unit 111 using formula (xx) based on the correlation between such Kalman gain K and the deviation degree De. At this time, the reliability calculation unit 113 normalizes the deviation degree De so that the scale of the deviation degree De is consistent with the scale of the Kalman gain K, and substitutes the normalized deviation degree De into formula (xx). Figure 6 is a diagram for explaining the reliability Re. Figure 6 As shown, in a state where the Kalman gain K converges to a certain value, when the difference between the value of the Kalman gain K and the deviation De increases, the value of the reliability Re decreases (time point t2).
[0065]
[0066] The recognition unit 114 recognizes the position of the road markings for defining the driving lane of the host vehicle 101 based on the reliability Re calculated by the reliability calculation unit 113. More specifically, when the Kalman gain K converges to a certain value and the prediction is stable, when the reliability Re is greater than the specified value Th, the recognition unit 114 trusts the observed value and recognizes the position of the road markings determined by the position determination unit 111 (the position of the virtual road markings PL and PR) as the position of the road markings OL and OR. On the other hand, even though the prediction is stable, when the reliability Re is lower than the specified value Th, the recognition unit 114 determines that the predicted value and the observed value deviate due to the abnormality of the observed value, and outputs an abnormal signal indicating that the road markings OL and OR cannot be recognized to the notification unit 115 and the driving assistance unit 116.
[0067] When an abnormal signal is input from the recognition unit 114, the notification unit 115 notifies the occupant of information to remind the occupant of the need for attention (hereinafter referred to as the reminder information). More specifically, the notification unit 115 sends a control signal for outputting the reminder information to the information notification unit 4 (display, speaker). The reminder information includes information (display information, sound information) for notifying that the vehicle 101 may deviate from the driving lane LN when the driving assistance using the lane keeping assist function is continued.
[0068] The driving assistance unit 116 controls the actuator AC when the lane keeping assistance function is effective so that the vehicle 101 travels near the center of the driving lane LN. When an abnormal signal is input from the recognition unit 114, the driving assistance unit 116 terminates the driving assistance using the lane keeping assistance function. That is, the lane keeping assistance function is invalidated. It should be noted that the driving assistance unit 116 can also output a control signal for outputting information (display information, sound information) to remind the driver to perform the end operation of the driving assistance function to the information notification unit 4 when terminating the driving assistance. In addition, the driving assistance unit 116 can also output a control signal for outputting information (display information, sound information) requiring the driver to perform manual operation to the information notification unit 4 when the driving assistance is terminated. In addition, the driving assistance unit 116 can also output a control signal for outputting information (display information, sound information) requiring the driver to perform manual operation to the information notification unit 4 when the driving assistance is terminated. In addition, the driving assistance unit 116 can also output a control signal for outputting information (display information, sound information) requiring the driver to perform manual operation to the information notification unit 4 when the driving assistance is terminated. In addition, the driving assistance can also be terminated when the driver's manual operation via the operation input unit 3 is detected.
[0069] Figure 7 2 is a flowchart showing an example of processing executed by the controller 10. The processing shown in the flowchart is repeatedly executed in a predetermined cycle when the lane keeping assist function of the host vehicle 101 is set to be effective, for example.
[0070] First, in step S11, the state X at the previous moment is obtained from the storage unit 12. k-1 (θ, ω). In step S12, the state X at the current time point is predicted k Specifically, using equation (i), according to the state X at the previous moment obtained in step S11, k-1 (θ, ω), calculate the prior estimated value (predicted value) X k|k-1 (θ, ω). It should be noted that in the first execution, the state X at the previous moment k-1 (θ, ω) is not stored in the storage unit 12, so the state X at the current time point is calculated using the initial value pre-stored in the storage unit 12. k The prediction of (θ, ω).
[0071] Next, in step S13, the observation value Z is obtained. k Specifically, the angle θ between the vehicle 101 and the driving lane LN is calculated based on the road marking information at the current time point input from the image processing ECU 20 of the camera 2. And the calculation result and the detection value (yaw angular velocity) ω input from the IMU 1 are obtained as the observation value Z k In step S14, the estimated state value X at the current time point is updated. k Specifically, using equation (vii), the predicted value X calculated in step S12 is k|k-1(θ, ω) and the observed value Z obtained in step S13 k , calculate the post-estimated value (updated value) X k|k In step S15, the predicted value X calculated in step S12 is calculated. k|k-1 (θ, ω) and the observed value Z obtained in step S13 k Specifically, using formula (xix), the deviation De between the predicted value calculated in step S12 and the updated value calculated in step S14 is calculated. In step S16, using formula (xx), the reliability Re is calculated based on the Kalman gain K and the deviation De calculated in step S15.
[0072] In step S17, it is determined whether the reliability Re calculated in step S16 is lower than the specified value Th. When step S17 is negative (S17: No), in step S18, it is determined that the driving assistance using the lane keeping assist function can be continued, and the processing is terminated. When step S17 is positive (S17: Yes), in step S19, it is determined that the driving assistance using the lane keeping assist function cannot be continued, and the occupant is notified of the reminder information. In addition, the lane keeping assist function is invalidated, and the driving assistance using the lane keeping assist function is terminated.
[0073] The following effects can be achieved by adopting the embodiments of the present invention.
[0074] (1) The road marking recognition device 50 includes: a camera 2 for detecting the surrounding conditions of the host vehicle 101; a position determination unit 111 for determining the position of the road marking defining the lane in which the host vehicle 101 is traveling based on the detection result (photographed image) of the camera 2; a prediction unit 112 for predicting the position of the road marking at the current time point based on the position of the road marking in the time series determined by the position determination unit 111 before the current time point using a Kalman filter; a reliability calculation unit 113 for calculating the reliability of the position of the road marking determined by the position determination unit 111; and a recognition unit 114 for recognizing the position of the road marking based on the reliability calculated by the reliability calculation unit 113. The reliability calculation unit 113 calculates the reliability based on the difference between the predicted value and the observed value, which is the position of the road marking at the current time point predicted by the prediction unit 112, and the Kalman gain of the Kalman filter. Thus, the recognition result of the road marking can be accurately evaluated. As a result, it is possible to accurately determine whether a driving assistance function such as a lane keeping assistance function can be continued, thereby improving safety during driving using the driving assistance function.
[0075] (2) The road marking recognition device 50 further includes a storage unit 12 for storing prediction values. The prediction unit 112 calculates the prediction value at the current time point by updating the prediction value at the previous time point based on the observed value and the prediction value at the previous time point stored in the storage unit 12. The difference between the prediction value and the observed value is the Mahalanobis distance between the prediction value at the previous time point and the prediction value at the current time point. The reliability calculation unit 113 calculates the logarithm representing the ratio of the Kalman gain to the Mahalanobis distance as the reliability (Formula (xx)). In this way, the reliability of the position of the observed road marking can be accurately calculated.
[0076] (3) The driving assistance device 100 includes: a road marking recognition device 50; and a notification unit 115, which notifies the occupant of information to call attention when the reliability calculated by the reliability calculation unit 113 is lower than a predetermined value. When the position of the road marking cannot be accurately recognized due to the influence of interference, etc., the occupant can be called to pay attention at an appropriate time.
[0077] (4) The driving assistance device 100 further includes a driving assistance unit 116 that performs driving assistance based on the reliability calculated by the reliability calculation unit 113. The driving assistance unit 116 stops driving assistance when the reliability calculated by the reliability calculation unit 113 is lower than a predetermined value. Thus, when the position of the road marking cannot be accurately recognized due to the influence of interference, etc., driving assistance can be stopped at an appropriate timing.
[0078] The above-mentioned embodiment can be modified into various forms. Several modified examples are described below. In the above-mentioned embodiment, the camera 2 as a detection unit detects the surrounding conditions of the vehicle 101, but the detection unit is not limited to the camera. The road marking recognition device 50 can also have a laser radar or a radar instead of the camera or as a detection unit together with the camera. In addition, in the above-mentioned embodiment, an example in which the camera 2 has an image processing ECU 20 is shown, but the image processing ECU can also be provided on the controller 10 side.
[0079] In addition, in the above-mentioned embodiment, the reliability calculation unit 113 as a calculation unit calculates the Mahalanobis distance between the predicted value at the previous moment and the predicted value at the current time point as information indicating the difference (deviation) between the predicted value and the observed value. However, the information indicating the difference between the predicted value and the observed value may also be information other than the Mahalanobis distance.
[0080] In the above embodiment, the driving assistance device 100 determines whether the lane keeping assist function can be continued based on the reliability Re. However, the reliability Re can also be used to determine whether driving assistance functions other than the lane keeping assist function can be continued.
[0081] Furthermore, in the above-mentioned embodiment, when the reliability Re is lower than the predetermined value Th, the recognition unit 114 determines that the determination of the position of the road marking line by the position determination unit 111 is abnormal, and outputs an abnormal signal indicating that the road marking lines OL and OR cannot be recognized to the notification unit 115 and the driving assistance unit 116. However, the recognition unit may output the abnormal signal to the notification unit 115 when the reliability Re is a first predetermined value, and output the abnormal signal to the driving assistance unit 116 when the reliability Re is a second predetermined value smaller than the first predetermined value. Thus, the occupant is alerted before the driving assistance is terminated, and the occupant can prepare for manual operation with some room.
[0082] The above description is merely an example, and the above embodiment and modified examples do not limit the present invention unless the features of the present invention are impaired. One or more of the above embodiment and modified examples may be arbitrarily combined, and the modified examples may be combined with each other.
[0083] The present invention can accurately evaluate the recognition result of road markings.
[0084] The present invention has been described above in conjunction with the preferred embodiments, but it should be understood by those skilled in the art that various modifications and changes can be made without departing from the scope of the disclosure of the claims set forth below.
Claims
1. A road marking recognition device, characterized in that: have: A detection unit (2) for detecting the surrounding conditions of the vehicle; A position determination unit (111) which determines the position of a road marking for defining a lane in which the host vehicle is traveling based on the detection result of the detection unit (2); A prediction unit (112) which uses a Kalman filter to predict the position of the road marking line at a current time point based on the position of the road marking line in a time series determined by the position determination unit (111) before the current time point; a calculation unit (113) for calculating the reliability of the position of the road marking determined by the position determination unit (111); as well as an identification unit (114) for identifying the position of the road marking according to the reliability calculated by the calculation unit (113), The calculation unit (113) calculates the reliability based on the difference between the position of the road marking at the current time point predicted by the prediction unit (112), i.e., the predicted value, and the position of the road marking at the current time point determined by the position determination unit (111), i.e., the observed value, and the Kalman gain of the Kalman filter.
2. The road marking recognition device according to claim 1, characterized in that: It also includes a storage unit (12) that stores the predicted value. The prediction unit (112) calculates the prediction value at the current time point by updating the prediction value at the previous time point based on the observed value and the prediction value at the previous time point stored in the storage unit (12). The difference between the predicted value and the observed value is the Mahalanobis distance between the predicted value at a previous moment and the predicted value at a current time point.
3. The road marking recognition device according to claim 2, characterized in that: The calculation unit (113) calculates a logarithm representing a ratio of the Kalman gain to the Mahalanobis distance as the reliability.
4. The road marking recognition device according to any one of claims 1 to 3, characterized in that: A notification unit (115) is provided for notifying the occupant of a warning message when the reliability calculated by the calculation unit (113) is lower than a predetermined value.
5. A driving assistance device, characterized in that: have: The road marking recognition device according to any one of claims 1 to 3; and a driving assistance unit (116) for performing driving assistance based on the reliability calculated by the calculation unit (113); The driving assistance unit (116) stops the driving assistance when the reliability calculated by the calculation unit (113) is lower than a predetermined value.
6. A driving assistance device, characterized in that: have: The road marking recognition device according to any one of claims 1 to 3; a notification unit (115) for notifying the occupant of a message to call attention to the situation when the reliability calculated by the calculation unit (113) is lower than a first predetermined value; as well as a driving assistance unit (116) for performing driving assistance based on the reliability calculated by the calculation unit (113); The driving assistance unit (116) stops the driving assistance when the reliability calculated by the calculation unit (113) is lower than a second predetermined value which is smaller than the first predetermined value.
Citation Information
Patent Citations
Lane mark detector and reliability calculation method of lane mark
JP2013191040A
