Vehicle control system

Abstract

To provide a vehicle control device that increases a possibility of reliably executing deviation warning or deviation control by improving recognition accuracy of a boundary of a travel area of a vehicle.SOLUTION: A vehicle control device recognizes a boundary of a travel area of a vehicle based on image data acquired by a camera which captures an image of a landscape in front of the vehicle, and the vehicle control device performs at least one of deviation warning and deviation control when the vehicle control device predicts that the vehicle may deviate from a reference line set based on the boundary or when the vehicle deviates from the boundary. Moreover, the vehicle control device determines whether or not a registered three-dimensional object that has been registered in advance is present based on the image data; and when the vehicle control device determines that the registered three-dimensional object is present, the vehicle control device makes the boundary easier to recognize than the normal region other than a simplified area in the image data in a predetermined simplified area set based on the position of the registered three-dimensional object in the image data.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device that recognizes a boundary of a travel area of a vehicle and performs at least one of departure warning and departure control when the vehicle departs from a reference line set based on the boundary. [Background Art]

[0002] Conventionally, vehicle control devices that perform at least one of departure warning and departure control have been known. For example, the vehicle control device described in Patent Document 1 (hereinafter referred to as "first conventional device") executes departure control based on "the amount of departure of the vehicle from a white line" and "the intersection angle between the vehicle and the white line".

[0003] Further, the vehicle control device described in Patent Document 2 (hereinafter referred to as "second conventional device") recognizes a white line using Hough transform. More specifically, the second conventional device applies a larger weight to the vote value of Hough transform in the lower region of image data acquired by a camera than to that in the upper region of the image data. Thereby, even if a small amount of noise is extracted in the upper region of the image data, a correct white line can be recognized as long as a correct white line edge is acquired in the lower region. The lower region of the image data is a region near the vehicle, and the upper region of the image data is a region far from the vehicle. [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2014-159249 [Patent Document 2] Japanese Unexamined Patent Application Publication No. 2007-179386 [Summary of the Invention]

[0005] In the second conventional device, the voting value may exceed the "recognition threshold for detecting white lines" only in the lower region of the image data, which may reduce the accuracy of white line recognition in the upper region. For example, in situations where it is difficult to recognize white lines far from the vehicle, the second conventional device may not be able to recognize distant white lines.

[0006] In this case, the second conventional device is highly likely to be unable to recognize a white line unless the vehicle approaches that white line that it failed to recognize. This is because the unrecognized white line only becomes part of the lower region of the image data when the vehicle approaches that white line. When such a situation occurs, a vehicle control device like the first conventional device is highly likely to execute a lane departure warning or lane departure control with a delay.

[0007] This invention was made to address the aforementioned problems. Specifically, one of the objectives of this invention is to provide a vehicle control device that increases the likelihood of reliably performing lane departure warnings or lane departure control by improving the accuracy of recognizing the boundaries of the driving area.

[0008] The vehicle control device of the present invention (hereinafter referred to as "the present invention device") is A camera (22) that photographs the scenery in front of the vehicle and acquires image data, A control unit (20) recognizes the boundary (BL) of the vehicle's driving area based on the image data, and when it is predicted that the vehicle will deviate from a reference line (Lth) set based on the boundary, or when an execution condition is met that is met when the vehicle deviates from the boundary, it performs at least one of a deviation warning regarding the vehicle's deviation from the reference line and deviation control to suppress the vehicle from deviating from the reference line (step 535). The control unit, Based on the aforementioned image data, it is determined whether or not a specific registered three-dimensional object that has been registered in advance exists (step 615), If it is determined that the registered object exists (step 615 "Yes"), the boundary is made easier to recognize in a predetermined simplified region (EA) set based on the position of the registered object in the image data, compared to the normal region other than the simplified region in the image data (step 630). It is structured in this way.

[0009] According to the present invention, an eased region is set in the image data based on the position of the registered three-dimensional object. Since three-dimensional objects with nearby boundaries are registered as registered three-dimensional objects, even in situations where the boundary near the registered three-dimensional object is difficult to recognize, the possibility of recognizing that boundary can be increased even if the vehicle is not approaching that boundary (even if the vehicle is away from that boundary). Therefore, even if the vehicle is away from that boundary, it is possible to determine whether or not the execution conditions are met, and the possibility of reliably executing at least one of the departure warning and departure control can be increased. Accordingly, according to the present invention, by improving the accuracy of recognizing the boundary of the driving area, the possibility of reliably executing departure warning or departure control can be increased. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram of a vehicle control device according to an embodiment of the present invention. [Figure 2] This is an explanatory diagram illustrating the operation of a vehicle control device according to an embodiment of the present invention. [Figure 3] This is an explanatory diagram of the simplified region set in the image data. [Figure 4] This is an explanatory diagram illustrating an example of a starting point object that is registered as a registered object. [Figure 5] This is a flowchart of the routines executed by the CPU of the vehicle control system. [Figure 6] This is a flowchart of the subroutines executed by the CPU of the vehicle control system. [Modes for carrying out the invention]

[0011] As shown in Figure 1, the vehicle control device according to this embodiment (hereinafter referred to as "this device 10") is applied to a vehicle VA and comprises the components shown in Figure 1.

[0012] The vehicle control ECU20 is an ECU that performs at least one of the following: a lane departure warning and a lane departure control, which is a type of autonomous driving described later. Hereafter, it will be referred to as "ECU20".

[0013] In this specification, "ECU" refers to an electronic control unit comprising a microcomputer as its main component. An ECU may also be referred to as a control unit, controller, or computer. The microcomputer includes a CPU (processor), ROM, RAM, and an interface (I / F), etc. The functions of ECU20 may be realized by multiple ECUs.

[0014] The front camera 22 acquires image data by photographing the scenery in front of the vehicle VA. The front camera 22 transmits the image data to the ECU 20.

[0015] The millimeter-wave radar 24 transmits millimeter waves in front of the vehicle VA and acquires radar object information by receiving reflected waves that have been reflected by the reflection point of an object. The radar object information includes "the position of the object relative to the vehicle VA" and "the relative velocity Vr of the object relative to the vehicle VA". The millimeter-wave radar 24 transmits the radar object information to the ECU 20.

[0016] The vehicle speed sensor 26 detects the vehicle speed Vs, which represents the speed of the vehicle VA. The yaw rate sensor 28 detects the yaw rate Yr acting on the vehicle VA. The ECU 20 acquires these detected values.

[0017] The steering motor 30 is incorporated in a steering mechanism 32. The steering mechanism 32 is a mechanism for turning steered wheels in accordance with an operation of a steering wheel. In response to an instruction from an ECU 20, the steering motor 30 causes the steering mechanism 32 to generate an assist torque for assisting the operation of the steering wheel, and causes the steering mechanism 32 to generate an automatic steering torque for changing the steering angle of the steered wheels.

[0018] A display device 34 displays a departure warning screen described later. A speaker 36 outputs a departure warning sound described later.

[0019] Hereinafter, departure warning and departure control will be described with reference to FIG. 2. The ECU 20 recognizes a boundary BL (a right boundary RBL and a left boundary LBL) of a travel area TA where the vehicle VA is traveling based on image data. Examples of the boundary BL include a white line on a road, a guardrail, a curb, a wall, and the like. The ECU 20 sets reference lines Lth (a right reference line RLth and a left reference line LLth) at positions separated from the boundary BL by a predetermined distance in a direction orthogonal to the boundary BL.

[0020] When either of the following condition 1 and condition 2 is satisfied, the ECU 20 determines that an execution condition is satisfied, and executes at least one of a departure warning and departure control. Condition 1: A predicted traveling path PR of the vehicle VA intersects the reference line Lth (that is, it is predicted that the vehicle VA will depart from the reference line Lth). Condition 2: The vehicle VA has departed from the reference line Lth.

[0021] The ECU 20 causes the display device 34 to display a departure warning screen for warning a driver of departure of the vehicle VA from the reference line Lth. The ECU 20 may cause the speaker 36 to output a departure warning sound for warning the driver of departure of the vehicle VA from the reference line Lth. The ECU 20 may simultaneously perform display of the departure warning screen and output of the departure warning sound.

[0022] The ECU 20 obtains a target steering angle θtgt to prevent the vehicle VA from deviating from (or having deviated from) the reference line Lth (i.e., to return the vehicle VA inside the reference line Lth). The ECU 20 controls the steering motor 30 so that the steering angle θ matches the target steering angle θtgt.

[0023] (Operation) The ECU20 determines, based on the image data, whether or not a registered object RO, which has been pre-registered in the ECU20, exists. If the ECU20 determines that a registered object RO exists, it sets an eased region EA (see Figures 2 and 3) based on the position of the registered object RO in the image data. A key feature of the ECU20 is that it makes it easier to recognize the boundary BL in the eased region EA of the image data than in the normal region outside the eased region RA. In other words, a key feature of the ECU20 is that it recognizes the boundary BL in the eased region RA at an earlier timing than in the normal region.

[0024] Here, we will explain in detail the process for recognizing boundary line blue (BL). The ECU20 extracts feature points of white lines, guardrails, curbs, and walls from image data. The ECU20 fits each type of feature point to an "approximation line represented by a multidimensional function (e.g., a 3D function)" and obtains a voting value DV based on the number of feature points located on the approximation line. The ECU20 identifies approximation lines whose voting value DV is greater than or equal to a predetermined recognition threshold DVth as boundary candidates, and the ECU20 recognizes the boundary from among the boundary candidates.

[0025] In the normal domain, the ECU20 recognizes boundary BL by using the normal recognition threshold DVnth as the recognition threshold DVth. In the simplified domain EA, it recognizes boundary BL by using the simplified recognition threshold DVeth, which is set to a smaller value than the normal recognition threshold DVnth, as the recognition threshold DVth. As a result, boundary BL is more easily recognized in the simplified domain EA than in the normal domain.

[0026] In ECU20, objects with a boundary BL nearby are registered as registered objects RO. An example of a registered object RO is the starting object SO shown in Figures 4(A) to (D).

[0027] Examples of starting point structures SO include the starting point (end) of a median strip MS (see Figures 4(A) and (B)) and the starting point (end) of a guardrail GR (see Figures 4(C) and (D)). In addition, starting point structures SO also include the starting point (end) of a wall.

[0028] In the image data, the simplified region EA is set based on the position of the registered object RO where a boundary BL exists nearby, thereby improving the accuracy of boundary BL recognition. Even in situations where it is difficult to recognize a boundary BL near a registered object RO, the possibility of the ECU 20 recognizing that boundary BL can be increased, even if the vehicle VA does not approach that boundary BL (even if the vehicle VA is away from that boundary BL). As a result, even if the vehicle VA is away from that boundary BL, it is possible to determine whether the execution conditions have been met, and the possibility of reliably executing at least one of the lane departure warning and lane departure control can be increased.

[0029] In the example shown in Figure 2, when vehicle VA enters an intersection, the driver steers the steering wheel to change the vehicle VA's course to the right. Before the vehicle VA's course is changed to the right, the ECU 20 determines, based on image data, that a registered object RO exists and sets an easy-to-understand area EA in the image data. As a result, the ECU 20 makes it easier to recognize the median strip MS behind the registered object RO at the far end of the intersection as the boundary BL. This allows the ECU 20 to predict that the vehicle VA's predicted course PR will deviate from the reference line Lth when the vehicle VA's course is changed to the right, and can perform at least one of a deviation warning and / or deviation control.

[0030] (Specific operation) <Deviation Control> The CPU of ECU20 executes the routine shown in the flowchart in Figure 5 at predetermined intervals. When the appropriate time arrives, the CPU starts processing from step 500 in Figure 5 and executes steps 505 to 515.

[0031] Step 505: The CPU acquires image data from the forward camera 22 and radar object information from the millimeter-wave radar 24. Step 510: The CPU executes a boundary recognition subroutine to recognize the boundary line (BL). The boundary recognition subroutine will be described later using Figure 6.

[0032] Step 515: The CPU determines whether it has recognized boundary BL using the boundary recognition subroutine. Specifically, the CPU determines whether it has recognized at least one of the right boundary RB and the left boundary LB. If boundary BL is not recognized (neither the right boundary RB nor the left boundary LB is recognized), the CPU determines "No" in step 515 and proceeds to step 595 to terminate this routine. If boundary BL is recognized (at least one of the right boundary RB and the left boundary LB is recognized), the CPU determines "Yes" in step 515 and executes steps 520 to 530.

[0033] Step 520: The CPU sets a reference line Lth at a predetermined distance from the boundary line BL. Step 525: The CPU obtains the predicted path PR of the vehicle VA based on the vehicle speed Vs and yaw rate Yr. Step 530: The CPU determines whether the predicted path PR intersects with the reference line Lth, thereby determining whether the vehicle VA is predicted to deviate from the reference line Lth.

[0034] If the predicted path RP intersects the reference line Lth, the CPU determines "Yes" in step 530 and proceeds to step 535. In step 535, the CPU performs deviation control. After that, the CPU proceeds to step 595 and terminates this routine.

[0035] In addition, the CPU may, in step 535, execute a deviation warning instead of deviation control, or it may execute both a deviation warning and deviation control.

[0036] On the other hand, if the predicted path RP does not intersect the reference line Lth, the CPU determines "No" in step 530 and proceeds to step 540. In step 540, the CPU determines whether or not the vehicle VA has deviated from the reference line Lth.

[0037] If vehicle VA deviates from the reference line Lth, the CPU determines "Yes" in step 540, executes deviation control in step 535, and proceeds to step 595 to terminate this routine.

[0038] If vehicle VA does not deviate from the reference line Lth, the CPU determines "No" in step 540 and proceeds to step 545. In step 545, the CPU determines, based on the predicted path RP, whether there is a possibility of vehicle VA colliding with registered object RO.

[0039] If there is a possibility of collision with the registered object RO, the CPU determines "Yes" in step 545, performs deviation control in step 535, and proceeds to step 595 to terminate this routine.

[0040] If there is no possibility of collision with the registered object RO, the CPU determines "No" in step 545 and proceeds to step 595, terminating this routine.

[0041] <Boundary Recognition Subroutine> When the CPU proceeds to step 510 in Figure 5, it starts processing from step 600 in Figure 6 and executes steps 605 and 610.

[0042] Step 605: The CPU extracts feature points for each type of boundary line (white lines, guardrails, curbs, and walls) from the image data. Step 610: The CPU identifies the traffic light distance D based on the image data and determines whether that traffic light distance D is less than or equal to the threshold distance Dth (i.e., whether the vehicle VA is less than or equal to the threshold distance Dth from the intersection).

[0043] If the signal distance D is less than or equal to the threshold distance Dth, the CPU determines "Yes" in step 610 and proceeds to step 615. In step 615, the CPU identifies the position of stationary objects based on radar object information and determines whether or not a registered object RO exists among the stationary objects in the image data.

[0044] If a registered object RO exists, the CPU determines "Yes" in step 615 and executes steps 620 to 650.

[0045] Step 620: The CPU sets an eased region EA in the image data based on the position of the registered object RO image in the image data. The eased region EA is set in the region behind the registered object RO (above the image data), and the eased region EA may include the image of the registered object EA. Step 625: The CPU obtains at least one approximation line based on the feature points of the simplified region EA, and obtains the voting value DV by dividing the number of feature points located in the simplified region EA by the number of pixels in the simplified region EA. The CPU then obtains an approximation line for each type of feature point and the voting value DV for each type of feature point.

[0046] Step 630: The CPU obtains an approximate line as a boundary candidate where the voting value DV is greater than or equal to the "facilitation threshold DVeth, which is set to a value smaller than the normal recognition threshold DVnth". Step 635: The CPU recognizes the boundary candidate with the highest confidence level among the boundary candidates on the right side of the vehicle VA, which has a confidence level equal to or greater than a predetermined threshold, as the right boundary RB, and recognizes the boundary candidate with the highest confidence level among the boundary candidates on the left side of the vehicle VA, which has a confidence level equal to or greater than a threshold, as the left boundary LB. For example, the CPU obtains confidence levels based on the following criteria 1 and 2. Perspective 1: The shorter the lateral distance in the vehicle width direction between the approximate candidate and the vehicle VA, the higher the reliability. Perspective 2: The shorter the distance between the estimated position (based on the previously recognized boundary BL relative to the vehicle VA and predicted path RP) and the position of the approximate candidate, the higher the confidence level. If no boundary candidates exist with a confidence level above the threshold, the CPU will not recognize the boundary.

[0047] Step 640: The CPU obtains at least one approximation line based on the feature points in the normal region and obtains the voting value DV by dividing the number of feature points located in the normal region by the number of pixels in the normal region. Step 645: The CPU obtains an approximate line as a boundary candidate where the voting value DV is greater than or equal to the normal recognition threshold DVnth. Step 650: The CPU recognizes the boundary candidate with the highest confidence level among the candidate boundary on the right side of the vehicle VA, which has a confidence level equal to or greater than a predetermined threshold, as the right boundary RB, and recognizes the boundary candidate with the highest confidence level among the candidate boundary on the left side of the vehicle VA, which has a confidence level equal to or greater than a threshold, as the left boundary LB. The method for obtaining the confidence level is the same as in Step 635. After that, the CPU proceeds to step 695 and terminates this routine.

[0048] On the other hand, if the signal distance D is greater than the threshold distance Dth when the CPU proceeds to step 610, or if it is determined from the image data that no signal TR exists, the CPU determines "No" in step 610 and proceeds to step 640. If the registered object RO does not exist when the CPU proceeds to step 615, the CPU determines "No" in step 615 and proceeds to step 640. In these cases, the entire area of ​​the image data is set to the normal area.

[0049] According to this embodiment, in the simplified region EA, the boundary BL is more easily recognized than in the normal region. Therefore, even in situations where the boundary BL near the registered object RO is difficult to recognize, the boundary BL can be recognized at an earlier timing. This increases the likelihood of reliably executing deviation warnings or deviation control, and reduces the possibility of delayed execution of deviation warnings and / or deviation control, as well as the possibility of deviation warnings or deviation control not being executed at all.

[0050] Furthermore, there are cases where the type of boundary line (BL) of the driving area differs before and after an intersection (for example, the type of boundary line (BL) changes from a white line to the curb of the median strip (MS) before and after an intersection). However, according to this embodiment, even in this case, the possibility of recognizing the boundary lines (BL) before and after the intersection can be increased.

[0051] The starting point object SO, which is registered as registered object RO, is often located at an intersection. Therefore, when the signal distance D becomes less than or equal to the threshold distance Dth (step 610 "Yes"), the ECU20 determines whether or not the registered object RO exists based on the image data (step 615). This reduces the possibility that the simplified area EA is incorrectly set and the boundary BL is incorrectly recognized.

[0052] Furthermore, there is a high probability that boundary lines (BL) such as white lines and curbs exist near utility poles and signs. For this reason, utility poles and signs may be registered as registered three-dimensional objects (RO). In this case, the ECU20 does not determine whether the signal distance D is less than or equal to the threshold distance Dth, but rather determines whether a registered three-dimensional object RO exists based on the image data.

[0053] Furthermore, the ECU20 determines that the execution conditions have been met even if there is a possibility that the vehicle VA will collide with the registered object RO in step 545 shown in Figure 5, and executes at least one of the departure warning and departure control. This reduces the possibility of the vehicle VA colliding with the registered object RO.

[0054] (modified version) The ECU20 may change the "type of boundary line (BL) to facilitate recognition" depending on the type of registered object RO. Details are explained below. (1) If the registered object RO is "an object that marks the starting point of a median strip MS" or "an object that marks the starting point of a utility pole or sign", the ECU20 sets the recognition threshold DVth for white lines and curbs in the simplified area EA lower than in the normal area. (2) If the type of registered object RO is "an object that serves as the starting point of a guardrail GR", the ECU20 sets the recognition threshold DVth for white lines and guardrails in the simplified region EA lower than in the normal region. (3) If the type of registered object RO is "object that serves as the starting point of a wall", the ECU20 sets the recognition threshold DVth for white lines, walls, and gutters in the simplified area EA lower than in the normal area. Since the type of boundary line (BL) that is most likely to be present near a registered object RO varies depending on the type of registered object RO, the ECU20 is configured to change the "type of boundary line that is easier to recognize" according to the type of registered object RO. This further reduces the possibility of boundary lines being misrecognized.

[0055] When ECU20 obtains an approximation line based on the feature points of the image data and obtains the voting value DV of the approximation line, the weight of feature points located in the simplified region EA may be made larger than the weight of feature points located in the normal region. This makes it easier to recognize the boundary BL of the simplified region EA.

[0056] Furthermore, the ECU20 may set the reference line Lth to be inside the boundary BL as the vehicle speed Vs increases. Alternatively, the ECU20 may set the reference line Lth to be in the same position as the boundary BL.

[0057] The vehicle control device 10 is applicable to vehicles such as engine-powered vehicles, hybrid vehicles, plug-in hybrid vehicles, fuel cell vehicles, and electric vehicles. [Explanation of Symbols]

[0058] 10...Vehicle control device, 20...Vehicle control ECU, 22...Forward camera, 30...Steering motor, 32...Steering mechanism, 34...Display device, 36...Speaker.

Claims

1. A camera that photographs the scenery in front of the vehicle and acquires image data, The system includes a control unit that recognizes the boundary of the vehicle's driving area based on the image data, and when it predicts that the vehicle will deviate from a reference line set based on the boundary, or when an execution condition is met that is met when the vehicle deviates from the reference line, it performs at least one of a deviation warning regarding the vehicle's deviation from the reference line and deviation control to suppress the vehicle from deviating from the reference line. The control unit, Based on the aforementioned image data, it is determined whether or not a specific registered three-dimensional object that has been registered in advance exists. If it is determined that the registered three-dimensional object exists, the boundary of a predetermined simplified region, which is set based on the position of the registered three-dimensional object in the image data, is made easier to recognize than the normal region outside the simplified region in the image data. It is configured in such a way, The control unit, From the aforementioned image data, feature points that can serve as boundaries are extracted. The voting value is obtained based on the number of feature points located on the approximation line obtained based on the feature points, An approximation line in which the voting value is equal to or greater than a predetermined recognition threshold is obtained as a candidate boundary. Recognize the boundary from among the candidate boundaries. It is configured in such a way, The control unit further, The recognition threshold in the simplified region is set to a lower value than the recognition threshold in the normal region, or the weight of the vote value in the simplified region is increased compared to the weight of the vote value in the normal region, thereby making the boundary easier to recognize in the simplified region than in the normal region. A vehicle control device configured as follows.

2. In the vehicle control device according to Claim 1, The control unit, The starting point object that serves as the starting point of the boundary is registered in advance as the registered object. Based on the image data, it is determined whether or not the starting point object exists as the registered object. It is configured in such a way. Vehicle control device.

3. In the vehicle control device according to Claim 1, The control unit is configured to satisfy the execution condition even when there is a possibility that the vehicle will collide with the registered object, and to execute at least one of the departure warning and the departure control. Vehicle control device.

4. In the vehicle control device according to Claim 1, The control unit is configured to change the type of boundary that is made easier to recognize in the simplified region, depending on the type of the registered three-dimensional object. Vehicle control device.

Images