Vehicle control device and vehicle control method

Abstract

To provide a vehicle control device which does not cancel start inhibition control even when vehicle peripheral information is shown on a monitor.SOLUTION: A vehicle control device (DS) comprises: a peripheral state display device capable of displaying a vehicle's surrounding image on a monitor (81) in a vehicle's interior; a driver monitoring device (100) which acquires driver information including a direction of a driver's line of sight and facial orientation of the driver; and a controller (10) capable of executing start inhibition control to prevent a vehicle from starting in an inconsistent situation where a vehicle's start direction estimated based on a shift range of the vehicle does not match the driver's facial orientation. The controller does not execute the start inhibition control when the driver's line of sight is directed at the monitor and the monitor displays the vehicle surrounding image even though the inconsistent situation occurs.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a vehicle control device and a vehicle control method that execute start suppression control for suppressing the start of a vehicle according to the situation when a driver performs an operation to start the vehicle.

Background Art

[0002] One of the conventional vehicle control devices (hereinafter referred to as "conventional device") executes start suppression control for suppressing the start of the vehicle when the driver's line of sight direction does not match the start direction of the vehicle determined based on the range of the transmission (shift range). Further, in order to prevent unnecessary start suppression control from being executed, the conventional device cancels the start suppression control when the vehicle surrounding information acquired by the vehicle surrounding monitoring system is notified to the driver (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] However, for example, when the vehicle surrounding information is notified by displaying a vehicle surrounding image on a monitor (display), the driver may not always be looking at the monitor, so it may not be preferable to cancel the start suppression control. The present invention has been made to solve such problems.

[0005] One aspect of the present invention is a camera device (51-54) that generates image data by imaging a peripheral area of the vehicle, a peripheral situation display device (55, 10, 80) configured to be able to display a vehicle surrounding image based on the image data on a monitor (81) in the vehicle interior, A driver monitoring device (100) that acquires driver information including the direction of the driver's gaze and the direction the driver's face is facing, A controller (10, 60, 70) is configured to perform a start suppression control to suppress the start of the vehicle when a mismatch occurs between the starting direction of the vehicle estimated based on the vehicle's shift range and the direction the driver is facing. Equipped with, The controller is configured to allow the start suppression control not to be performed even if the mismatch occurs, if the driver's line of sight is directed toward the monitor (Yes in steps 320, 330, and 340) (step 370).

[0006] More specifically, if the aforementioned mismatch occurs but the driver's gaze is directed toward the monitor, the controller will not execute the start suppression control when the monitor is displaying the vehicle surroundings image, but will execute the start suppression control when the monitor is not displaying the vehicle surroundings image.

[0007] According to an aspect of the present invention, if a mismatch occurs where the vehicle's starting direction and the direction the driver's face is facing do not match, and it is presumed that the driver is looking at the vehicle's surroundings image displayed on the monitor, the starting suppression control will not be executed. Therefore, it is possible to avoid situations where the starting suppression control is not performed when the driver starts the vehicle without looking at either the vehicle's starting direction or the vehicle's surroundings image displayed on the monitor, and situations where the starting suppression control is performed when it is considered unnecessary because the driver is looking at the vehicle's surroundings image displayed on the monitor, causing discomfort to the driver.

[0008] Furthermore, the present invention also extends to the vehicle control method and program implemented by the above-mentioned vehicle control device. In addition, in the above description, in order to aid in understanding the invention, the reference numerals used in the embodiments are indicated in parentheses for the constituent elements of the invention corresponding to the embodiments. However, the constituent elements of the present invention are not limited to the embodiments defined by the above-mentioned reference numerals. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of a vehicle control device according to an embodiment of the present invention. [Figure 2] This is an example of an image of the area around a vehicle displayed on a monitor (display). [Figure 3] This is a flowchart showing the routines executed by the CPU of the vehicle control ECU. [Figure 4] This is a flowchart showing the routines executed by the CPU of the vehicle control ECU. [Figure 5] This is a flowchart showing the routines executed by the CPU of the vehicle control ECU. [Modes for carrying out the invention]

[0010] (composition) A vehicle control device DS (hereinafter referred to as "device DS") according to an embodiment of the present invention comprises the components shown in Figure 1 (camera, sensor, ECU, actuator, etc.). Device DS is applied to (mounted on) a vehicle.

[0011] In this specification, "ECU" refers to an Electronic Control Unit (ECU) that primarily comprises a microcomputer including a CPU (processor) and memory, and is also referred to as a controller. The components shown in Figure 1, which include multiple ECUs, are connected to each other via a Controller Area Network (CAN) to enable information exchange.

[0012] The vehicle control ECU10 performs the "start suppression control as a vehicle control function," which will be described later.

[0013] The forward camera system 20 includes a forward camera 21 and an image ECU 22. The forward camera 21 captures the "scene in front of the vehicle (including the road surface and objects)" at predetermined intervals and acquires forward camera image data. The image ECU 22 analyzes the forward camera image data from the forward camera 21 to generate forward camera object information, including the "position, relative longitudinal speed, relative lateral speed, and type" of objects in the forward camera image data, and transmits it to the vehicle control ECU 10.

[0014] The forward radar system 30 is a well-known device that acquires information about targets located in front of the vehicle using millimeter-wave radio waves, and includes a forward radar 31 and a forward radar ECU 32. The forward radar 31 transmits information about transmitted radio waves and information about received reflected waves to the forward radar ECU 32 at predetermined intervals. Based on the information from the forward radar 31, the forward radar ECU 32 acquires "forward radar target information," which is target information about targets located within the detection range of the forward radar 31, and transmits it to the vehicle control ECU 10. The forward radar target information includes the distance to the target, the bearing of the target, and the relative speed of the target.

[0015] The ultrasonic sensor device 40 includes a plurality of ultrasonic sensors (clearance sonars) 41 and a sonar ECU 42. Each of the plurality of ultrasonic sensors 41 is positioned at a predetermined location around the vehicle body and emits ultrasonic waves and receives ultrasonic waves reflected by objects. Based on "information about the timing of ultrasonic wave emission and the timing of reflected ultrasonic wave reception" from each ultrasonic sensor 41, the sonar ECU 42 acquires information about objects present around the vehicle as "sonar object information" and transmits it to the vehicle control ECU 10. The sonar object information includes the position of the object relative to the vehicle and the distance between the vehicle body and the object.

[0016] The PVM (Panoramic View Monitor) camera system 50 includes a front PVM camera 51, a left PVM camera 52, a right PVM camera 53, a rear PVM camera 54, and a PVM ECU 55. Each of the PVM cameras 51-54 is equipped with an ultra-wide-angle lens.

[0017] The front PVM camera 51, left PVM camera 52, right PVM camera 53, and rear PVM camera 54 capture images of the front and front side of the vehicle, the left side of the vehicle, the right side of the vehicle, and the rear of the vehicle, respectively, at predetermined intervals, and acquire image data.

[0018] The PVM ECU 55 generates overhead view image data and forward-moving view image data of the vehicle based on the image data transmitted from the PVM cameras 51-54, and transmits them to the vehicle control ECU 10. The structure of the PVM cameras 51-54, the overhead view image and forward-moving view image are well known (see, for example, Japanese Patent Publication No. 2022-86516, Japanese Patent Publication No. 2020-117128, and Japanese Patent Publication No. 2019-016825, etc.).

[0019] When the predetermined peripheral monitoring conditions are satisfied, the vehicle control ECU 10 transmits the "data of the bird's-eye view image and the data of the forward direction image" from the PVM·ECU 55 to the display ECU 80. The display ECU 80 causes the monitor 81 to display the "bird's-eye view image and the forward direction image" based on those image data as an "image for monitoring the situation around the vehicle (peripheral vehicle image)" (see FIG. 2). The predetermined peripheral monitoring conditions are satisfied, for example, when the vehicle speed SPD is less than or equal to a predetermined peripheral monitoring vehicle speed threshold value and the vehicle peripheral monitoring function is set to on by a touch operation on the driver's monitor 81. When the peripheral monitoring conditions are satisfied and the shift range obtained from the shift position sensor 94 described later is the forward range or other ranges, the forward direction image is an image of the front of the vehicle based on the image data acquired by the front PVM camera 51. When the peripheral monitoring conditions are satisfied and the shift range obtained from the shift position sensor 94 described later is the reverse range, the forward direction image is an image of the rear of the vehicle based on the image data acquired by the rear PVM camera 54. When the peripheral monitoring conditions are not satisfied, the display ECU 80 displays an "image other than the peripheral vehicle image (for example, a map image)" on the monitor 81.

[0020] The power train ECU 60 is connected to be able to drive the power train actuator 61. The actuator 61 can change the torque generated by the drive device of the vehicle (the power source of the host vehicle). The torque generated by the drive device is transmitted to the drive wheels via the gear mechanism. Therefore, the power train ECU 60 can control the driving force of the vehicle.

[0021] When the drive device of the vehicle is an internal combustion engine fueled by gasoline, the actuator 61 is, for example, a throttle valve actuator that changes the opening degree of the throttle valve. The vehicle may be an electric vehicle. In that case, the actuator 61 is an inverter that can change the torque of the electric motor. Further, the vehicle may be a hybrid vehicle. In that case, the actuator 61 includes an inverter for the electric motor and a throttle valve actuator for the internal combustion engine.

[0022] The brake ECU 70 controls the friction brake devices (braking devices) installed on each wheel of the vehicle by driving the brake actuator 71, thereby changing the braking force (friction braking force) applied to the vehicle. Therefore, the brake ECU 70 is capable of controlling the braking force.

[0023] The display ECU 80 can display a predetermined image on the monitor (display) 81. The monitor 81 is fixed in a predetermined position (for example, in the center cluster area) in the vehicle interior, in front of the driver's seat (i.e., in front of the driver seated in the driver's seat). Therefore, when the driver is looking at the monitor 81, the direction the driver's face is facing is in front of the vehicle.

[0024] The vehicle control ECU10 is further connected to the sensors described below, and receives the output values ​​(detected values) of these sensors as input. • A vehicle speed sensor 91 that detects the vehicle's speed (i.e., vehicle speed SPD). • An accelerator pedal operation amount sensor 92 detects the amount AP of operation of the vehicle's accelerator pedal. • Brake pedal operation amount sensor 93 detects the amount of operation BP of the vehicle's brake pedal. • A shift position sensor 94 detects the position (shift range) of the vehicle's shift lever. In this example, the shift position sensor 94 outputs a signal indicating whether the shift range is the forward range (D) for moving the vehicle forward, the reverse range (R) for moving the vehicle backward, or any other range (N / P). The vehicle control ECU 10 is also connected to other sensors that indicate the driving status of the vehicle.

[0025] The driver monitoring device (driver monitor) 100 is a device that acquires information (driver information) representing the state of the vehicle driver (including the direction of the driver's gaze and the direction the driver's face is facing), and includes a driver monitor camera 101 and a driver monitor ECU 102. The driver monitoring device 100 itself is well known and is disclosed, for example, in Japanese Patent Publication No. 2019-87143, Japanese Patent Publication No. 2019-87029, Japanese Patent Publication No. 2016-38866 and Japanese Patent Publication No. 2013-152700.

[0026] The driver monitor camera 101 is installed in a suitable location in front of the driver's seat of the vehicle (for example, on top of the steering column) and captures the driver's face at predetermined intervals to generate face image data. The driver monitor ECU 102 acquires the above driver information based on the face image data transmitted from the driver monitor camera 101 and transmits it to the vehicle control ECU 10.

[0027] In short, the driver monitor ECU 102 generates a face image based on face image data from the driver monitor camera 101, rotates the generated face image, and obtains the driver's face orientation based on the rotation angle of the face image that maximizes the match rate between the rotated face image and the pre-stored "face shape data when the driver is facing forward".

[0028] The driver monitor ECU 102 identifies the facial region from the generated driver's face image and detects facial components by extracting feature points of facial parts such as the eyes, nose, and mouth. Furthermore, the driver monitor ECU 102 detects the position of the Purkinje image (corneal reflection image) and the position of the pupil center, and determines the positional relationship between the Purkinje image and the pupil center. Then, based on the positional relationship between the Purkinje image and the pupil center, and the direction the driver's face is facing, the driver monitor ECU 102 obtains the driver's gaze direction.

[0029] (Summary of operation) The device DS determines whether the driver is performing an abnormal starting operation based on the estimated starting direction of the vehicle based on the shift range, the driver's line of sight direction, the direction the driver's face is facing, and whether or not an image of the area around the vehicle is displayed on the monitor 81. Based on the result, it executes or cancels the start suppression control to suppress the vehicle's starting. The start suppression control is a control that maintains the vehicle's driving force at the level necessary for creep driving (creep driving driving force) by, for example, setting the throttle valve opening to zero, regardless of the accelerator pedal operation amount AP. In other words, the start suppression control is a control that generates a driving force smaller than the normal driving force determined by the accelerator pedal operation amount AP. The start suppression control may further include braking force control that applies a braking force to the vehicle until the vehicle is stopped or the vehicle speed decreases to below a very low creep speed.

[0030] (Specific operation) The CPU of the vehicle control ECU 10 (hereinafter simply referred to as "CPU") executes the routines shown in the flowcharts in Figures 3 to 5 at predetermined intervals.

[0031] <Detection of abnormal starting operation when the starting direction is reverse> When an appropriate time arrives, the CPU starts processing from step 300 in Figure 3 and proceeds to step 310, where it determines whether the vehicle speed SPD is less than or equal to the first vehicle speed threshold SPD1th. The first vehicle speed threshold SPD1th is set to an extremely small predetermined value so that it is possible to determine whether the vehicle is in a state before starting.

[0032] If the vehicle speed SPD is less than or equal to the first vehicle speed threshold SPD1th, the CPU proceeds from step 310 to step 320 and determines whether the shift range is in the reverse range (whether the vehicle is moving in the reverse direction) based on the signal from the shift position sensor 94.

[0033] If the shift range is in the reverse range, the CPU proceeds from step 320 to step 330 and determines, based on the driver information described above, whether the driver's face is facing forward or not. For example, the CPU determines that the driver's face is facing forward if the driver's face is within a 90-degree range from the vehicle's longitudinal axis in the left-right direction. If the driver's face is not facing forward (i.e., the driver is facing backward), the CPU proceeds directly from step 330 to step 370, which will be described later.

[0034] In contrast, if the driver's face is facing forward (i.e., a mismatch occurs where the starting direction and the driver's face direction do not match), the CPU proceeds from step 330 to step 340 to determine whether the driver's gaze direction is toward the monitor 81 (monitor direction). If the driver's gaze direction is not toward the monitor direction, the CPU considers this to be an abnormal starting operation and proceeds from step 340 to step 350, sets the value of the abnormal starting operation flag Xijo to "1", and then proceeds to step 395 to terminate this routine.

[0035] In contrast, if the driver's gaze is directed towards the monitor, the CPU proceeds from step 340 to step 360 to determine whether or not an image of the vehicle's surroundings is displayed on the monitor 81. If an image of the vehicle's surroundings is not displayed, the CPU determines that it is an abnormal starting operation and proceeds from step 360 to step 350, sets the value of the starting operation abnormality flag Xijo to "1", and then proceeds to step 395.

[0036] In contrast, if an image of the area around the vehicle is displayed, the CPU proceeds from step 360 to step 370, considering it a normal starting operation, sets the value of the starting operation abnormality flag Xijo to "0", and then proceeds to step 395. Note that the value of the starting operation abnormality flag Xijo is set to "0" when the vehicle stops.

[0037] Furthermore, if at least one of the conditions determined in step 310 and step 320 is not met, the CPU proceeds directly from step 310 or step 320 to step 395.

[0038] <Detection of abnormal starting operation when the starting direction is forward> When an appropriate time arrives, the CPU starts processing from step 400 in Figure 4 and proceeds to step 410 to determine whether the vehicle speed SPD is less than or equal to the first vehicle speed threshold SPD1th. If the vehicle speed SPD is less than or equal to the first vehicle speed threshold SPD1th, the CPU proceeds from step 410 to step 420 to determine whether the shift range is in the forward range (whether the vehicle is moving in the forward direction) based on the signal from the shift position sensor 94.

[0039] If the shift range is in the forward range, the CPU proceeds from step 420 to step 430 and determines, based on the driver information described above, whether the driver's face is facing forward or not. If the driver's face is not facing forward (i.e., a mismatch occurs), since the monitor 81 is in front of the driver, it is unlikely that the driver's line of sight is towards the monitor. In other words, there is a risk of an abnormal start operation occurring. Therefore, the CPU proceeds from step 430 to step 440 and sets the value of the start operation abnormality flag Xijo to "1", and then proceeds to step 495 to terminate this routine.

[0040] In contrast, if the driver's face is facing forward, the CPU proceeds from step 430 to step 450 to determine whether the driver's gaze is directed towards the monitor. If the driver's gaze is not directed towards the monitor, the driver is looking directly ahead at the vehicle. In this case, the CPU considers this to be a normal starting operation and proceeds directly to step 470, which will be described later.

[0041] In contrast, if the driver's gaze is directed towards the monitor, the CPU proceeds from step 450 to step 460 to determine whether or not an image of the vehicle's surroundings is displayed on the monitor 81. If an image of the vehicle's surroundings is not displayed, the CPU determines that it is an abnormal starting operation and proceeds from step 460 to step 440 to set the value of the starting operation abnormality flag Xijo to "1", and then proceeds to step 495.

[0042] In contrast, if an image of the area around the vehicle is displayed, the CPU considers it a normal starting operation and proceeds from step 460 to step 470, sets the value of the starting operation abnormality flag Xijo to "0", and then proceeds to step 495.

[0043] If at least one of the conditions determined in step 410 and step 420 is not met, the CPU proceeds directly from step 410 or step 420 to step 495.

[0044] <Start suppression control> Furthermore, when an appropriate time arrives, the CPU starts processing from step 500 in Figure 5 and proceeds to step 510, where it determines whether the value of the launch suppression control flag Xyk is "0". In other words, the CPU determines whether or not launch suppression control is currently being performed.

[0045] If the value of the launch suppression control flag Xyk is "0", the CPU proceeds from step 510 to step 520 to determine whether the vehicle speed SPD is less than or equal to the "second vehicle speed threshold SPD2th, which is higher than the first vehicle speed threshold SPD1th". If the vehicle speed SPD is not less than or equal to the second vehicle speed threshold SPD2th, the CPU proceeds to step 595 to terminate this routine. On the other hand, if the vehicle speed SPD is less than or equal to the second vehicle speed threshold SPD2th, the CPU proceeds from step 520 to step 530 to determine whether the value of the launch operation abnormality flag Xijo is "1".

[0046] If the value of the starting operation abnormality flag Xijo is "1", the CPU proceeds from step 530 to step 540 and sets the value of the starting suppression control flag Xyk to "1" in order to execute starting suppression control. After that, the CPU proceeds to step 595 and terminates this routine. The CPU then executes a routine (not shown) and, if the value of the starting suppression control flag Xyk is "1", sends instructions to the powertrain ECU 60 and brake ECU 70 to execute the starting suppression control described above.

[0047] When the CPU proceeds to step 530, if the value of the starting operation abnormality flag Xijo is "0", the CPU proceeds from step 530 to step 550 to determine whether there has been a sudden depression of the accelerator pedal and whether there is a possibility of the vehicle colliding with an obstacle (i.e., whether the collision condition is met). Specifically, the CPU determines whether the increase in the amount of accelerator pedal operation AP per unit time is greater than or equal to a threshold increase, and whether there is an obstacle located in the direction of the vehicle's departure estimated from the shift range and at a distance less than or equal to a threshold distance from the vehicle, based on the amount of accelerator pedal operation AP, forward camera target information, radar target information, and sonar target information, etc. If the collision condition is met, the CPU proceeds from step 550 to step 540, and if the collision condition is not met, the CPU proceeds from step 550 to step 595.

[0048] If the CPU determines "No" in step 510, the CPU proceeds from step 510 to step 560 to determine whether the termination condition for the start suppression control has been met. This termination condition is met, for example, when the brake pedal operation amount BP is greater than or equal to the threshold BP (i.e., the brakes are being applied) and the vehicle speed SPD becomes "0". If the termination condition has not been met, the CPU proceeds from step 560 to step 595. If the termination condition has been met, the CPU proceeds from step 560 to step 570 to terminate the start suppression control by setting the value of the start suppression control flag Xyk to "0" and proceeding to step 595.

[0049] As explained above, the device DS executes a start suppression control when the above-mentioned mismatch occurs, but does not execute the start suppression control if it is presumed that the driver is paying attention to the vehicle surrounding information displayed on the monitor 81 even when the above-mentioned mismatch occurs. Therefore, the execution / non-execution (cancellation of execution) of the start suppression control can be appropriately controlled. The device DS is also applicable to vehicles that perform automatic driving control when predetermined conditions are met. In addition, the monitor 81 may be installed in multiple locations inside the vehicle, in which case a driver monitor camera is provided in a position that can determine whether or not the driver's gaze is directed towards each monitor. [Explanation of Symbols]

[0050] 10...Vehicle control ECU, 20...Forward camera system, 30...Forward radar system, 40...Ultrasonic sensor system, 50...PVM camera system, 60...Powertrain ECU, 70...Brake ECU, 81...Monitor (display), 94...Shift position sensor, 100...Driver monitoring system (driver monitor).

Claims

[Claim 1] A camera device that generates image data by capturing the area surrounding a vehicle using a camera equipped with an ultra-wide-angle lens, A surrounding conditions display device configured to display images of the vehicle's surroundings based on the aforementioned image data on a monitor inside the vehicle, A driver monitoring device that acquires driver information including the direction of the driver's gaze and the direction the driver's face is facing, A controller configured to perform a start suppression control to suppress the start of the vehicle when a mismatch occurs between the starting direction of the vehicle estimated based on the vehicle's shift range and the direction the driver is facing, Equipped with, The monitor is positioned in front of the driver when the driver is seated in the driver's seat of the vehicle. The aforementioned controller, Even if the aforementioned mismatch occurs, the system is configured to allow the start suppression control not to be executed if the driver's gaze direction is toward the monitor. In a vehicle control system, The aforementioned controller, When predetermined surrounding monitoring conditions are met, the system is configured to display an overhead view image and a forward-moving image of the vehicle, based on the image data, on the monitor as images of the area around the vehicle. The aforementioned forward-direction image is an image of the front of the vehicle when the vehicle's shift range is in the forward range, and an image of the rear of the vehicle when the vehicle's shift range is in the reverse range. Furthermore, the controller, One of the aforementioned discrepancies is when the vehicle starts in the reverse direction and the driver's face is facing forward. When the driver's line of sight is directed toward the monitor and the monitor is displaying an overhead view of the vehicle and an image of the area behind the vehicle as images of the area around the vehicle, the start suppression control is not executed. When the driver's line of sight is not directed toward the monitor, and when the monitor is not displaying the overhead view of the vehicle and the rear view of the vehicle as the vehicle surroundings image, the start suppression control is executed. As one of the aforementioned discrepancies, if the vehicle's starting direction is forward and the driver's face is not facing forward, the starting suppression control is executed. Furthermore, If the vehicle starts in the forward direction and the driver's face is facing forward, When the driver's line of sight is not directed toward the monitor, and when the driver's line of sight is directed toward the monitor and the monitor is displaying an overhead view of the vehicle and an image of the area in front of the vehicle as the vehicle surroundings image, the start suppression control is not executed. When the driver's line of sight is directed toward the monitor and the monitor is not displaying the overhead view of the vehicle or the image in front of the vehicle as the vehicle surroundings image, the start suppression control is executed. It is configured in such a way. Vehicle control system.

Images