Vehicle control device and vehicle control method

Abstract

To provide a vehicle control device which does not cancel start inhibition control even when vehicle peripheral information is shown on a monitor.SOLUTION: A vehicle control device (DS) comprises: a peripheral state display device capable of displaying a vehicle's surrounding image on a monitor (81) in a vehicle's interior; a driver monitoring device (100) which acquires driver information including a direction of a driver's line of sight and facial orientation of the driver; and a controller (10) capable of executing start inhibition control to prevent a vehicle from starting in an inconsistent situation where a vehicle's start direction estimated based on a shift range of the vehicle does not match the driver's facial orientation. The controller does not execute the start inhibition control when the driver's line of sight is directed at the monitor and the monitor displays the vehicle surrounding image even though the inconsistent situation occurs.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a vehicle control device and a vehicle control method that executes a start suppression control for suppressing the start of the vehicle depending on the situation when a driver performs an operation to start the vehicle. [Background technology]

[0002] One conventional vehicle control device (hereinafter referred to as the "conventional device") executes start suppression control to suppress the start of the vehicle when the driver's line of sight does not match the vehicle's starting direction determined based on the transmission range (shift range). Furthermore, in order to prevent unnecessary start suppression control from being executed, the conventional device cancels the start suppression control when the driver is notified of vehicle surroundings information acquired by a vehicle surroundings monitoring system (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-78960 A Summary of the Invention

[0004] However, for example, when the vehicle surroundings information is notified by displaying a vehicle surroundings image on a monitor (display), the driver is not necessarily looking at the monitor, so there are cases where it is undesirable to cancel the start suppression control.

[0005] One aspect of the present invention is a camera device (51-54) for capturing an image of a surrounding area of ​​the vehicle to generate image data; a surroundings condition display device (55, 10, 80) configured to be able to display a vehicle surroundings image based on the image data on a monitor (81) inside the vehicle; A driver monitoring device (100) that acquires driver information including a line of sight direction of a driver of the vehicle and a face direction that is a direction in which the face of the driver is facing; a controller (10, 60, 70) configured to execute a start suppression control for suppressing the start of the vehicle when a mismatch situation occurs in which a start direction of the vehicle estimated based on a shift range of the vehicle and a face direction of the driver do not match; Equipped with The controller is configured to allow the start suppression control not to be executed (step 370) even if the inconsistency situation occurs, if the driver's line of sight is toward the monitor (Yes in each of steps 320, 330, and 340).

[0006] More specifically, when the inconsistency situation occurs but the driver's line of sight is toward the monitor, the controller does not execute the departure inhibition control when the monitor is displaying the vehicle surroundings image, and executes the departure inhibition control when the monitor is not displaying the vehicle surroundings image.

[0007] According to the aspect of the present invention, when a mismatch occurs in which the starting direction of the vehicle and the direction of the driver's face do not match, the start suppression control is not executed when it is estimated that the driver is looking at the vehicle surroundings image displayed on the monitor. Therefore, it is possible to avoid a situation in which the start suppression control is not executed when the driver performs a starting operation in a situation in which the driver is not looking at either the starting direction of the vehicle or the vehicle surroundings image displayed on the monitor, and a situation in which the start suppression control is executed when it is considered that the start suppression control is unnecessary because the driver is looking at the vehicle surroundings image displayed on the monitor, causing the driver to feel uncomfortable.

[0008] The present invention also covers a vehicle control method and a program thereof implemented by the vehicle control device. Furthermore, in the above description, in order to facilitate understanding of the invention, the reference symbols used in the embodiments are enclosed in parentheses with respect to the constituent elements of the invention corresponding to the embodiments. However, each constituent element of the present invention is not limited to the embodiment defined by the reference symbols. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic configuration diagram of a vehicle control device according to an embodiment of the present invention; [Diagram 2] 2 is an example of a vehicle surroundings image displayed on a monitor (display). [Diagram 3] 4 is a flowchart showing a routine executed by a CPU of a vehicle control ECU. [Figure 4] 4 is a flowchart showing a routine executed by a CPU of a vehicle control ECU. [Diagram 5] 4 is a flowchart showing a routine executed by a CPU of a vehicle control ECU. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (composition) A vehicle control device DS (hereinafter referred to as "device DS") according to an embodiment of the present invention includes the components (camera, sensor, ECU, actuator, etc.) shown in Fig. 1. The device DS is applied to (mounted on) a vehicle.

[0011] In this specification, an "ECU" is an electronic control unit having a microcomputer including a CPU (processor) and memory as its main part, and is also called a controller. The components shown in Fig. 1, including multiple ECUs, are connected to each other via a CAN (Controller Area Network) so that they can exchange information with each other.

[0012] The vehicle control ECU 10 executes "start suppression control as vehicle control" which will be described later.

[0013] The front camera device 20 includes a front camera 21 and an image ECU 22. The front camera 21 captures an image of a "scene in front of the vehicle (including the road surface and targets)" at predetermined time intervals to obtain front camera image data. The image ECU 22 analyzes the front camera image data from the front camera 21 to generate front camera target information including the "position, relative longitudinal speed, relative lateral speed, and type" of the target in the front camera image data, and transmits the information to the vehicle control ECU 10.

[0014] The forward radar device 30 is a well-known device that acquires information about a target present in front of the vehicle using millimeter-wave band radio waves, and includes a forward radar 31 and a forward radar ECU 32. The forward radar 31 transmits information about the transmitted radio waves and information about the received reflected waves to the forward radar ECU 32 every time a predetermined time elapses. The forward radar ECU 32 acquires "forward radar target information," which is target information about targets present within the detection range of the forward radar 31, based on the information from the forward radar 31, and transmits the information to the vehicle control ECU 10. The forward radar target information includes the distance to the target, the direction of the target, and the relative speed of the target, etc.

[0015] The ultrasonic sensor device 40 includes a plurality of ultrasonic sensors (clearance sonar) 41 and a sonar ECU 42. Each of the plurality of ultrasonic sensors 41 is disposed at a predetermined position around the vehicle body, and emits ultrasonic waves and receives ultrasonic waves reflected by an object. The sonar ECU 42 acquires information about targets present around the vehicle as "sonar target information" based on "information about the timing of emission of ultrasonic waves and the timing of reception of reflected ultrasonic waves" from each ultrasonic sensor 41, and transmits the information to the vehicle control ECU 10. The sonar target information includes the position of the target relative to the vehicle and the distance between the vehicle body and the target.

[0016] The PVM (Panoramic View Monitor) camera device 50 includes a front PVM camera 51, a left PVM camera 52, a right PVM camera 53, a rear PVM camera 54, and a PVM ECU 55. Each of the PVM cameras 51-54 includes an ultra-wide-angle lens.

[0017] The front PVM camera 51, left PVM camera 52, right PVM camera 53 and rear PVM camera 54 capture images of the scenes in front and on the sides of the vehicle, the scenes to the left of the vehicle, the scenes to the right of the vehicle and the scenes behind the vehicle at predetermined intervals to obtain image data.

[0018] The PVM·ECU 55 generates data of an overhead image of the vehicle and data of an image in the direction of travel based on the image data transmitted from the PVM cameras 51-54, and transmits the data to the vehicle control ECU 10. Note that the structure of the PVM cameras 51-54, the overhead image, and the image in the direction of travel are well known (see, for example, JP 2022-86516 A, JP 2020-117128 A, and JP 2019-016825 A, etc.).

[0019] When a predetermined periphery monitoring condition is satisfied, the vehicle control ECU 10 transmits "data of the overhead image and the forward direction image" from the PVM·ECU 55 to the display ECU 80, and the display ECU 80 displays "the overhead image and the forward direction image" based on the image data on the monitor 81 as "images for monitoring the situation around the vehicle (vehicle periphery image)" (see FIG. 2). The predetermined periphery monitoring condition is satisfied, for example, when the vehicle speed SPD is equal to or lower than a predetermined periphery monitoring vehicle speed threshold and the vehicle periphery monitoring function is turned on by the driver's touch operation on the monitor 81. When the periphery monitoring condition is satisfied and the shift range obtained from the shift position sensor 94 described later is the forward range or another range, the forward direction image is an image of the vehicle's front based on image data acquired by the forward PVM camera 51. When the periphery monitoring condition is satisfied and the shift range obtained from the shift position sensor 94 described later is the reverse range, the forward direction image is an image of the vehicle's rear based on image data acquired by the rear PVM camera 54. If the surroundings monitoring condition is not satisfied, the display ECU 80 displays on the monitor 81 "an image other than the vehicle surroundings image (for example, a map image)."

[0020] The powertrain ECU 60 is connected to a powertrain actuator 61 so as to be capable of driving the powertrain actuator 61. The actuator 61 can change the torque generated by a drive device of the vehicle (a drive power source of the host vehicle). The torque generated by the drive device is transmitted to the drive wheels via a gear mechanism. Therefore, the powertrain ECU 60 can control the drive power of the vehicle.

[0021] If the drive device of the vehicle is a gasoline-fueled internal combustion engine, the actuator 61 is, for example, a throttle valve actuator that changes the opening of a throttle valve. The vehicle may be an electric vehicle, in which case the actuator 61 is an inverter that can change the torque of an electric motor. Furthermore, the vehicle may be a hybrid vehicle, in which case the actuator 61 includes an inverter for the electric motor and a throttle valve actuator for the internal combustion engine.

[0022] The brake ECU 70 drives the brake actuator 71 to control friction brake devices (braking devices) provided on each wheel of the vehicle, thereby changing the braking force (friction braking force) applied to the vehicle. Thus, the brake ECU 70 is capable of controlling the braking force.

[0023] The display ECU 80 can display a predetermined image on a monitor (display) 81. The monitor 81 is fixed in a predetermined position (for example, a center cluster portion) in front of the driver's seat of the vehicle (i.e., in front of the driver sitting in the driver's seat) inside the vehicle. Therefore, when the driver is looking at the monitor 81, the direction in which the driver's face is facing is the front of the vehicle.

[0024] The vehicle control ECU 10 is further connected to sensors described below, and receives output values ​​(detection values) of these sensors. A vehicle speed sensor 91 for detecting the vehicle speed (i.e., vehicle speed SPD). An accelerator pedal operation amount sensor 92 that detects the amount of operation AP of the accelerator pedal of the vehicle. A brake pedal operation amount sensor 93 detects an operation amount BP of a brake pedal of the vehicle. A shift position sensor 94 detects the position (shift range) of the vehicle's shift lever. In this example, the shift position sensor 94 outputs a signal indicating whether the shift range is a forward range (D) for moving the vehicle forward, a reverse range (R) for moving the vehicle backward, or another range (N·P). The vehicle control ECU 10 is also connected to other sensors that indicate the driving state of the vehicle.

[0025] The driver monitoring device (driver monitor) 100 is a device that acquires information (driver information) representing the state of the driver of the vehicle (including the driver's line of sight and the driver's facial direction), and includes a driver monitor camera 101 and a driver monitor ECU 102. The driver monitoring device 100 itself is well known and is disclosed, for example, in Japanese Patent Application Publication No. 2019-87143, Japanese Patent Application Publication No. 2019-87029, Japanese Patent Application Publication No. 2016-38866, and Japanese Patent Application Publication No. 2013-152700.

[0026] The driver monitor camera 101 is disposed at a suitable position (e.g., above the steering column) in front of the driver's seat of the vehicle, and captures the face of the driver at predetermined intervals to generate facial image data. The driver monitor ECU 102 acquires the driver information based on the facial image data transmitted from the driver monitor camera 101, and transmits it to the vehicle control ECU 10.

[0027] In short, the driver monitor ECU 102 generates a facial image based on facial image data from the driver monitor camera 101, rotates the generated facial image, and acquires the direction of the driver's face based on the rotation angle of the facial image at which the match rate between the rotated facial image and pre-stored "facial shape data when the driver is facing forward" is maximized.

[0028] The driver monitor ECU 102 identifies a face area from the generated face image of the driver and detects face parts by extracting feature points of the face parts such as the eyes, nose, and mouth. Furthermore, the driver monitor ECU 102 detects the position of the Purkinje image (corneal reflection image) and the position of the pupil center, and obtains the positional relationship between the Purkinje image and the pupil center. Then, the driver monitor ECU 102 obtains the line of sight of the driver based on the positional relationship between the Purkinje image and the pupil center and the direction of the driver's face.

[0029] (Overview of operation) The device DS judges whether the driver is performing an abnormal starting operation based on the starting direction of the vehicle estimated based on the shift range, the driver's line of sight direction, the driver's face direction, and whether the vehicle surroundings image is displayed on the monitor 81, and executes or cancels the execution of the starting inhibition control based on the result. The starting inhibition control is a control that maintains the driving force of the vehicle at a driving force required for creeping (driving force for creeping) by, for example, setting the opening degree of the throttle valve to zero, regardless of the accelerator pedal operation amount AP. In other words, the starting inhibition control is a control that generates a driving force smaller than the normal driving force determined by the accelerator pedal operation amount AP. The starting inhibition control may further include a braking force control that applies a braking force to the vehicle until the vehicle is stopped or the vehicle speed is reduced to an extremely slow creep vehicle speed or lower.

[0030] (Specific operation) The CPU of the vehicle control ECU 10 (hereinafter simply referred to as "CPU") executes the routines shown in the flowcharts of FIGS. 3 to 5 every time a predetermined time elapses.

[0031] <Determining abnormal starting operation when the starting direction is reverse> When an appropriate time arrives, the CPU starts the process from step 300 in Fig. 3 and proceeds to step 310 to determine whether the vehicle speed SPD is equal to or less than a first vehicle speed threshold SPD1th. The first vehicle speed threshold SPD1th is set to an extremely small predetermined value so that it is possible to determine whether the vehicle is in a state before starting.

[0032] If the vehicle speed SPD is equal to or less than the first vehicle speed threshold SPD1th, the CPU proceeds from step 310 to step 320, and determines based on the signal from the shift position sensor 94 whether the shift range is the reverse range (whether the starting direction of the vehicle is the reverse direction).

[0033] When the shift range is the reverse range, the CPU proceeds from step 320 to step 330 and determines whether the driver's face is facing forward based on the driver information described above. For example, the CPU determines that the driver's face is facing forward when the driver's face is within a range of 90 degrees to the left or right from the longitudinal axis of the vehicle. If the driver's face is not facing forward (i.e., the driver is facing backward), the CPU proceeds directly from step 330 to step 370 described later.

[0034] On the other hand, if the driver's face is facing forward (i.e., if a mismatch occurs between the starting direction and the face direction), the CPU proceeds from step 330 to step 340 and determines whether the driver's line of sight is toward the monitor 81 (monitor direction). If the driver's line of sight is not in the monitor direction, the CPU proceeds from step 340 to step 350 as an abnormal starting operation and sets the value of the starting operation abnormal flag Xijo to "1", then proceeds to step 395 and ends this routine.

[0035] On the other hand, if the driver's line of sight is in the monitor direction, the CPU proceeds from step 340 to step 360 and determines whether or not a vehicle surroundings image is displayed on the monitor 81. If the vehicle surroundings image is not displayed, the CPU determines that an abnormal starting operation has occurred and proceeds from step 360 to step 350, sets the value of the starting operation abnormality flag Xijo to "1", and then proceeds to step 395.

[0036] On the other hand, if the vehicle surroundings image is displayed, the CPU determines that the starting operation is normal, proceeds from step 360 to step 370, sets the value of the starting operation abnormality flag Xijo to "0", and then proceeds to step 395. The value of the starting operation abnormality flag Xijo is set to "0" when the vehicle stops.

[0037] Furthermore, if at least one of the conditions determined in step 310 and step 320 is not satisfied, the CPU proceeds directly from step 310 or step 320 to step 395.

[0038] <Determining abnormal starting operation when the starting direction is forward> 4, and proceeds to step 410 to determine whether or not the vehicle speed SPD is equal to or less than the first vehicle speed threshold SPD1th. If the vehicle speed SPD is equal to or less than the first vehicle speed threshold SPD1th, the CPU proceeds from step 410 to step 420 to determine whether or not the shift range is a forward range (whether or not the starting direction of the vehicle is a forward direction) based on a signal from the shift position sensor 94.

[0039] If the shift range is the forward range, the CPU proceeds from step 420 to step 430 and determines whether the driver's face is facing forward based on the driver information described above. If the driver's face is not facing forward (i.e., if a mismatch occurs), the monitor 81 is in front of the driver, so there is no possibility that the driver's line of sight is in the monitor direction. In other words, there is a risk of an abnormal starting operation occurring. Therefore, the CPU proceeds from step 430 to step 440 and sets the value of the starting operation abnormality flag Xijo to "1", and then proceeds to step 495 to end this routine.

[0040] On the other hand, if the driver's face is facing forward, the CPU proceeds from step 430 to step 450 and determines whether the driver's line of sight is in the monitor direction. If the driver's line of sight is not in the monitor direction, the driver is gazing ahead of the vehicle. In this case, the CPU determines that this is a normal starting operation and proceeds directly to step 470, which will be described later.

[0041] On the other hand, if the driver's line of sight is in the monitor direction, the CPU proceeds from step 450 to step 460 and determines whether or not a vehicle surroundings image is displayed on the monitor 81. If the vehicle surroundings image is not displayed, the CPU determines that this is an abnormal starting operation, proceeds from step 460 to step 440, sets the value of the starting operation abnormality flag Xijo to "1", and then proceeds to step 495.

[0042] On the other hand, if the vehicle surroundings image is being displayed, the CPU determines that the starting operation is normal, proceeds from step 460 to step 470, sets the value of the starting operation abnormality flag Xijo to “0”, and then proceeds to step 495.

[0043] If at least one of the conditions determined in step 410 and step 420 is not satisfied, the CPU proceeds directly from step 410 or step 420 to step 495.

[0044] <Start suppression control> Furthermore, when an appropriate time arrives, the CPU starts the process from step 500 in Fig. 5 and proceeds to step 510 to determine whether the value of the start suppression control flag Xyk is "0." That is, the CPU determines whether the start suppression control is not currently being executed.

[0045] If the value of the start suppression control flag Xyk is "0", the CPU proceeds from step 510 to step 520 and determines whether the vehicle speed SPD is equal to or less than a "second vehicle speed threshold SPD2th higher than the first vehicle speed threshold SPD1th". If the vehicle speed SPD is not equal to or less than the second vehicle speed threshold SPD2th, the CPU proceeds to step 595 and temporarily ends this routine. On the other hand, if the vehicle speed SPD is equal to or less than the second vehicle speed threshold SPD2th, the CPU proceeds from step 520 to step 530 and determines whether the value of the start operation abnormality flag Xijo is "1".

[0046] If the value of the starting operation abnormality flag Xijo is "1", the CPU proceeds from step 530 to step 540, and sets the value of the starting inhibition control flag Xyk to "1" in order to execute the starting inhibition control. After that, the CPU proceeds to step 595 and ends this routine for the time being. Note that the CPU executes a routine not shown, and if the value of the starting inhibition control flag Xyk is "1", it transmits instructions to the powertrain ECU 60 and the brake ECU 70 to execute the above-mentioned starting inhibition control.

[0047] When the CPU proceeds to step 530, if the value of the starting operation abnormality flag Xijo is "0", the CPU proceeds from step 530 to step 550 to determine whether the accelerator pedal has been suddenly depressed and whether there is a possibility that the vehicle will collide with an obstacle (i.e., whether the collision condition is established). Specifically, the CPU determines whether the increase in the accelerator pedal operation amount AP per unit time is equal to or greater than a threshold increase amount, and whether an obstacle is present in the starting direction of the vehicle estimated from the shift range and at a position less than a threshold distance from the vehicle, based on the accelerator pedal operation amount AP, front camera target information, radar target information, sonar target information, etc. Then, if the collision condition is established, the CPU proceeds from step 550 to step 540, and if the collision condition is not established, the CPU proceeds from step 550 to step 595.

[0048] If the CPU determines "No" in step 510, the CPU proceeds from step 510 to step 560 and determines whether or not a termination condition for the start inhibition control is satisfied. This termination condition is satisfied, for example, when the brake pedal operation amount BP is equal to or greater than the threshold value BP (i.e., the brakes are being operated) and the vehicle speed SPD becomes "0". If the termination condition is not satisfied, the CPU proceeds from step 560 to step 595. If the termination condition is satisfied, the CPU proceeds from step 560 to step 570, sets the value of the start inhibition control flag Xyk to "0" to terminate the start inhibition control, and proceeds to step 595.

[0049] As described above, the device DS executes the start inhibition control when the above-mentioned inconsistent situation occurs, but does not execute the start inhibition control when it is estimated that the driver is watching the vehicle surroundings information displayed on the monitor 81 even when the above-mentioned inconsistent situation occurs. Therefore, it is possible to appropriately execute or not execute the start inhibition control (cancel execution). The device DS is also applicable to a vehicle that executes automatic driving control when a predetermined condition is satisfied. The monitors 81 may be provided at multiple locations in the vehicle cabin, and in that case, a driver monitor camera is provided at a position where it can be determined whether the driver's line of sight is directed toward each monitor. [Explanation of symbols]

[0050] 10...vehicle control ECU, 20...front camera device, 30...front radar device, 40...ultrasonic sensor device, 50...PVM camera device, 60...power train ECU, 70...brake ECU, 81...monitor (display), 94...shift position sensor, 100...driver monitoring device (driver monitor).

Claims

1. a camera device that captures an image of a surrounding area of ​​a vehicle to generate image data; a surroundings situation display device configured to be able to display a vehicle surroundings image based on the image data on a monitor inside the vehicle; A driver monitoring device that acquires driver information including a line of sight direction of a driver of the vehicle and a face direction that is a direction in which the face of the driver is facing; A controller configured to execute a start suppression control for suppressing the start of the vehicle when a mismatch situation occurs in which a start direction of the vehicle estimated based on a shift range of the vehicle and a face direction of the driver do not match; In a vehicle control device comprising: The controller: Even when the inconsistency occurs, when the driver's line of sight is toward the monitor, the start suppression control is allowed not to be executed. Vehicle control device.

2. The vehicle control device according to claim 1, The controller: If the inconsistency situation occurs but the driver's line of sight is toward the monitor, The start suppression control is not executed when the monitor displays the vehicle surroundings image, The start suppression control is executed when the monitor is not displaying the vehicle surroundings image. It was configured as follows: Vehicle control device.

3. The vehicle control device according to claim 1, the monitor is disposed in front of the driver when the driver is seated in the driver's seat of the vehicle, The controller: As one of the inconsistent situations, when a situation occurs in which the starting direction of the vehicle is a backward direction and the facing direction of the driver is forward, When the driver's line of sight is directed toward the monitor and the monitor is displaying the vehicle surroundings image, the start suppression control is not executed, When the driver's line of sight is not directed toward the monitor, or when the monitor does not display the vehicle surroundings image, the start suppression control is executed. It was configured as follows: Vehicle control device.

4. The vehicle control device according to claim 1, the monitor is disposed in front of the driver when the driver is seated in the driver's seat of the vehicle, The controller: When a situation occurs in which the starting direction of the vehicle is a forward direction and the facing direction of the driver is not forward, as one of the inconsistent situations, the starting suppression control is executed, When the starting direction of the vehicle is a forward direction and the driver's face is facing forward, When the driver's line of sight is not directed toward the monitor, or when the driver's line of sight is directed toward the monitor and the monitor is displaying the vehicle surroundings image, the start suppression control is not executed, When the driver's line of sight is directed toward the monitor and the monitor is not displaying the vehicle surroundings image, the start suppression control is executed. It was configured as follows: Vehicle control device.

5. When the starting direction of the vehicle estimated based on the shift range of the vehicle is a backward direction and the driver's face is facing forward, When the driver's line of sight is not directed toward a monitor disposed in front of a driver's seat within the vehicle cabin, or when an image of a surrounding situation of the vehicle is not displayed on the monitor, a start suppression control is executed to suppress the start of the vehicle, When the driver's line of sight is directed toward the monitor and an image of the surrounding situation of the vehicle is displayed on the monitor, the start suppression control is not executed. A vehicle control method.

Images