Vehicle control method and vehicle control system

Abstract

To appropriately check whether a driver is awake or not during automatic driving of a vehicle.SOLUTION: A vehicle comprises a first visual device and a second visual device. A forward direction is a direction from a driver's seat where a driver of a vehicle sits toward a front of the vehicle. A first direction is a direction from the driver's seat toward the first visual device. A second direction is a direction from the driver's seat toward the second visual device. A first angle between the first direction and the forward direction is greater than a second angle between the second direction and the forward direction. The vehicle control method includes: issuing a first visual notification from the first visual device when checking whether the driver is awakening during automatic driving of the vehicle; and determining that the driver is awakening to continue the automatic driving when the driver reacts to the first visual notification.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a technology for checking whether a driver is awake during automatic driving of a vehicle. [Background technology]

[0002] Patent Document 1 discloses a vehicle control system. The vehicle control system estimates the level of wakefulness of a driver in a vehicle. When the level of wakefulness decreases, the vehicle control system increases the task load required of the driver. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 130552 Summary of the Invention [Problem to be solved by the invention]

[0004] Even during automatic driving of the vehicle, the driver may be required to monitor the surrounding environment of the vehicle. Therefore, the system may check whether the driver is monitoring the surrounding environment of the vehicle. In other words, the system may check whether the driver is awake. For example, in order to confirm that the driver is awake, the system may request the driver to perform a task such as a switch operation. However, if the driver is required to perform an excessive task, the driver may not be able to monitor the surrounding environment during that time. In addition, if the driver is required to perform an excessive task, the driver may not be able to perform a driving operation (driving behavior) immediately.

[0005] One object of the present disclosure is to provide a technology that can appropriately check whether a driver is awake during automatic driving of a vehicle. [Means for solving the problem]

[0006] In one aspect, a technique for controlling a vehicle is provided. The vehicle includes a first visual device and a second visual device. The forward direction is a direction from a driver's seat where a driver of the vehicle sits toward the front of the vehicle. The first direction is a direction from the driver's seat to the first visual device. The second direction is a direction from the driver's seat to the second visual device. A first angle between the first direction and the forward direction is greater than a second angle between the second direction and the forward direction. The vehicle control method includes: When checking whether a driver is awake during automatic driving of a vehicle, issuing a first visual notification from a first visual device; If the driver responds to the first visual notification, the system determines that the driver is awake and allows the vehicle to continue autonomously. Includes. Effect of the Invention

[0007] In the driver wakefulness check process, a first visual notification is issued from the first visual device. If the driver responds to the first visual notification, the driver is determined to be awake. The driver's response to the first visual notification is a change in line of sight or facial direction. Since the driver does not need to press a switch, the driver's monitoring of the surrounding environment is not hindered. In addition, since the driver does not need to take his / her hands off the steering wheel, the driver's instantaneous driving operation is not hindered. These are desirable from the viewpoint of ensuring safety during automated driving.

[0008] In addition, the first visual device that issues the first visual notification is relatively far from the forward direction as viewed from the driver's seat where the driver sits. Therefore, the line of sight of the driver who responds to the first visual notification is guided in a direction away from the forward direction. In other words, the field of vision of the driver who responded to the first visual notification also includes the surrounding environment in a direction relatively far from the forward direction. This means that the driver's monitoring of the surrounding environment is promoted. In other words, the driver wakefulness check process according to the present disclosure can check whether the driver is awake and at the same time actively guide the driver's line of sight to the surrounding environment to promote monitoring of the surrounding environment. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram for explaining an overview of a vehicle and a vehicle control system according to an embodiment; [Diagram 2] FIG. 2 is a block diagram showing a configuration related to a notification process according to an embodiment. [Diagram 3] 1 is a diagram showing an example of a plurality of visual devices mounted on a vehicle according to an embodiment; [Figure 4] FIG. 2 is a conceptual diagram for explaining the positional relationship between a first visual device and a second visual device in the embodiment. [Diagram 5] 5 is a flowchart showing a process related to a driver wakefulness check process according to the embodiment; [Figure 6] FIG. 4 is a conceptual diagram for explaining a driver wakefulness check process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0011] 1. Vehicle Control System FIG. 1 is a diagram for explaining an overview of a vehicle 1 and a vehicle control system 10 according to this embodiment. The vehicle 1 has an automatic driving function. The automatic driving function automatically performs at least one of steering, acceleration, and deceleration. An example of an automatic driving level is one that assumes that the driver does not necessarily have to concentrate 100% on driving (so-called level 3 or higher). A driver 3 sits in the driver's seat 2 of the vehicle 1. Note that even during automatic driving, the operator sitting in the driver's seat 2 is called the driver 3.

[0012] The vehicle control system 10 is mounted on the vehicle 1 and controls the vehicle 1. For example, the vehicle control system 10 controls automatic driving of the vehicle 1. In addition, the vehicle control system 10 issues various notifications to a driver 3 sitting in a driver's seat 2.

[0013] For example, the vehicle control system 10 includes a visual device 20 , a speaker 30 , a driver monitor 40 , a sensor group 50 , a driving device 60 , and a control device 100 .

[0014] The visual device 20 issues a visual notification to the driver 3. Examples of the visual device 20 include a light-emitting device and a display panel. An example of the light-emitting device is an LED (Light Emitting Diode). The light-emitting device can provide (express) different information by emitting light in different colors. Examples of the display panel include a liquid crystal panel, an organic EL panel, and the like. Typically, the visual device 20 is installed in the interior of the vehicle 1.

[0015] The speaker 30 issues an audio notification to the driver 3. Typically, the speaker 30 is installed inside the vehicle 1.

[0016] The driver monitor 40 is a device for detecting the state and operation of the driver 3. For example, the driver monitor 40 includes a camera installed at a position where the driver 3 can be imaged. By analyzing the image obtained by the camera, the driver 3's facial direction, line of sight, eye opening degree, mouth opening degree, etc. can be detected. The driver monitor 40 may include a steering touch sensor installed on a steering wheel operated by the driver 3. The driver monitor 40 may detect the driving operation by the driver 3.

[0017] The sensor group 50 includes a recognition sensor, a vehicle state sensor, a position sensor, etc. The recognition sensor recognizes (detects) the situation around the vehicle 1. Examples of the recognition sensor include a camera, a LIDAR (Laser Imaging Detection and Ranging), a radar, etc. The vehicle state sensor detects the state of the vehicle 1. The vehicle state sensor includes a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, etc. The position sensor detects the position and orientation of the vehicle 1. For example, the position sensor includes a GNSS (Global Navigation Satellite System).

[0018] The traveling device 60 includes a steering device, a drive device, and a braking device. The steering device steers the wheels. For example, the steering device includes an electric power steering (EPS) device. The drive device is a power source that generates a driving force. Examples of the drive device include an engine, an electric motor, and an in-wheel motor. The braking device generates a braking force.

[0019] The control device 100 is a computer that controls the vehicle 1. The control device 100 includes one or more processors 110 (hereinafter simply referred to as processor 110) and one or more storage devices 120 (hereinafter simply referred to as storage devices 120). The processor 110 executes various processes. For example, the processor 110 includes a CPU (Central Processing Unit). The storage device 120 stores various information. Examples of the storage device 120 include a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like.

[0020] The vehicle control program 130 is a computer program executed by the processor 110. The functions of the control device 100 are realized by cooperation between the processor 110, which executes the vehicle control program 130, and the storage device 120. The vehicle control program 130 is stored in the storage device 120. Alternatively, the vehicle control program 130 may be recorded in a computer-readable recording medium.

[0021] The control device 100 acquires driving environment information 150 indicating the driving environment of the vehicle 1 using the sensor group 50. The driving environment information 150 is stored in the storage device 120.

[0022] The driving environment information 150 includes surrounding situation information indicating the recognition result by the recognition sensor. For example, the surrounding situation information includes an image captured by a camera. The surrounding situation information may include object information regarding objects around the vehicle 1. Examples of objects around the vehicle 1 include pedestrians, other vehicles (preceding vehicles, parked vehicles, etc.), white lines, traffic lights, signs, roadside structures, etc. The object information indicates the relative position and relative speed of the object with respect to the vehicle 1.

[0023] The driving environment information 150 also includes vehicle state information indicating the vehicle state detected by the vehicle state sensor.

[0024] Furthermore, the driving environment information 150 includes vehicle position information indicating the position and direction of the vehicle 1. The vehicle position information is obtained by a position sensor. Highly accurate vehicle position information may be obtained by a self-position estimation process (localization) using map information and surrounding situation information (object information).

[0025] Furthermore, the driving environment information 150 includes information obtained by the driver monitor 40 .

[0026] The control device 100 executes vehicle driving control for controlling the driving of the vehicle 1. The vehicle driving control includes steering control, drive control, and braking control. The control device 100 executes the vehicle driving control by controlling the driving device 60 (the steering device, the drive device, and the braking device).

[0027] Furthermore, the control device 100 may perform automatic driving control based on the driving environment information 150. For example, the control device 100 generates a driving plan for the vehicle 1 based on the driving environment information 150. Furthermore, the control device 100 generates a target trajectory required for the vehicle 1 to drive according to the driving plan based on the driving environment information 150. The target trajectory includes a target position and a target speed. Then, the control device 100 performs vehicle driving control so that the vehicle 1 follows the target trajectory.

[0028] Furthermore, the control device 100 performs a "notification process" for issuing various notifications to the driver 3 as necessary. The notification process will be described in detail below.

[0029] 2. Notification Processing 2 is a block diagram showing a configuration related to the notification process. The control device 100 issues a notification to the driver 3 when a predetermined notification condition is met.

[0030] For example, the notification condition is that during the automatic driving of the vehicle 1, it becomes necessary to check whether the driver 3 is awake. In other words, the notification condition is that during the automatic driving of the vehicle 1, it becomes necessary to check whether the driver 3 is monitoring the surrounding environment of the vehicle 1. As another example, the notification condition is that the wakefulness of the driver 3 falls below a threshold. As yet another example, the notification condition is that during the automatic driving of the vehicle 1, it becomes necessary to request the driver 3 to hold the steering wheel. As yet another example, the notification condition is that during the automatic driving of the vehicle 1, it becomes necessary to request the driver 3 to switch to manual driving.

[0031] The types of notifications to the driver 3 include "visual notification NV" and "audio notification NA." Visual notification NV is a notification by visual information such as light emission or display. Audio notification NA is a notification by audio information such as a voice message or buzzer. The control device 100 can issue the visual notification NV through the visual device 20. In addition, the control device 100 can issue the audio notification NA through the speaker 30.

[0032] The vehicle control system 10 may include multiple visual devices 20. For example, the vehicle control system 10 includes a first visual device 20-1 and a second visual device 20-2. The first visual device 20-1 issues a first visual notification NV1 to the driver 3. The second visual device 20-2 issues a second visual notification NV2 to the driver 3.

[0033] 3 shows an example of a plurality of visual devices 20 mounted on a vehicle 1. The plurality of visual devices 20 include door illuminations 21 installed on the doors, pillar illuminations 22 installed on the window frames, instrument panel illuminations 23 installed on the instrument panel, and steering illuminations 24 installed on the steering wheel. Each illumination is a light-emitting device including a plurality of LEDs. Each illumination is arranged in a strip shape.

[0034] The forward direction D0 is the direction from the driver's seat 2 toward the front of the vehicle 1. In the example shown in Fig. 3, the door illumination 21L, the pillar illumination 22L, and the instrument panel illumination 23L are installed to the left of the forward direction D0. On the other hand, the door illumination 21R, the pillar illumination 22R, and the instrument panel illumination 23R are installed to the right of the forward direction D0. The steering illumination 24 is installed approximately in the forward direction D0.

[0035] FIG. 4 is a conceptual diagram for explaining the positional relationship between the first visual device 20-1 and the second visual device 20-2 according to this embodiment. The forward direction D0 is the direction from the driver's seat 2 toward the front of the vehicle 1. The first direction D1 is the direction from the driver's seat 2 toward the first visual device 20-1. The second direction D2 is the direction from the driver's seat 2 toward the second visual device 20-2. The first angle θ1 is the angle between the first direction D1 and the forward direction D0. The second angle θ2 is the angle between the second direction D2 and the forward direction D0. As shown in FIG. 4, the first angle θ1 is larger than the second angle θ2. Therefore, when the driver 3 sitting in the driver's seat 2 faces forward (forward direction D0), the first visual device 20-1 is far from the line of sight of the driver 3, and the second visual device 20-2 is close to the line of sight of the driver 3.

[0036] The human visual field is divided into an effective visual field (central visual field) and a peripheral visual field. A human can clearly perceive an object that is in the effective visual field. On the other hand, a human can vaguely perceive an object that is in the peripheral visual field. Preferably, when a driver 3 sitting in the driver's seat 2 faces forward (forward direction D0), the first visual device 20-1 is in the peripheral visual field of the driver 3, and the second visual device 20-2 is in the effective visual field (central visual field) of the driver 3.

[0037] Here, let us consider the average angle of a human's field of view. The average angle of a human's field of view is the average value of the angles of the fields of view of a certain number of humans, and is generally known. For example, the average human field of view is about 100 degrees to the left and right (total of 200 degrees), and the average human effective field of view is about 30 degrees to the left and right (total of 60 degrees). The reference angle α is half the angle of the average human effective field of view (e.g., 30 degrees). The limit angle β is half the angle of the average human field of view (e.g., 100 degrees). The first angle θ1 between the first direction D1 and the forward direction D0 may be larger than the reference angle α and smaller than the limit angle β. The second angle θ2 between the second direction D2 and the forward direction D0 may be smaller than the reference angle α.

[0038] In the example shown in Fig. 3, the door illumination 21L, the pillar illumination 22L, the instrument panel illumination 23L, and the door illumination 21R may belong to the first visual device 20-1, while the steering illumination 24 may belong to the second visual device 20-2.

[0039] 3. Driver Awareness Check Processing Even during automatic driving of the vehicle 1, the driver 3 may be required to monitor the surrounding environment of the vehicle 1. Therefore, the vehicle control system 10 (control device 100) may check whether the driver 3 is monitoring the surrounding environment of the vehicle 1. In other words, the vehicle control system 10 (control device 100) may check whether the driver 3 is awake or not. This process is hereinafter referred to as a "driver wakefulness check process." The driver wakefulness check is performed during automatic driving of the vehicle 1.

[0040] First, as a comparative example, consider a case where the driver 3 is requested to perform a switch operation in the driver awakening check process. In this case, while performing the switch operation, the driver 3 is unable to monitor the surrounding environment, and is also unable to perform instantaneous driving operations. In this way, when an excessive task is requested of the driver 3, there is a risk that the driver 3 will be unable to monitor the surrounding environment during that time. In addition, when an excessive task is requested of the driver 3, there is a risk that the driver 3 will be unable to perform instantaneous driving operations (driving actions).

[0041] Therefore, this embodiment proposes a technique for more appropriately performing the driver alertness check process while the vehicle 1 is being driven autonomously.

[0042] FIG. 5 is a flowchart showing the process related to the driver awakening check process according to the present embodiment.

[0043] 3-1. Step S110 In step S110, the control device 100 determines whether or not a driver awakening check process is necessary. For example, the driver awakening check process is periodically executed. Alternatively, the driver awakening check process may be executed in response to a predetermined trigger. Examples of triggers include a plan for the autonomous driving level to be lowered from level 3 to level 2, changing lanes, and the like. If the driver awakening check process is necessary (step S110; Yes), the process proceeds to step S120.

[0044] As an option, the control device 100 may consider whether the vehicle 1 is in a situation suitable for the driver awakening check process. A situation suitable for the driver awakening check process is a relatively relaxed situation. For example, when the road width on which the vehicle 1 is traveling is equal to or larger than a predetermined width, the execution of the driver awakening check process may be permitted. As another example, when the curvature of the road on which the vehicle 1 is traveling is less than a predetermined value, the execution of the driver awakening check process may be permitted. The width and curvature of the road are acquired based on the driving environment information 150 (surrounding situation information). Alternatively, the width and curvature of the road may be acquired based on map information and vehicle position information.

[0045] 3-2. Step S120 FIG. 6 is a conceptual diagram for explaining step S120. In step S120, the control device 100 issues a first visual notification NV1 from at least the first visual device 20-1. For example, if the first visual device 20-1 is a light-emitting device, the control device 100 causes the first visual device 20-1 to emit light or blink for a certain period of time. In the example shown in FIG. 3, the door illumination 21L, the pillar illumination 22L, the instrument panel illumination 23L, and the door illumination 21R may belong to the first visual device 20-1.

[0046] The control device 100 may issue the first visual notification NV1 from the first visual device 20-1 without issuing the second visual notification NV2 from the second visual device 20-2. The control device 100 may issue the first visual notification NV1 from the first visual device 20-1 without issuing an audio notification NA from the speaker 30.

[0047] 3-3. Step S130 In step S130 after step S120, the control device 100 determines whether the driver 3 has responded to the first visual notification NV1. For example, the response of the driver 3 is that the line of sight or the facial direction of the driver 3 approaches the first direction D1 toward the first visual device 20-1. The response of the driver 3 may be that the line of sight or the facial direction of the driver 3 approximately coincides with the first direction D1 toward the first visual device 20-1.

[0048] The presence or absence of a reaction of the driver 3 can be determined based on information obtained by the driver monitor 40. More specifically, the control device 100 detects the facial direction and line of sight of the driver 3 by analyzing an image obtained by the camera. Then, the control device 100 determines whether or not the driver 3 has reacted to the first visual notification NV1 based on the facial direction and line of sight of the driver 3.

[0049] If the driver 3 reacts to the first visual notification NV1 within a certain period of time after step S120 (step S130; Yes), the process proceeds to step S140. On the other hand, if the driver 3 does not react to the first visual notification NV1 within a certain period of time after step S120 (step S130; No), the process proceeds to step S150.

[0050] 3-4. Step S140 In step S140, the control device 100 determines that the driver 3 is awake. That is, the control device 100 determines that the driver 3 is monitoring the surrounding environment of the vehicle 1. In this case, the control device 100 continues the automatic driving of the vehicle 1.

[0051] 3-5. Step S150 In step S150, the control device 100 determines that the driver 3 is not monitoring the surrounding environment of the vehicle 1. In this case, the control device 100 may issue a transition demand. The transition demand requests the driver 3 to transition to manual driving. For example, the control device 100 issues the transition demand from the visual device 20 and the speaker 30. When the driver 3 starts manual driving, the automatic driving of the vehicle 1 ends (step S160).

[0052] 4.Effects As described above, according to this embodiment, in the driver awakening check process, the first visual device 20-1 issues the first visual notification NV1. Then, if the driver 3 reacts to the first visual notification NV1, the driver 3 is determined to be awakened. The driver 3 reacts to the first visual notification NV1 by changing his / her line of sight or face direction. Since the driver 3 does not need to press a switch, the driver 3 is not hindered from monitoring the surrounding environment. In addition, the driver 3 does not need to take his / her hands off the steering wheel, so the driver 3's instantaneous driving operation (driving behavior) is not hindered. These are preferable from the viewpoint of ensuring safety during automatic driving.

[0053] In addition, the first visual device 20-1 that issues the first visual notification NV1 is relatively far from the forward direction D0 as viewed from the driver's seat 2 where the driver 3 sits. Therefore, the line of sight of the driver 3 who responds to the first visual notification NV1 is guided in a direction away from the forward direction D0. That is, the field of view of the driver 3 who responded to the first visual notification NV1 also includes the surrounding environment in a direction relatively far from the forward direction D0. For example, the field of view of the driver 3 who responded to the first visual notification NV1 includes roadside signs, pedestrians, structures with blind spots, and the like (see FIG. 6). This means that the monitoring of the surrounding environment by the driver 3 is promoted. The driver wakefulness check process according to this embodiment can check whether the driver 3 is awake and at the same time actively guide the line of sight of the driver 3 to the surrounding environment to promote monitoring of the surrounding environment.

[0054] In the driver awakening check process, the second visual device 20-2, which is closer to the forward direction D0, does not need to be used. When the second visual device 20-2 is not used and only the first visual device 20-1 is used, it is possible to more effectively guide the line of sight of the driver 3 in a direction away from the forward direction D0.

[0055] In the driver awakening check process, the voice notification NA does not have to be output from the speaker 30. The stimulus given to the driver 3 by the voice notification NA is stronger than the stimulus given to the driver 3 by the visual notification NV. Because the stimulus of the voice notification NA is strong, the driver 3 may feel that the voice notification NA is excessive. That is, the driver 3 may feel that the voice notification NA is annoying. In addition, the voice notification NA can be heard not only by the driver 3 but also by the passenger. Therefore, the passenger may also feel that the voice notification NA is annoying. By deliberately not outputting such a voice notification NA, the annoyance felt by the driver 3 and the passenger is suppressed. [Explanation of symbols]

[0056] 1...vehicle, 2...driver's seat, 3...driver, 10...vehicle control system, 20...visual device, 20-1...first visual device, 20-2...second visual device, 30...speaker, 40...driver monitor, 50...sensor group, 60...travel device, 100...control device, D0...forward direction, D1...first direction, D2...second direction, NV1...first visual notification, NV2...second visual notification

Claims

1. A vehicle control method for controlling a vehicle, comprising: the vehicle includes a first visual device and a second visual device; The forward direction is a direction from a driver's seat where a driver of the vehicle sits toward the front of the vehicle, the first direction is a direction from the driver's seat to the first visual device, the second direction is a direction from the driver's seat to the second visual device, a first angle between the first direction and the forward direction is greater than a second angle between the second direction and the forward direction; The vehicle control method includes: when checking whether the driver is awake during automatic driving of the vehicle, issuing a first visual notification from the first visual device without issuing a visual notification from the second visual device; If the driver responds to the first visual notification, the driver is determined to be awake, and the automated driving of the vehicle is continued. Includes A vehicle control method.

2. 2. A vehicle control method according to claim 1, The reference angle is half the angle of the average human visual field, The limiting angle is half the angle of the average human visual field, The first angle between the first direction and the forward direction is greater than the reference angle and less than the limit angle. A vehicle control method.

3. A vehicle control system for controlling a vehicle, the vehicle includes a first visual device and a second visual device; The forward direction is a direction from a driver's seat where a driver of the vehicle sits toward the front of the vehicle, the first direction is a direction from the driver's seat to the first visual device, the second direction is a direction from the driver's seat to the second visual device, a first angle between the first direction and the forward direction is greater than a second angle between the second direction and the forward direction; The vehicle control system includes one or more processors. The one or more processors: when checking whether the driver is awake during automatic driving of the vehicle, issuing a first visual notification from the first visual device without issuing a visual notification from the second visual device; If the driver responds to the first visual notification, the driver is determined to be awake and the automated driving of the vehicle is continued. It was configured as Vehicle control system.

Images