Driving support control device, driving support method, and computer program

Abstract

The invention provides a driving support control device, a driving support method, and a computer program. After a vehicle is automatically stopped, a passenger can safely get off the vehicle. A driving assistance control device (7) for a vehicle (100) is configured to perform driving assistance for stopping the vehicle (100) when it is determined that a driver of the vehicle (100) is in an abnormal state in which it is difficult to continue driving of the vehicle (100), and to perform a notification in accordance with a confirmation result of a surrounding situation of the vehicle (100) when a passenger is present during the driving assistance.

Description

Technical Field

[0001] This invention relates to a driving assistance control device, a driving assistance method, and a computer program. Background Technology

[0002] In Patent Document 1, a conventional vehicle control device is disclosed with the following configuration: if an abnormality of the driver is detected while the vehicle is in motion, a notification control (warning tone output control) is executed to notify the abnormality; if a predetermined time has elapsed since the execution of the notification control and no release operation has been performed, then vehicle stop control is initiated.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2022-42665 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] One safety measure to address situations where a driver finds themselves in an abnormal state where driving is difficult is to implement, such as deceleration and stop control, which automatically slows the vehicle down and keeps it at a stop. When passengers are seated in the front passenger seat or rear seats of a vehicle with deceleration and stop control implemented, it is necessary to ensure their safe disembarkation after the vehicle has come to an automatic stop. However, the aforementioned conventional technologies do not consider this need for passengers after the vehicle has come to an automatic stop.

[0008] This invention was made with the aim of addressing such a problem, and its purpose is to enable passengers to safely disembark after the vehicle has come to an automatic stop.

[0009] Technical solutions for solving the problem

[0010] To address the aforementioned issues, one aspect of the present invention provides a vehicle driving assistance control device configured to, when it is determined that the driver of the vehicle is in an abnormal state where it is difficult to continue driving the vehicle, implement driving assistance to stop the vehicle, and, if a passenger is present during driving assistance, provide notification corresponding to the confirmation result of the vehicle's surrounding conditions.

[0011] In addition, one aspect of the vehicle driving assistance method of the present invention implements driving assistance to stop the vehicle when it is determined that the driver of the vehicle is in an abnormal state that makes it difficult to continue driving the vehicle, and provides notification corresponding to the confirmation result of the surrounding conditions of the vehicle when there is a passenger during driving assistance.

[0012] In addition, a computer program (program product) of one aspect of the present invention causes the computer to perform the following processing: when it is determined that the driver of the vehicle is in an abnormal state that makes it difficult to continue driving the vehicle, driving assistance is implemented to stop the vehicle; and when there is a passenger in the vehicle during driving assistance, notification is implemented in accordance with the confirmation result of the surrounding conditions of the vehicle.

[0013] The effects of the invention

[0014] According to these technical solutions of the present invention, since a notification is made in accordance with the confirmation result of the vehicle's surrounding conditions when a passenger is present during driving assistance, the passenger can safely disembark after the vehicle is automatically stopped. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of a vehicle according to one embodiment of the present invention.

[0016] Figure 2 This is a flowchart illustrating the notification control during deceleration and stop control according to one embodiment of the present invention.

[0017] Explanation of reference numerals in the attached figures

[0018] 7: Control device (driving assistance control device); 100: Vehicle. Detailed Implementation

[0019] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Furthermore, in the following description, the same reference numerals will be used to denote the same constituent elements.

[0020] Figure 1 This is a schematic configuration diagram of a vehicle 100 according to one embodiment of the present invention.

[0021] The vehicle 100 includes a peripheral sensor 1, a vehicle sensor 2, a driver sensor 3, a passenger sensor 4, an HMI (Human Machine Interface) 5, an actuator 6, and a control device 7. The peripheral sensor 1, vehicle sensor 2, driver sensor 3, passenger sensor 4, HMI 5, actuator 6, and control device 7 are connected in a communicable manner via an in-vehicle network 9 that conforms to a standard controller area network.

[0022] The peripheral sensor 1 is a sensor used to generate peripheral data representing the condition of the area surrounding the vehicle 100. In this embodiment, the vehicle 100 includes one or more external cameras 11 used to capture images of the area surrounding the vehicle 100 as peripheral sensor 1. The external cameras 11 capture images of the area surrounding the vehicle 100 at a predetermined frame rate (e.g., 10 Hz to 40 Hz) to generate a peripheral image reflecting the area surrounding the vehicle 100. Whenever a peripheral image is generated, the external cameras 11 send the generated peripheral image as peripheral data to the control device 7.

[0023] In addition to the external camera 11, a range sensor, which measures the distance to targets and objects on the ground around the vehicle 100, can also be included as the peripheral sensor 1. Examples of range sensors include LiDAR (Light Detection and Ranging), which illuminates radar light and measures distance based on its reflected light, and millimeter-wave radar sensors, which illuminate radio waves and measure distance based on their reflected waves.

[0024] Vehicle sensor 2 is a sensor used to generate vehicle data representing the state of vehicle 100. In this embodiment, vehicle 100 includes a speed sensor 21 that generates speed data representing the vehicle 100's travel speed, and a positioning sensor 22 that generates current position data such as latitude and longitude representing the vehicle 100's current position, as vehicle sensor 2. However, vehicle sensor 2 is not limited to these sensors. The data acquired by each sensor 21 and 22 is transmitted to the control device 7 as vehicle data.

[0025] The driver sensor 3 is a sensor used to generate driver data representing the driver's state. In this embodiment, the vehicle 100 includes a driver monitoring camera 31 as the driver sensor 3, used to capture images of the driver's appearance, including the driver's face. The driver monitoring camera 31 captures the driver's appearance at a predetermined frame rate (e.g., 10 Hz to 40 Hz) and generates an appearance image reflecting the driver's appearance. Whenever an appearance image of the driver is generated, the driver monitoring camera 31 sends the generated appearance image as driver data to the control device 7.

[0026] The passenger sensor 4 is a sensor used to detect the presence or absence of a passenger. In this embodiment, the vehicle 100 includes an in-vehicle camera 41 for capturing images of the vehicle interior as the passenger sensor 4. The in-vehicle camera 41 captures images of the vehicle interior at a predetermined frame rate (e.g., 10 Hz to 40 Hz) to generate an in-vehicle image. Whenever an in-vehicle image is generated, the in-vehicle camera 41 sends the generated image to the control device 7.

[0027] It should be noted that, in addition to the in-vehicle camera 41, or in place of the in-vehicle camera 41, other sensors such as seatbelt sensors that detect whether the seatbelts of each seat in the vehicle 100 are fastened, and seating sensors that detect the load applied to the seat surface of each seat in the vehicle 100 may also be used as passenger sensors 4.

[0028] HMI5 is a user interface used for exchanging information between vehicle 100 and its occupants. HMI5 includes an output device 51 for notifying vehicle occupants through sensory inputs (e.g., visual, auditory, and tactile sensations) and an input device 52 for occupants to perform input and response operations. Output device 51 may be a display (e.g., instrument cluster display, central display, head-up display, etc.) or a speaker. Input device 52 may be a touch panel or a microphone.

[0029] HMI5 notifies the vehicle occupants of information corresponding to the output signals received from the control unit 7 via output device 51, and sends the data input by the vehicle occupants via input device 52 to the control unit 7.

[0030] The HMI5 can be pre-installed in vehicle 100, or it can be a terminal such as a smartphone owned by the vehicle occupants (driver and passengers). In the latter case, for example, information can be exchanged through short-range wireless communication between vehicle 100 and the vehicle occupants' terminals, or through communication between the vehicle occupants' terminals and an external server (not shown), exchanging information indirectly via the server.

[0031] Actuator 6 is a device for driving control of vehicle 100. In this embodiment, vehicle 100 includes an acceleration actuator 61 (e.g., at least one of engine and electric motor) for accelerating vehicle 100, a brake actuator 62 (e.g., hydraulic actuator) for braking vehicle 100, and a steering actuator 63 (e.g., steering motor) for steering (steering) vehicle 100 as actuator 6.

[0032] The control device 7 is an ECU (Electronic Control Unit) that includes a communication unit 71, a storage unit 72, and a processing unit 73.

[0033] The communication unit 71 includes an interface circuit for connecting the control device 7 to the in-vehicle network 9. The communication unit 71 supplies various data received from various sensors, HMI 5, etc., to the processing unit 73. Furthermore, the communication unit 71 outputs various signals from the processing unit 73 to HMI 5, actuator 6, etc.

[0034] Storage unit 72 includes storage media such as HDD (Hard Disk Drive), SSD (Solid Disk Drive), and semiconductor memory, as well as various computer programs and data used in the storage processing unit 73.

[0035] The processing unit 73 has one or more CPUs (Central Processing Units) and their peripheral circuitry, and executes various computer programs stored in the storage unit 72. The processing unit 73 is, for example, a processor. The processing unit 73 may also have other arithmetic circuitry such as a logic unit, a numerical operation unit, or a graphics processing unit. By executing processing according to the computer program, the processing unit 73 functions as an exception state determination unit 81, an identification unit 82, and a driving assistance unit 83, operating as a functional unit (module) that implements the prescribed functions. In the following description, when the processing is described with each functional unit 81 to 83 as the subject, it indicates that the processing unit 73 executes the program that implements each functional unit 81 to 83.

[0036] The specific processing performed in the control device 7 will be described below. Specifically, the contents of each functional unit 81 to 83, which are implemented by the processing unit 73 according to the computer program, will be explained.

[0037] The abnormal state determination unit 81 determines, for example, whether the driver has fallen into an abnormal state (hereinafter referred to as "abnormal state") that makes it difficult to continue driving due to a sudden change in physical condition. In this embodiment, the abnormal state determination unit 81 determines that the driver has fallen into an abnormal state if the predetermined abnormal presumption state has lasted for a predetermined time T1.

[0038] An abnormal presumed state is a state in which the driver is considered to be in an abnormal state. For example, if a driver assistance system is used to control the vehicle 100 at a driving control level of 2 or lower, a state in which the driver is not performing steering wheel operations can be cited. Whether the driver is performing steering wheel operations can be determined, for example, based on the magnitude of the steering torque acting on the steering shaft, which rotates integrally with the steering wheel. Alternatively, regardless of the driving control level, it can also be determined whether the driver is in an abnormal presumed state, for example, based on the image from the driver monitoring camera 31, or based on the driver's appearance.

[0039] Furthermore, in this embodiment, when the presumed abnormal state persists for a predetermined time T0 (T0 being a shorter time than T1), a warning is initiated to the driver via HMI5. This allows deceleration and stopping control to be implemented only when the driver is in an abnormal state where they are unable to respond to the warning. Alternatively, other control devices can determine whether the driver is in an abnormal state and obtain the determination result.

[0040] The recognition unit 82 identifies objects and ground objects surrounding the vehicle 100. For example, the recognition unit 82 sequentially inputs surrounding images received from the external camera 11 into the recognizer to identify other vehicles, bicycles, pedestrians, curbs or guardrails, and other similar structures (hereinafter referred to as "dividing objects"), road markings (e.g., lane markings), and other ground objects within the surrounding images. The recognizer can be configured, for example, as a convolutional neural network (CNN) having multiple convolutional layers connected in series from the input side to the output side. Furthermore, the recognition unit 82 calculates the distance from the vehicle 100 to the objects and ground objects using, for example, the standard sizes of the objects and ground objects stored in the storage unit 72 according to each type of object and ground object, and the sizes of the objects and ground objects identified in the surrounding images, and calculates the positions of the objects and ground objects. Moreover, the method for identifying objects and ground objects is not limited to this method; any known method can be used for identification.

[0041] The driving assistance unit 83 controls the actuators 6 based on the target objects and ground objects identified by the recognition unit 82, and implements driving assistance that accompanies the driving control of the vehicle 100. In this embodiment, the driving assistance unit 83 can implement driving assistance that accompanies the driving control of the vehicle 100 at a driving control level based on the SAE (Society of Automotive Engineers) definition of Level 3, that is, a driving control level that does not require the driver to operate each actuator 61-63 or monitor the surroundings. In addition, the driving assistance unit 73 can also perform driving assistance that accompanies the driving control of the vehicle 100 at a driving control level in which the driver participates in driving the vehicle 100, such as a driving control level based on the SAE definition of Level 1 or Level 2.

[0042] As one of the driving assistance systems accompanying the driving control of vehicle 100, the driving assistance unit 83 implements deceleration and stop control to slow down vehicle 100 and keep vehicle 100 in a stopped state when the driver is in an abnormal state.

[0043] Regarding how to decelerate / stop vehicle 100, only the risk of contact with road users outside the vehicle when implementing deceleration and stopping control needs to be considered; there are no particular limitations. For example, if the lane markings for the vehicle's driving lane are identified, vehicle 100 will decelerate along the lane markings and remain at a stop. If the lane markings for the vehicle's driving lane are not identified, vehicle 100 will decelerate along the trajectory of the vehicle in front and remain at a stop. If neither the lane markings for the vehicle's driving lane nor the vehicle in front are identified, vehicle 100 can decelerate and remain at a stop while driving straight.

[0044] When deceleration and stopping control are implemented, vehicle 100 eventually stops in the lane (the part of the road marked by curbs, guardrails, or other similar structures or road markings for the passage of vehicles) or shoulder.

[0045] Therefore, if a passenger inadvertently exits the front passenger seat or rear seat of a vehicle 100 that has implemented deceleration and stop control, there is a possibility that the passenger may come into contact with a road user outside the vehicle, such as another vehicle approaching from behind.

[0046] Therefore, in this embodiment, when a passenger is present during the deceleration stop control, the surrounding conditions are confirmed based on surrounding data indicating the conditions around the vehicle 100, and the passenger is notified via HMI4 in accordance with the confirmation result. This prevents passengers from unknowingly exiting the front passenger seat or rear seats of the vehicle 100 when the deceleration stop control has stopped and the vehicle 100 has come to a stop.

[0047] Figure 2 This is a flowchart explaining the notification control in the deceleration and stop control of this embodiment, implemented by the driving assistance unit 83 and even the control device 7. The control device 7 repeatedly executes this routine at a predetermined operation cycle.

[0048] In step S1, the control device 7 determines whether deceleration stop control is being implemented. If deceleration stop control is being implemented, the control device 7 proceeds to step S2. On the other hand, if deceleration stop control is not being implemented, the control device 7 terminates the current process.

[0049] In step S2, the control device 7 determines whether there is a passenger. If there is a passenger, the control device 7 proceeds to step S3. Otherwise, if there is no passenger, the control device 7 terminates the current process.

[0050] In step S3, the control device 7 determines whether it is a notification timer for the passenger. The notification timer can be set to any time, such as before the vehicle stops or after the vehicle stops. In this embodiment, the control device 7 determines whether the vehicle 100 has stopped. If the vehicle 100 has stopped, it is determined that it is a notification timer and the process proceeds to step S4. If the vehicle 100 has not stopped, the current process ends.

[0051] In step S4, the control device 7 determines whether there is a moving object (hereinafter referred to as a "risk moving object") around the vehicle 100 that could potentially come into contact with the passenger when the passenger gets off the vehicle 100. For example, if the control device 7 detects by the identification unit 82 that another vehicle is behind the vehicle 100 and is traveling in the lane where the vehicle 100 is stopped (hereinafter referred to as the "stop lane") or an adjacent lane to the stop lane, it can determine that a risk moving object exists around the vehicle 100. Additionally, if the control device 7 detects by the identification unit 82 that a two-wheeled vehicle or pedestrian is approaching the vehicle 100, it can also determine that a risk moving object exists around the vehicle 100. If a risk moving object exists around the vehicle 100, the control device 7 proceeds to step S5. On the other hand, if no risk moving object exists around the vehicle 100, the control device 7 proceeds to step S6.

[0052] In step S5, the control device 7 notifies the passenger not to get off the vehicle.

[0053] In step S6, the control device 7 informs the passenger that "you may get off the vehicle after confirming the surroundings."

[0054] The control device 7 (driving assistance control device) of the vehicle 100 described above in this embodiment is configured to implement deceleration and stop control (driving assistance) to stop the vehicle 100 when it is determined that the driver of the vehicle 100 is in an abnormal state that makes it difficult to continue driving the vehicle 100. If there is a passenger present during the deceleration and stop control, a notification is issued corresponding to the confirmation result of the surrounding situation of the vehicle 100. Specifically, the control device 7 is configured to notify the passenger not to get out of the vehicle when it is confirmed that a moving object is approaching from behind the vehicle 100 (for example, in the driving lane where the vehicle 100 is stopped (stop lane) or in the adjacent lane of the stop lane).

[0055] In this way, according to this embodiment, when there are passengers present during deceleration and stop control, a notification can be made corresponding to the confirmation result of the surrounding conditions of the vehicle 100, so that passengers can safely get off the vehicle after the vehicle 100 is automatically stopped by deceleration and stop control.

[0056] The embodiments of the present invention have been described above, but the above embodiments only show a part of the application examples of the present invention, and the technical scope of the present invention is not limited to the specific configuration of the above embodiments.

[0057] For example, in the above-described embodiments, the computer program executed in the control device 7 may also be provided in the form of a computer-readable, portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium.

Claims

1. A driving assistance control device, which is a driving assistance control device for a vehicle, configured as follows: If it is determined that the driver of the vehicle is in an abnormal state that makes it difficult for them to continue driving, a driving assistance mechanism is implemented to bring the vehicle to a stop. In the presence of a passenger during the driving assistance operation, a notification is issued corresponding to the confirmation result of the vehicle's surrounding conditions.

2. The driving assistance control device according to claim 1, configured as follows: Upon confirming a moving object approaching from behind the vehicle, the passenger is instructed not to disembark.

3. The driving assistance control device according to claim 2, The moving body is another vehicle traveling in the lane where the vehicle is stopped or in an adjacent lane.

4. A driving assistance method, which is a driving assistance method for a vehicle. If it is determined that the driver of the vehicle is in an abnormal state that makes it difficult for them to continue driving, a driving assistance mechanism is implemented to bring the vehicle to a stop. In the presence of a passenger during the driving assistance operation, a notification is issued corresponding to the confirmation result of the vehicle's surrounding conditions.

5. A computer program that causes a computer to perform the following processing: If it is determined that the driver of the vehicle is in an abnormal state that makes it difficult for them to continue driving, a driving assistance mechanism is implemented to stop the vehicle. In the presence of a passenger during the driving assistance operation, a notification is issued corresponding to the confirmation result of the vehicle's surrounding conditions.

Images