Automatic driving control device, automatic driving control method, and program

Abstract

The present disclosure relates to an automatic driving control device, and an automatic driving control method and program. A method for switching a mode for operating a vehicle, a non-transitory computer readable medium for executing the method, and an information processing apparatus. The method includes determining, by a circuit of the information processing apparatus, whether a mode for operating the vehicle is to be switched from one of an autonomous and manual driving mode to another of the autonomous and manual driving mode. When it is determined that the operating mode of the vehicle is switched, a state of a driver of the vehicle is acquired. The method further includes switching, by the circuit, the mode for operating the vehicle from one of the autonomous and manual driving mode to another of the autonomous and manual driving mode based on the acquired state of the driver.

Description

[0001] This application is a divisional application of the invention patent application filed on December 4, 2015, with application number 201911336458.3 and entitled "Autonomous Driving Control Device and Autonomous Driving Control Method and Program". Technical Field

[0002] This disclosure relates to an autonomous driving control device, an autonomous driving control method and program, and more specifically to an autonomous driving control device, an autonomous driving control method and program capable of safely performing autonomous driving.

[0003] Cross-references to related applications

[0004] This application claims priority to Japanese priority patent application JP 2014-251493, filed on December 12, 2014, the entire contents of which are incorporated herein by reference. Background Technology

[0005] In recent years, rapid progress has been made in the development of technologies to achieve autonomous driving in automobiles. For example, lane departure warning systems, which warn when a cruise vehicle leaves its lane, have recently been developed as a technological advancement related to autonomous driving. When a driver's alertness decreases on a monotonous road and the cruise vehicle begins to drift out of its lane, the lane departure warning system issues a warning using vibration, sound, and other means. It is widely acknowledged that such lane departure warning systems are effective in preventing lane departure accidents, and statistically, it is known that lane departure warning systems significantly reduce lane departure accidents in groups of vehicles equipped with these systems.

[0006] A technique has been developed to detect obstacles using submillimeter-wave radar, LIDAR (light detection and ranging) using lasers instead of submillimeter waves, or stereo cameras, as a means of sensing the area around a vehicle to measure the distance and arrangement between the vehicle and vehicles driving in front (see PTL 1). Therefore, applications such as adaptive cruise control systems for maintaining distance between the vehicle and vehicles driving in front, and further, emergency collision reduction / prevention braking systems, have been implemented.

[0007] When using autonomous driving, the level of driving skill varies depending on the driver, and it is considered very important to monitor the driver's condition.

[0008] However, systems where passengers do not need to intervene at all during the vehicle's journey from origin to destination are only feasible in closed environments such as vehicles traveling on tracks (like electric trains) and monorails. Currently, there is no system in mind that allows for the introduction of autonomous driving on roads in general use.

[0009] On the other hand, the introduction of autonomous driving begins in a certain way. At this point, manual intervention in driving is expected. When the vehicle enters autonomous driving mode and cuts off driver intervention, the lower level of manual driving capability upon resumption is considered a risk of accidents and traffic disruption.

[0010] For example, unlike beverages and driving, even if the driver is a little tired, the vehicle is considered to have a psychological impact on people who want to move in order to actively drive the vehicle, and there is an expectation of having mechanical or system mechanisms to reduce dangerous operations as an automatic driving system.

[0011] Reference List

[0012] Patent documents

[0013] PTL 1: JP 2000-329852 A Summary of the Invention

[0014] Technical issues

[0015] From the perspective of the above operations, in order to implement autonomous driving more safely, before the driver plans to drive on a manual driving section, the driver needs to have the ability to recover their driving skills. Furthermore, it is necessary to have a mechanism that allows dedicated autonomous driving cruise control when it is determined whether the driver has the ability to recover their driving skills. In addition, if it is confirmed that the ability to recover to a manual driving section has become low, it is necessary to take measures to prevent the initiation of autonomous driving.

[0016] This disclosure is made in view of this situation, and is intended to enable safer autonomous driving.

[0017] Solution to the problem

[0018] In embodiments of this disclosure, a method for switching modes for operating a vehicle is provided. The method includes determining, via circuitry of an information processing device, whether to switch the mode for operating the vehicle from one of autonomous and manual driving modes to another of autonomous and manual driving modes. The method includes acquiring the state of the vehicle's driver when it is determined that switching modes for operating the vehicle is to be performed. The method further includes switching the mode for operating the vehicle from one of autonomous and manual driving modes to another of autonomous and manual driving modes via circuitry based on the acquired driver state.

[0019] In another embodiment of this disclosure, a non-transitory computer-readable medium is provided that stores instructions, when executed by a computer, to cause the computer to perform a method for switching a mode for operating a vehicle. The method includes determining whether the mode for operating the vehicle needs to be switched from one of an autonomous and a manual driving mode to another of the autonomous and manual driving modes. The method includes acquiring the state of the vehicle's driver when it is determined that the mode for operating the vehicle needs to be switched. The method further includes switching the mode for operating the vehicle from one of the autonomous and a manual driving mode to another of the autonomous and manual driving modes based on the acquired driver state.

[0020] In another embodiment of this disclosure, an information processing apparatus is provided, including circuitry configured to determine whether a mode for operating a vehicle needs to be switched from one of an autonomous and a manual driving mode to the other. The circuitry is configured to acquire the state of the vehicle's driver when it is determined that a switch for operating the vehicle mode is to be made. The circuitry is also configured to switch the mode for operating the vehicle from one of the autonomous and manual driving modes to the other based on the acquired driver state.

[0021] In another embodiment of this disclosure, an information processing apparatus is provided, including means for determining whether a mode for operating a vehicle needs to be switched from one of an autonomous and a manual driving mode to another of an autonomous and a manual driving mode, means for acquiring the state of the vehicle driver when it is determined to switch the mode for operating the vehicle, and means for switching the mode for operating the vehicle from one of an autonomous and a manual driving mode to another of an autonomous and a manual driving mode based on the acquired driver state.

[0022] Beneficial effects of the invention

[0023] According to one aspect of this disclosure, autonomous driving can be performed more safely. Attached Figure Description

[0024] [ Figure 1 ] Figure 1 This is a block diagram illustrating a configuration example of an embodiment of an autonomous driving control device to which the present technology is applied.

[0025] [ Figure 2 ] Figure 2 This is a flowchart illustrating the autonomous driving termination process when autonomous driving is terminated.

[0026] [ Figure 3 ] Figure 3 It is a diagram used to illustrate the road environment related to autonomous driving processing in various road environments.

[0027] [ Figure 4 ] Figure 4 This is a flowchart illustrating the process for determining the likelihood / impossibility of autonomous driving in a lane and the likelihood / impossibility of continuous cruising.

[0028] [ Figure 5 ] Figure 5 This is a flowchart illustrating the process of automatically evacuating to evacuation lanes in an emergency.

[0029] [ Figure 6 ] Figure 6 This is a flowchart used to illustrate the dynamic correction process of braking force.

[0030] [ Figure 7 ] Figure 7 This is a diagram used to illustrate the termination sequence of autonomous cruise.

[0031] [ Figure 8 ] Figure 8 A and 8B are diagrams used to illustrate a driver's understanding of changes in road conditions.

[0032] [ Figure 9 ] Figure 9 This is a block diagram illustrating an example configuration of an autonomous driving control system.

[0033] [ Figure 10 ] Figure 10 This is a flowchart used to illustrate the initial determination process.

[0034] [ Figure 11 ] Figure 11 It is a flowchart used to illustrate the setting of input processing.

[0035] [ Figure 12 ] Figure 12 This is a flowchart used to illustrate the comprehensive assessment process for recycling capacity.

[0036] [ Figure 13 ] Figure 13 This is a flowchart used to illustrate the process of initiating an emergency evacuation mode.

[0037] [ Figure 14 ] Figure 14 This is a flowchart used to illustrate the prediction process for autonomous driving termination.

[0038] [ Figure 15 ] Figure 15 This is a flowchart used to illustrate the termination preparation process.

[0039] [ Figure 16 ] Figure 16 This is a flowchart used to illustrate automated evacuation procedures.

[0040] [ Figure 17 ] Figure 17This is a diagram used to illustrate the flashing of a hazard warning light.

[0041] [ Figure 18 ] Figure 18 This is a flowchart used to illustrate the process of determining manual driving capability.

[0042] [ Figure 19 ] Figure 19 This is a flowchart used to illustrate the driver wake-up process.

[0043] [ Figure 20 ] Figure 20 This is a flowchart used to illustrate emergency evacuation and relocation procedures.

[0044] [ Figure 21 ] Figure 21 A and 21B are diagrams used to illustrate the driver's operational delay.

[0045] [ Figure 22 ] Figure 22 This is a flowchart used to illustrate the braking assistance intervention process.

[0046] [ Figure 23 ] Figure 23 This is a diagram used to illustrate the process of predicting the predicted time when switching to manual driving occurs.

[0047] [ Figure 24 ] Figure 24 This is a block diagram illustrating a configuration example of a computer to which the present technology is applied. Detailed Implementation

[0048] The following will describe in detail specific embodiments of the application of this technology with reference to the accompanying drawings.

[0049] Figure 1 This is a block diagram illustrating a configuration example of an embodiment of an autonomous driving control device to which the present technology is applied.

[0050] like Figure 1 The autonomous driving control device 11 shown is incorporated as part of an autonomous driving system installed in a car capable of autonomous driving, and is configured to include a recovery determination unit 12 and a driving execution unit 13. For example, the autonomous driving control device 11 takes control when the car, which is performing autonomous driving, changes from manual driving to autonomous driving, or when it becomes difficult to continue continuous autonomous driving during autonomous driving due to some dependence on the vehicle itself or the environment, or when the car returns to being driven by a driver.

[0051] The recovery determination unit 12 determines whether the driver has a sufficient level of driving ability to resume manual driving from automatic driving. As shown in the figure, the recovery determination unit 12 is configured to include a monitoring unit 21, an output unit 22, a road information acquisition unit 23, an input unit 24, a recording unit 25, and a determination processing unit 26. It should be noted that the manual driving operation according to the embodiments of this disclosure is not limited to driving operation using a manual transmission, and the manual driving operation according to the embodiments of this disclosure generally refers to driving operation in which the driver intervenes in the driving operation of the vehicle by directly causing a change in the driving of the vehicle in some way.

[0052] The monitoring unit 21 continuously monitors the driver's state to detect the driver's alertness and provides the driver's state, indicating the driver's alertness, to the determination and processing unit 26. For example, see the following reference. Figure 8 The monitoring unit 21 identifies the driver's gaze direction to obtain a driver state indicating whether the driver is in an alert state capable of normal driving. Furthermore, the monitoring unit 21 can monitor the driver's state using various methods, such as recognizing the driver's facial posture, detecting the driver's heart state, tracking steering stability, detecting the driver's body posture, detecting the driver's fatigue level / heart rate changes, and the operational stability of the brake and accelerator pedals. The monitoring unit 21 can monitor the driver's driving ability using one or more methods, including driver facial recognition, head posture recognition, gaze direction recognition, and statistical determination processing of head posture or gaze direction time loss stability. Additionally, the monitoring unit 21 can monitor for vehicle malfunctions and whether the vehicle is in an emergency. Furthermore, information about the planned driving route is obtained in a timely manner from the infrastructure side of the monitoring unit 21 and from vehicles surrounding the vehicle under consideration, and the vehicle's driving route can be pre-monitored by monitoring traffic conditions such as roads. It should be noted that when the monitoring unit 21 continuously monitors the driver's state, it can temporarily suspend direct monitoring of the driver on driving sections where autonomous driving is guaranteed and driver intervention is completely unnecessary.

[0053] For example, the output unit 22 is connected to a speaker (not shown) and outputs various announcement and warning sounds through the speaker according to instructions provided based on the results obtained when the determination processing unit 26 performs determination processing.

[0054] The road information acquisition unit 23 acquires road information around the vehicle based on the vehicle's driving route (e.g., road information about ordinary roads, fully automated driving lanes, and buffer zone roads), and notifies the determination processing unit 26 of the road information.

[0055] Input unit 24 receives, for example, a driving route from a setting device used by the driver to set a driving route when autonomous driving begins, and receives self-diagnostic results related to the autonomous driving of the vehicle from a diagnostic device used to diagnose the condition of each unit of the vehicle. Input unit 24 provides the information thus received to determination processing unit 26.

[0056] Recording unit 25 records various processes performed by the autonomous driving control device 11. For example, recording unit 25 records and saves some or all of the determination results made by determination processing unit 26. In particular, for example, in the switching of driving modes related to decisions made by the driver, which will be explained later, the records made by the driver himself are saved as at least indelible records.

[0057] The determination processing unit 26 performs various determination processes based on the driving status provided by the monitoring unit 21, the driving route and self-diagnosis results provided by the input unit 24, and the road information provided by the road information acquisition unit 23.

[0058] The driving execution unit 13 performs the vehicle's automatic cruise control. As shown in the figure, the driving execution unit 13 is configured to include an environment recognition unit 31, a driving processing unit 32, a steering wheel control unit 33, a speed control unit 34, and a hazard flashing light control unit 35.

[0059] The environment recognition unit 31 uses devices such as stereo cameras and lidar to identify the external environment of the vehicle and provides information indicating the external environment to the driving processing unit 32.

[0060] When the driving processing unit 32 receives an emergency stop command for emergency stopping from the determination processing unit 26 of the recovery determination unit 12, which determines the possibility / impossibility of switching from automatic driving to manual driving, the driving processing unit 32 issues instructions to the steering wheel control unit 33, the speed control unit 34, and the hazard flashing light control unit 35, and performs driving processing to perform emergency stopping of the vehicle.

[0061] The steering wheel control unit 33 performs steering control to determine the vehicle's direction based on instructions from the driving processing unit 32 by centrally performing confirmation processing on the planned driving route based on the planned route map and detailed lane information obtained from the environment recognition unit 31, as well as relevant environmental obstacle information related to the considered lane. The speed control unit 34 performs braking and acceleration control to control the vehicle's speed based on instructions from the driving processing unit 32. The hazard warning light control unit 35 controls the illumination of the hazard warning lights (emergency flashing indicator lights) installed in the vehicle based on instructions from the driving processing unit 32. Although the steering wheel control unit 33 is referred to as a steering wheel control unit, steering control also includes vehicle steering control achieved by individually controlling the wheels using electronic brakes, as this is not directly related to this disclosure.

[0062] In the automated driving control device 11 configured as described above, the determination processing unit 26 can determine whether the driver has a sufficient level of driving ability to revert to manual driving based on the driver status provided by the monitoring unit 21. Then, the determination processing unit 26 can allow switching to automated driving mode based on the determination result.

[0063] For example, the monitoring unit 21 provides the determination processing unit 26 with the driver's status, including parameters such as the driver's state of consciousness, psychological state, stress level, and degree of drug influence. Then, the determination processing unit 26 determines thresholds for these parameters to determine whether the driver has a sufficient level of driving ability to recover to manual driving and whether it is permissible to switch to automatic driving mode.

[0064] When an alcohol detection sensor is connected to the monitoring unit 21, the monitoring unit 21 detects the driver's alcohol intake level based on the alcohol concentration detected from the driver's breath and provides the alcohol intake level to the determination processing unit 26 as the driver's state. Therefore, when switching to autonomous driving mode, the determination processing unit 26 can determine whether the driver has a sufficient level of driving ability to revert to manual driving based on the driver's alcohol intake level.

[0065] Furthermore, when switching to autonomous driving mode, the determination processing unit 26 uses the determination result based on the driver's alcohol intake level and observes past changes in the driver's level of consciousness. If the driver's decision-making delay increases, the driving assistance is terminated, and the vehicle is forced to quickly and forcibly evacuate to an evacuation lane, enabling the vehicle to stop. Specifically, when a vehicle starting to drive along a planned route enters an autonomous driving section from a manual driving section, the driver's driving recovery ability is determined, as will be described later. Figure 3The evacuation will proceed in the previous buffer zone R2, and evacuation control will be implemented to allow vehicles to be driven into the emergency temporary evacuation lanes without expecting drivers to regain their ability to drive.

[0066] Subsequently, refer to Figure 2 The flowchart illustrates the autonomous driving termination process performed when autonomous driving terminates.

[0067] For example, when the determination processing unit 26 identifies, based on infrastructure, vehicle driving history, location information, etc., that the current driving point is before the end of a fully automated driving lane, it begins processing to determine the possibility / impossibility of manual driving for the vehicle's driver. For example, when the vehicle enters the lane described below... Figure 3 The determination process begins when a buffer zone is set up before the end of a fully automated driving lane (road environment R7).

[0068] In step S11, the processing unit 26 determines that the output unit 22 outputs an audible notification indicating the end of the autonomous driving trajectory. Therefore, the output unit 22 uses a speaker (not shown) to output, for example, an audible notification "Autonomous driving is about to terminate" or a corresponding specific alarm sound. Alternatively, the end of the autonomous driving trajectory can be indicated by vibration of the steering wheel or seat, or by providing tactile feedback such as pulling the seatbelt.

[0069] In step S12, the determination processing unit 26 obtains the driver status from the monitoring unit 21 to determine whether the driver has the driving ability level to recover to manual driving. For example, the monitoring unit 21 continuously monitors the driver before the start of the automatic driving termination process and provides the determination processing unit 26 with the driver status during a predetermined period before the start of the automatic driving termination process.

[0070] Or, when the vehicle enters Figure 3 Before the R7 road environment is determined, a notification input can be given to the driver based on any of the auditory, visual, tactile, or other methods, and this can be confirmed by providing appropriate feedback to the determination processing unit 26 to confirm the driver's response to the instruction. Appropriate feedback means that even if the driver is in a conscious state during the process of waking up from sleep, and more specifically, even if the environmental determination performance is insufficient level of arousal, the driver is prevented or made less likely to hastily disengage the automatic driving without consideration by introducing, for example, a button-pressing procedure that changes each time according to the guidance of two different buttons executed in a more thoughtful manner through a determination procedure, rather than an operation of a simple button that can be operated by reflexive action.

[0071] In step S13, the determination processing unit 26 determines whether the driver has a sufficient level of driving ability to resume manual driving based on the driver state obtained in step S12.

[0072] In step S13, if the determining processing unit 26 determines that the driver does not have sufficient driving ability to resume manual driving, and more precisely, if the determining processing unit 26 determines that the driver's driving ability is insufficient, then the processing in step S14 is subsequently executed.

[0073] In step S14, the determination processing unit 26 controls the output unit 22 to issue a warning to the driver to perform a safety awareness recovery. Therefore, the output unit 22 outputs audible announcements, warning sounds, etc., using a speaker or similar means to prompt the driver to perform a safety awareness recovery. For example, if a device or display device for issuing warnings to the driver tactilely using tactile technology such as force feedback is connected to the output unit 22, the output unit 22 can prompt the driver to perform a safety awareness recovery by outputting tactile notifications and visual warnings.

[0074] In step S15, following the processing in step S14, the determination processing unit 26 performs the re-determination of the driver's driving ability as in step S13.

[0075] If the determination processing unit 26 determines in step S15 that the driver's driving ability is insufficient, then proceed to step S16. In step S16, the determination processing unit 26 instructs the driving processing unit 32 of the driving execution unit 13 to enter the evacuation lane according to road signs and stop the vehicle. Therefore, the driving processing unit 32 controls the steering wheel control unit 33 and the speed control unit 34 based on information indicating the external environment provided by the environment recognition unit 31, and performs automatic driving to stop the vehicle after it enters the evacuation lane. Then, after the vehicle stops in the evacuation lane, the automatic driving termination process is terminated.

[0076] On the other hand, if the processing unit 26 determines in step S13 or S15 that the driver's driving ability is sufficient (the driver's driving ability is adequately maintained or restored), then the processing in step S17 is subsequently executed.

[0077] In step S17, the determination processing unit 26 notifies the driving processing unit 32 of the driving execution unit 13 that the driving processing unit 32 is permitted to switch to manual driving. Therefore, the driving processing unit 32 prepares to switch from automatic driving to manual driving.

[0078] Subsequently, for example, when the driver operates the steering wheel or accelerator, the driving processing unit 32 switches the vehicle's driving from automatic to manual driving in step S18. Then, manual driving by the driver begins, and driving proceeds onto a regular road, continuing manual driving until the automatic driving termination process is terminated. This transition to manual driving typically does not require explicit operation as long as the driver's steering remains stable, and the transition can be seamless.

[0079] Using the autonomous driving control device 11, before the vehicle enters the buffer zone road immediately preceding the end of the fully autonomous driving lane, the output unit 22 can output an audible notification to inform the passengers in the vehicle of the end of the fully autonomous driving lane. Subsequently, if the driver's driving ability is determined to be insufficient, the determination processing unit 26 can control the output unit 22 to activate the device using haptic technology such as force feedback as described above, to awaken the driver. Even if the driver's driving ability remains insufficient after that, the determination processing unit 26 can control the driving processing unit 32 to cause the vehicle to automatically evacuate into the evacuation lane and suddenly decelerate within a safe range for vehicles behind.

[0080] In this context, for example, the significance of intentionally slowing down the vehicle suddenly is that the driver's return to manual driving is the initially desired operational procedure, and when multiple vehicles are repeatedly and continuously evacuated from a road maintenance perspective, a certain number or more vehicles may be unsuitable, which is undesirable. More specifically, the vehicle controller does not want to bring the vehicle to a stop by using a sudden deceleration that is not comfortable for riding; therefore, a comfortable return to manual driving is expected by seamlessly advancing to manual driving beforehand.

[0081] When the automatic driving control device 11 is equipped with an emergency automatic stop operation unit that can be easily operated by a non-driver, the driving processing unit 32 can be configured to initiate an emergency stop sequence when the emergency automatic stop operation unit is used.

[0082] Furthermore, utilizing the automatic driving control device 11, when transitioning from automatic driving mode to manual driving mode, the output unit 22 can execute control to repeat the irregular switching of illumination at the rear of the vehicle. In this case, the output unit 22 executes control to sequentially turn on the lights from the center of the vehicle outwards, so that when viewed from behind, the lights can be visually confirmed in a way that appears closer and magnified. More specifically, when viewed from behind, the driver of the following vehicle experiences the illusion of an increased light interval, thus creating the feeling of being perceived as closer to the vehicle. The driver of the following vehicle receives this feeling of approaching as a psychological perception, and therefore naturally observes the apparent deceleration of the vehicle in front, thus providing a notification.

[0083] Subsequently, refer to Figure 3 This describes the road environment in relation to the autonomous driving and processing performed for each road environment.

[0084] Road environment R1 is the ordinary road before entering the dedicated lane for fully automated driving. When the vehicle is started manually by the driver and enters the dedicated lane for fully automated driving within the route, determination processing is performed in road environment R1 to determine whether the driver has the ability to recover from fully automated driving while driving forward. When fully automated driving begins from a starting point, determination processing regarding the driver's ability to recover from fully automated driving is performed immediately after the route is set at the start of automated driving.

[0085] Road environment R2 is a buffer zone road before entering the fully automated driving lane from a regular road. In this buffer zone road, if the driver is deemed capable of recovery as a result of the aforementioned determinations, the vehicle is essentially considered to have entered the fully automated driving lane, and various notifications are given (route guidance to the destination, road conditions, weather, rest stops, accident information, etc.). In this situation, automatic cruise control is prepared, and the driver-independent autonomous cruise control mode or the occasionally following driver mode is set to a permissible driving mode. Furthermore, in the buffer zone road, a termination notification for the regular road exit, a notification to passengers, and a traffic congestion information announcement are given. Additionally, preparations are made for activating the unexperienced road driving mode, emergency warning mode, etc., and a confirmation prompt is given.

[0086] On the other hand, as a result of the determination process performed before the buffer zone road, if it is determined that the driver is not capable of recovery, the driver is prompted to choose either evacuation rest or driving in the fully automated driving lane in an emergency.

[0087] Road environment R3 refers to emergency temporary evacuation lanes located at the roadside of each segment of the route dedicated to fully automated driving lanes. These lanes are used, for example, when vehicles need to evacuate due to a decline in driving ability as they revert from fully automated to manual driving, or when an emergency evacuation is required due to a system malfunction. Figure 3 In this diagram, due to limitations, a single road environment R3 is drawn; however, it is assumed, for example, that road environments R3 are set every few kilometers.

[0088] Road environment R4 is a dedicated lane for fully automated driving exits between major highways. On this dedicated lane for fully automated driving exits between major highways, guidance is provided to the driver as needed, including guidance with temporary wake-up recovery, enabling assistance to detect momentary emergencies.

[0089] Road environment R5 is an entrance lane to a parking area for temporary evacuation. This parking area is preferably a rest stop into which vehicles can enter in fully automated driving, even when it is not an emergency but the driver needs to rest due to accumulated fatigue, or preferably a facility corresponding to a rest stop.

[0090] Road environment R6 is a recovery lane used to restore a parking lot used for temporary evacuation to a dedicated lane for fully automated driving, and to confirm recovery capabilities, such as necessary notifications for driving recovery.

[0091] Road environment R7 is a buffer zone road before entering the general road from the fully automated driving lane. This buffer zone road lane is used as a driving segment to determine the possibility / impossibility of guiding vehicles to the evacuation lane in the termination of the fully automated driving lane, where it is determined that the driver does not have the ability to resume driving, and to provide automatic guidance to the evacuation lane in cases where it is determined that seamless manual driving to continue cruising is difficult or dangerous due to the determination results. In order to avoid the area being completely blocked due to continuous evacuation even when the evacuation lane is used as a pool lane for a short period of time, space for following vehicles can be ensured by automatically parking and stopping the vehicle via the guidance path R8 to the parking area after the vehicle has basically decelerated and entered.

[0092] In road environment R9, the vehicle switches from a fully automated driving lane to a manual driving general lane and connects to the manual driving general lane, and the vehicle arrives at its destination (road environment R10).

[0093] Road environment R13 is the manual driving entry lane, which is the section of road where vehicles leave the fully automated driving lane and enter the manual driving general vehicle lane.

[0094] In the general-purpose lanes for manual driving, which are set up alongside the lanes designated for fully automated driving, the area outside the manual driving entry lane (Road Environment R13) is where lane changes that obstruct major traffic are essentially prohibited. Therefore, this area is not the intended driving zone for entering the general-purpose lanes for manual driving.

[0095] A manual driving entry lane is an area intended for entry when manually operating the turn signals, and regular vehicles are obligated to allow priority entry into the manual driving entry lane. For example, a manual driving entry lane is used, for instance, at a branch point preceding it, to prepare to move onto a road that does not have any dedicated lanes for fully automated driving.

[0096] Subsequently, it will be used Figure 4 The flowchart illustrates the process for determining the likelihood / impossibility of autonomous driving in lanes and the likelihood / impossibility of continuous cruising.

[0097] For example, when the driver displays an expression to start autonomous driving, or more specifically, when the autonomous driving start operation unit is executed, processing begins.

[0098] In step S21, the processing unit 26 determines that it obtains self-diagnostic results related to the autonomous driving of the vehicle via the input unit 24.

[0099] In parallel with step S21, in step S22, the processing unit 26 determines the driver's status from the monitoring unit 21 in order to determine whether the driver has the driving ability level to recover to manual driving.

[0100] In step S23, the determining processing unit 26 determines whether autonomous driving can be initiated. For example, if a fault occurs in at least some devices based on the self-diagnostic results obtained in step S21, the determining processing unit 26 determines that autonomous driving cannot be initiated. Alternatively, if the driver lacks driving ability or has a low level of driving ability based on the driver status obtained in step S22, the determining processing unit 26 determines that autonomous driving cannot be initiated. To simplify the above example, fully autonomous driving in dedicated cruise control lanes is possible, or entry into dedicated cruise control lanes may be prohibited, making an exclusive choice possible. However, in practice, driving may be possible even if the driver does not have the full recovery ability to drive manually. For example, as described later, driving is anticipated in emergency situations such as emergency evacuation or in hospital emergency driving mode, for pregnant women, patients who cannot wait for an ambulance, etc.

[0101] If the processing unit 26 determines in step S23 that autonomous driving cannot be initiated, then the process proceeds to step S24. In step S24, the processing unit 26 determines that autonomous driving is prohibited, for example, prohibiting entry into the dedicated autonomous cruise lane, and terminates the process.

[0102] On the other hand, if the determination processing unit 26 determines in step S23 that autonomous driving can be started, then the processing in step S25 is performed, and the determination processing unit 26 instructs the driving processing unit 32 of the driving execution unit 13 to prepare for the start of autonomous driving.

[0103] In step S26, the driving processing unit 32 controls the steering wheel control unit 33 and the speed control unit 34 to perform automatic driving based on information provided by the environment recognition unit 31 and obtained by recognizing the surrounding environment information, and initiates driving in automatic driving mode.

[0104] In step S27, the determination processing unit 26 monitors the driver's state provided by the monitoring unit 21 and the surrounding environment provided by the environment recognition unit 31. In this case, for example, when the determination processing unit 26 detects certain irregular events occurring during the automatic cruise mode based on changes in the driver's state or the surrounding environment, the determination processing unit 26 determines in step S29 whether the event poses a certain driving risk due to continuous automatic driving, and if a risk exists, provides guidance to the driver, and, if necessary, based on appropriate actions, the vehicle transitions to a mode that terminates automatic driving.

[0105] In step S28, the determination processing unit 26 identifies the driver's state based on the driver's state provided by the monitoring unit 21. Then, the determination processing unit 26 determines whether the driver has a sufficient level of driving ability to resume manual driving.

[0106] In step S29, the determination processing unit 26 determines whether to terminate the automatic driving, and if it determines that the automatic driving will not be terminated, it executes the processing of step S27 again, and then repeats the same processing.

[0107] On the other hand, if in step S29 the processing unit 26 determines that autonomous driving will be terminated, then the processing in step S30 is executed, and reference is made to... Figure 2 The flowchart illustrates the autonomous driving termination process. Then, when the autonomous driving termination process is complete, the process terminates. Although... Figure 2 The automated driving termination procedure shown is described as considering the termination of section R7 on roads dedicated to autonomous driving vehicles, but... Figure 2The autonomous driving termination procedure shown can also be a procedure for unexpectedly entering the parking area R5, or a procedure when moving from the autonomous driving dedicated vehicle road segment R7 to the manual driving entrance lane R13 of the general lane in order to move to the destination location before the end of the autonomous driving dedicated vehicle road segment R7.

[0108] As described above, in the event of at least some equipment malfunction, or in the event that the driver has no driving ability or has a low level of driving ability and may not be able to return to manual driving, the automatic driving control device 11 prohibits entry into the dedicated automatic cruise lane, thereby enabling safer execution of automatic driving.

[0109] Subsequently, refer to Figure 5 The flowchart illustrates the process of automatically evacuating to evacuation lanes in an emergency.

[0110] When driving is initiated, processing begins, and in step S41, it is determined that the processing unit 26 obtains self-diagnostic results related to the vehicle's autonomous driving via the input unit 24. It may also collect environmental information required for autonomous cruise control of the overall planned driving route, and information on the possibility / impossibility of obtaining previous update information.

[0111] In step S42, the determination processing unit 26 monitors the driver status provided by the monitoring unit 21 and the surrounding environment provided by the environment recognition unit 31. Prior environmental information required for autonomous cruise along the planned driving route can be acquired via means such as radio.

[0112] In step S43, the determination processing unit 26 issues a warning to the driver corresponding to the occurrence of the irregular state, and notifies the driver tactilely, for example, using haptic technology such as force feedback. The irregular state referred to here refers to a state where the planned changes in the vehicle, road, weather, etc., are guaranteed even when the driver is not present in fully autonomous driving, as well as a state that deviates from the design environment of fully autonomous driving due to, for example, equipment problems with submillimeter-wave radar affecting vehicle driving, problems with road guidance information, delays in updating information, and unexpected deterioration of weather conditions. In this case, measures to deal with the irregular state are implemented in two main ways. One is that the vehicle continues to drive, as if the driver can regain normal awareness; the other is that the vehicle stops from a dedicated lane where temporary stopping is expected due to the driver's lack of awareness. Continuing to drive is only desired if the autonomous driving system can ensure a certain level or higher level of safety risk by intervening in an assisted manner even if the driver cannot fully regain normal awareness.

[0113] In step S44, the processing unit 26 determines whether the driver responds to the warning in step S43 by exhibiting an indication of normal alertness. For example, when... Figure 21 When the driver responds to a warning as shown in A and 21B (e.g., the level of consciousness is within the normal range), the determination processing unit 26 determines that the driver has responded and performs the processing in step S42 again, and then repeats the same processing.

[0114] On the other hand, if the determination processing unit 26 determines that the driver is unresponsive (or when the driver's arousal is insufficient), then the processing in step S45 is performed. In step S45, the determination processing unit 26 determines, based on the driver's status provided by the monitoring unit 21, whether the driver's state is "loss of consciousness," "decreased arousal," or "no problem."

[0115] In step S45, if the determination processing unit 26 determines that the driver's status is "no problem", the processing of step S42 is performed again, and then the same processing is repeated.

[0116] On the other hand, if the determination processing unit 26 determines in step S45 that the driver's state is equal to or less than the "awake state", then the processing in step S46 is performed. In step S46, the determination processing unit 26 performs a wake-up operation to increase the driver's wakefulness, and for example, the control output unit 22 outputs an audio announcement, a warning sound, etc.

[0117] In step S47, the determination processing unit 26 determines whether the driver can perform normal driving based on the driver status provided by the monitoring unit 21.

[0118] In step S47, if the determination processing unit 26 determines that the driver is capable of normal driving, the determination processing unit 26 attracts attention, for example, by moving to the nearest waiting area. Furthermore, the determination processing unit 26 monitors whether the driver performs the corresponding action based on the attention attraction, and after the driver performs the action, the processing in step S42 is repeated, followed by the same processing.

[0119] On the other hand, if it is determined in step S45 that the driver's state is "loss of consciousness", or if it is determined in step S47 that the driver is unable to drive normally, then the process in step S49 is carried out.

[0120] In step S49, the determination processing unit 26 determines that the driver cannot drive normally, and causes the driving execution unit 13's driving processing unit 32 to perform an evacuation operation to the evacuation lane according to road signs, and issues an instruction to stop the vehicle. Therefore, the driving processing unit 32 controls the steering wheel control unit 33 and the speed control unit 34 based on information indicated by the external environment provided by the environment recognition unit 31, and executes automatic driving for stopping after entering the evacuation lane. The driving processing unit 32 controls the hazard flashing light control unit 35 to warn following vehicles. Then, when the vehicle completes stopping in the evacuation lane, the automatic driving termination process is terminated.

[0121] Next, we will refer to Figure 6 The flowchart illustrates the dynamic correction process for braking force.

[0122] When driving is started, processing begins, and in step S61, the driving processing unit 32 sets the braking coefficient of the braking operation to an initial value.

[0123] In step S62, the determination processing unit 26 obtains self-diagnostic results related to the vehicle's autonomous driving via the input unit 24, and in parallel, in step S63, the determination processing unit 26 obtains the driver's status from the monitoring unit 21 in order to determine whether the driver has the level of driving ability to recover to manual driving.

[0124] In addition, in parallel with steps S62 and S63, the environment recognition unit 31 determines the previous conditions of the environment and road surface associated with the planned driving route in step S64. For example, the environment recognition unit 31 determines the previous conditions of the environment and road surface based on past driving history, weather information, vehicle braking equipment maintenance status, traffic conditions, congestion conditions, etc.

[0125] In step S65, the driving processing unit 32 sets the road surface predicted friction coefficient based on the previous conditions of the environment and road surface as determined by the environment recognition unit 31 in step S64.

[0126] In step S66, the driving processing unit 32 estimates the braking distance based on the driving route, vehicle load (whether the vehicle is carrying heavy objects), etc.

[0127] After the processing in steps S62 to S66, the processing in step S67 is then performed, and it is determined that the processing unit 26 monitors the driver status provided by the monitoring unit 21 and the surrounding environment provided by the environment recognition unit 31.

[0128] In step S68, the processing unit 26 determines whether there is a change in the state, and if the processing unit 26 determines that there is no change in the state, the processing in step S67 is repeated.

[0129] If the processing unit 26 determines that the state has changed in step S68, then the processing in step S69 is performed, and it is determined that the processing unit 26 has determined which of the following has occurred: the braking distance has changed, the driver has changed, or factors other than the braking distance and the driver have changed.

[0130] If the processing unit 26 determines in step S69 that the braking distance has changed, then the processing in step S70 is performed. In step S70, the driving processing unit 32 adjusts the brake steering response coefficient to compensate for the change in braking distance.

[0131] If the processing unit 26 determines in step S69 that the driver has changed, then the processing in step S71 is performed. In step S71, the driving processing unit 32 sets a weighted coefficient correction value for the driver response characteristics to address the change in the driver.

[0132] In step S69, if the determination processing unit 26 determines that factors other than braking distance and driver have changed, then the process proceeds to step S72. In step S72, the determination processing unit 26 determines whether the change is caused by a series of actions desired by the driver.

[0133] If, in step S72, the processing unit 26 determines that the change is not caused by a series of operations desired by the driver, it confirms an update to the vehicle's braking characteristics based on the operations during continuous cruise, and then proceeds to step S73. Step S73 follows the processing in steps S70 and S71.

[0134] In step S73, the driving processing unit 32 sets an environmental application correction value for changes in circumstances, performs the processing of step S67 again, and then performs the same processing.

[0135] On the other hand, if the processing unit 26 determines in step S72 that the change is caused by a series of operations desired by the driver, then the processing in step S74 is performed. In step S74, the driving processing unit 32 intervenes in a series of braking operations of the vehicle, and then terminates the processing by performing a process to stop the vehicle. In this case, the driving processing unit 32 performs steering intervention assistance based on the amount of the driver's autonomous brake pedal operation, taking into account correction values ​​for stopping (brake steering response coefficient correction, driver response characteristic weighted coefficient correction value, and environmental application correction value for situational changes). For example, in the simplest case, if a change in the driver's situational recognition delay is detected, resulting in an average reaction delay of 0.2 seconds, correction is performed to increase the vehicle's braking force, enabling the vehicle to stop and decelerate in a short time at a distance corresponding to the braking distance traveled by the vehicle in a 0.2-second time period.

[0136] Figure 7 This is a diagram used to illustrate the termination sequence of autonomous cruise.

[0137] Entering a general manual driving lane extension while the driver's alertness is still low at the end of the autonomous driving lane is dangerous. Therefore, a mechanism for guiding vehicles to a mandatory stopping lane and evacuation path b by setting road lines at the end of the autonomous driving lane is effective.

[0138] In this evacuation lane guidance path, firstly, regardless of the driver's state of consciousness, the vehicle slows down and enters section b to guide the white road line to the evacuation lane for awake, unrecoverable drivers. The vehicle then proceeds through deceleration section c to stop, almost completely stopped in the half-stop lane d. Automatic guidance to an evacuation parking space (not shown) is then provided, enabling emergency evacuation even if the driver is not fully awakened from autonomous steering. When multiple vehicles are anticipated to enter consecutively, the vehicle is not brought to a complete stop in the half-stop lane d. Instead, the vehicle performs automatic steering for low-speed autonomous evacuation stopping by recognizing vacant spaces in adjacent evacuation parking areas, ensuring that following vehicles do not collide even when multiple vehicles arrive in succession. The road markings c and d shown in the figure are examples of dedicated road signs indicating the positions of deceleration sections and half-stop lanes using perception-based image processing or image processing with cameras, etc.

[0139] Provides deterministic functions to allow continuous cruising on ordinary roads by audibly, visually, and tactilely notifying the driver of an autonomous driving termination notice at least 30 seconds before the end of the autonomous driving lane (and preferably 2 or 3 minutes before normal driving), performing preparations to cancel the autonomous driving mode in segment a, and recognizing the result of prompting the driver to perform deterministic operations according to a predefined procedure, and, in the event that the driver's awakening recovery state is not preferred, enabling the vehicle to actually automatically enter the evacuation lane of segment b.

[0140] With sufficient pre-emptive awakening and recovery, the vehicle deviates from the lane markings in the direction of the evacuation white line of section b. Under the driver's conscious guidance, the vehicle performs normal manual driving on the extended ordinary road section and makes the driving track a continuous cruise.

[0141] As described above, the automatic driving control device 11 also has the function of determining the driver's ability to regain consciousness. In a vehicle with some kind of automatic driving function, the driver can continue to cruise even if the driver cannot maintain a high level of awareness of all driving-related equipment due to the vehicle's automatic driving system.

[0142] Typical examples of this feature are lane keeping (departure prevention) systems and forward-following systems that cover both stopping and starting (stopping and moving forward), such as CACC (Cooperative Adaptive Cruise Control). These devices have the effect of reducing the driver's burden, but the driver relies on the system and can drive normally and steadily forward even in a state of low awareness, and therefore the vehicle can be driven physically in a stable manner even when the driver's alertness is low.

[0143] Generally, when a person recovers from a low state of consciousness, they are often confused because they are unaware of their surroundings, or because they remained conscious during the dream. They are busy trying to understand their real-world surroundings and cannot make sudden, accurate assessments. Furthermore, they may perform actions that would not be expected in a waking state, such as reflexive muscle twitches in a dream. Therefore, as autonomous driving becomes widespread, drowsy driving in a deeper state and with a lower level of driving consciousness will occur. Thus, determining the driver's state of consciousness plays a crucial role during manual driving recovery, and it is necessary to establish the appropriate ability to assess the surrounding environment.

[0144] For example, a unit that intentionally inputs visually nonspecific information to a driver from an infrastructure environment (such as a road) and determines the driver's state of alertness based on whether the result appropriately relates to the driver's response to the content of the nonspecific information is effective as a method for actively and effectively performing appropriate environmental situation determination.

[0145] In one embodiment, variable symbols are placed on the road to determine the driver's level of consciousness / alertness, and the driver inputs these symbols into a determination device to determine the driver's state of alertness. For example, numerical values ​​are continuously displayed on two parts as the driver moves along the road, and the driver's level of consciousness is determined in response to the numerical values ​​and the results of the actions, taking into account the correct manipulation of buttons, accelerator, and steering wheel.

[0146] Visual feedback methods include moving and displaying multiple visual targets in a time sequence (e.g., sequentially opening traffic signs), and statistically determining the appearance of the driver's gaze trajectory in the driver's gaze direction in sync with the timing of the moving displays, and determining whether the driver has recovered to the gaze operation level characteristic of the driver's consciousness, wherein the displayed content of the multiple visual targets changes in multiple directions unspecific to the driver's position on the road and his surroundings. Insufficient arousal is determined in the presence of signs of decreased arousal level (such as prolonged delays in capturing the target object's gaze or large differences in gaze direction).

[0147] In another embodiment, the automatic driving cancellation reservation is performed in a buffer zone at the end of the automatic lane, and for example, the lane is designed to meander gently to the right and left on the dedicated road, and a determination is made as to whether the vehicle is driving along the lane in that zone under the driver's awareness.

[0148] The automated driving control device 11 can take measures during congestion and when an emergency automated driving evacuation parking lot is full. For example, when vehicles that cannot be switched back to manual driving are gathered in the evacuation parking lot, it is necessary to prevent the parking lot from becoming full and to prevent failure to ensure the evacuation of vehicles driven later.

[0149] Therefore, vehicles following behind switch to the general manual lane earlier based on the traffic volume of the dedicated lane. This effectively reduces the number of vehicles failing to wake up and enter the evacuation parking area by providing early wake-up guidance to vehicles in autonomous cruise control at a time earlier than the normally designated pre-prepared time. More specifically, vehicles failing to wake up from autonomous cruise control and enter evacuation parking spaces face limited space and the risk of evacuation parking spaces suddenly becoming full. Therefore, it is effective to have autonomous vehicles arriving from behind switch their driving trajectory to manual driving at an earlier time, thereby improving the reliability of the transition to the general road.

[0150] The autonomous driving control device 11 also has the ability to determine changes in the driver's road environment by using actively illuminated signs synchronized with approaching vehicles.

[0151] Reference Figure 8 This describes a driver's ability to determine changes in road conditions.

[0152] Figure 8 A shows the same Figure 7 The road corresponding to section a, and Figure 8 B shows the interior of a vehicle equipped with an automatic driving control device 11.

[0153] like Figure 8 As shown in Figure B, a camera 41 installed inside the vehicle is arranged to capture the driver's face. For example, the image captured by the camera 41 is provided to a monitoring unit 21, and the monitoring unit 21 identifies the driver's gaze, thereby recognizing the driver's state.

[0154] Therefore, for example, when the driver moves their line of sight by observing an illuminated sign, the monitoring unit 21 improves reliability by using multiple operations (e.g., from t=t1 to t=t3) to determine the response based on the display sequence. Figure 8 As shown in A, or by using operations (button operations in response to numerical values, etc.) in response to the displayed content (numerical values, etc.) to ensure the reliability of the wake-up determination.

[0155] Then, monitoring unit 21 determines whether the driver is visually and appropriately identifying the illuminated and displayed symbols from the infrastructure side, and when the driver performs an operation based on the symbol content, monitoring unit 21 determines that the state of alertness is high. On the other hand, when the response is slow and there is no corresponding response or the corresponding response is insufficient, monitoring unit 21 determines that the recovery ability is poor. Examples of easy human-machine interface operations that impose the least burden on the driver in determining the response include button operation, continuous and repeated opening and closing of the eyelids, slight right and left turns of the steering wheel, and shallow operation of the accelerator pedal.

[0156] As described above, the autonomous driving control device 11 can determine driving ability by analyzing the driver's level of awareness of the surrounding environment from the infrastructure-side environment input and in response to the input. More specifically, such as Figure 8 As shown, the autonomous driving control device 11 visually displays unique content to be recognized by the driver on a sign installed on the infrastructure side, and the driver responds to the display content with necessary actions, thereby determining the state of consciousness. Figure 8 Visual input has been shown as an example, but arousal status can also be determined in the same way using verbal responses to radio-based voice guidance and button directions.

[0157] The autonomous driving control device 11 determines the driver's environmental certainty based on the driver's awareness determination results, which are based on unique visual information about the road from the infrastructure being driven by the vehicle or auditory information given via units such as radio. It can also adjust the intervention ratio of the safe autonomous driving system based on the certainty of the determination results. Figure 6 (Correction of coefficients).

[0158] It should be noted that the monitoring unit 21 can not only perform this kind of gaze recognition by the driver, but also acquire the driver's reaction operation behavior (e.g., operation transition observation to various vehicle control input units (such as the vehicle's accelerator pedal operation state, brake pedal operation state, clutch operation state, and steering wheel operation state, as well as further driver's head posture state, gaze movement tracking, gaze saccade movement analysis, etc.)), and detect the decline in arousal level by detecting changes in arousal state from the time-varying changes during periods of high arousal. For example, when the arousal level declines, the optimal driving steering amount and vehicle speed begin to deviate due to the decline in awareness during driving operations, so unnatural sudden steering increases compared to normal slight steering, for example, the driver repeats sudden corrective operations to try to correct the deviation. In gaze saccade movement, the operation of repeatedly making the gaze gradually follow the target object at high speed is repeated, but when the arousal state declines, the saccade movement used to track obstacles with the gaze usually becomes sluggish.

[0159] More specifically, vehicles equipped with gaze recognition and obstacle detection systems track changes in the driver's gaze direction. Simultaneously, vehicle monitoring analyzes and estimates changes in the instantaneous matching degree of obstacles relative to the driver's direction based on the arrangement of obstacles in the vehicle's driving direction. Furthermore, it monitors changes in the stability of so-called gaze saccades (high-speed movements of the gaze), allowing for the determination of the driver's alertness level. When the vehicle is driving in a normal, stable steering direction, vehicles in the same driving direction are moving forward in the same direction as the vehicle in question, while the surrounding environment moves in a relative manner with a velocity vector in the opposite direction to that of the vehicle in question. Therefore, the driver continues driving when necessary to shift their gaze to the forward-moving vehicles and the surrounding environment.

[0160] The line-of-sight analysis described below is effective as a means of determining the driver's alertness. More specifically, during actual driving operations, the driver cannot see every piece of environmental information for a period of time. In reality, depending on the situation and also on the driver's experience, environmental awareness, and current capabilities, the line of sight moves rapidly, and the driver repeats these movements to identify the relationship between the position and shape of obstacles ahead and the environment, which is related to the degree of impact on the driver and the vehicle. Then, when further considering the degree of monitoring regarding how the driver's own line of sight matches the direction of the obstacle detected by the aforementioned obstacle recognition device, line-of-sight movement targeting obstacle candidates is necessary for subsequent operations during the driver's line of sight movements.

[0161] In this context, saccades are operations that rapidly move the direction of the gaze center to the direction that is considered necessary for people to pay attention to in the environmental field of vision, while saliency maps are schemes used to estimate the parts of an image that people are likely to pay attention to.

[0162] As described above, recognizing the movement of the driver's gaze and performing driver arousal determination, such as in the case of a vehicle equipped with an ADAS (Advanced Driver Assistance System) and performing forward obstacle direction detection, is expected to produce effects such as time-series analysis of the correlation between monitoring and obstacle recognition equipment and the driver's gaze, and time-series changes in the evaluation of the agility assessment points of the driver's gaze sweeping movements. In any case, the driver's ability to perceive the driving environment has driver-specific physical characteristics; therefore, in state determination, the vehicle can be configured to learn a threshold for determining the point of decline in the driver's state of consciousness.

[0163] In this case, reference will be made Figures 9 to 23 Further explanation is provided of the various processes performed in autonomous driving control using this technology.

[0164] Figure 9 This is a block diagram illustrating a configuration example of an autonomous driving control system applying this technology.

[0165] like Figure 9As shown, the autonomous driving control system 101 is configured to include a sonar device 111, a radar device 112, a lidar device 113, a single camera 114, a stereo camera 115, a ToF (Time-of-Flight) camera 116, an environmental sensor 117, an input device 118, a short-range communication device 119, a mobile communication device 120, a beacon device 121, a measuring device 122, a gyroscope sensor 123, an acceleration operation detection unit 124, a braking operation detection unit 125, a steering operation detection unit 126, a driver monitoring device 127, an engine activation device 128, a brake activation device 129, a steering activation device 130, a light control unit 131, an attention attraction processing device 132, an instrument panel 133, a head-up display 134, a virtual image display unit 135, and an electronic control unit 151.

[0166] Sonar device 111, radar device 112 and lidar device 113 detect objects around the vehicle (e.g., another vehicle, pedestrians, etc.) to enable the vehicle to perform autonomous driving and provide the relative distance to the detected objects to the autonomous driving controller 154 of the electronic control unit 151.

[0167] A single camera 114, a stereo camera 115, and a ToF camera 116 identify objects shown in images acquired by capturing the area around the vehicle, enabling the vehicle to perform autonomous driving, and provide information about the identified objects to the autonomous driving controller 154 of the electronic control unit 151. For example, the stereo camera 115 and the ToF camera 116 measure the distance to the identified objects and provide the distance 151 to the identified objects to the autonomous driving controller 154 of the electronic control unit 151.

[0168] The environmental sensor 117 detects the environment around the vehicle, such as temperature, humidity, weather, road conditions, etc., and provides the environmental detection results to the electronic control unit 151.

[0169] Input device 118 includes, for example, a microphone for the driver to give input using sound, and an operation unit for the driver to give input using buttons and a touch panel. When the driver gives input, input device 118 provides the input content to electronic control unit 151.

[0170] The short-range communication device 119 uses DSRC (Dedicated Short-Range Communication), a wireless communication technology specifically designed for vehicle radio communication, to communicate with ITS (Intelligent Transportation System) points installed along the road and to obtain traffic congestion information, regulation information, etc.

[0171] The mobile communication device 120 communicates according to the communication standards (3G / 4G) based on its communication by the cellular phone. For example, the mobile communication device 120 obtains various information through networks such as the Internet.

[0172] For example, beacon device 121 communicates with roadside equipment installed on the roadside to support safe driving and obtain various traffic information.

[0173] The measuring device 122 uses a satellite navigation system such as GPS (Global Positioning System) to measure the vehicle's current position, for example, by using artificial satellites.

[0174] The gyroscope sensor 123 detects the vehicle's acceleration and angular velocity used in autonomous position analysis processing in order to analyze the vehicle's current position autonomously.

[0175] Acceleration operation detection unit 124 detects the driver's operation using the accelerator, braking operation detection unit 125 detects the driver's operation using the brake, and steering operation detection unit 126 detects the driver's operation using the steering gear. Acceleration operation detection unit 124, braking operation detection unit 125, and steering operation detection unit 126 acquire, for example, the timing and amount of the operation performed by the driver as operation detection results, and provide these results to the driving behavior analysis unit 152 of the electronic control unit 151.

[0176] The driver monitoring device 127 monitors the driver by using various sensors (e.g., alcohol sensor, pulse sensor, odor sensor, etc.) to detect the driver's alertness. For example, the driver monitoring device 127 uses a camera to capture images of the driver's face, identify the driver's gaze, and monitor the movement of the gaze. The driver monitoring device 127 then provides the monitoring results obtained by monitoring the driver to the driver state determination unit 153 of the electronic control unit 151.

[0177] The engine activation device 128 controls the starting of the vehicle's engine according to the instructions given by the automatic driving controller 154 of the electronic control unit 151.

[0178] The brake activation device 129 controls the activation of the vehicle's brakes according to the instructions given by the automatic driving controller 154 of the electronic control unit 151.

[0179] The steering activation device 130 controls the activation of the vehicle's steering system according to instructions given by the automatic driving controller 154 of the electronic control unit 151.

[0180] For example, the light control unit 131 controls the flashing of the hazard lights when the vehicle enters the evacuation lane, based on instructions given by the automatic driving controller 154 of the electronic control unit 151. Figure 17 As shown.

[0181] Attention attraction processing device 132 performs processing to attract the driver's attention according to instructions given by the autopilot controller 154 of electronic control unit 151. For example, attention attraction processing device 132 provides tactile feedback such as vibrating the steering wheel and seat and pulling the seat belt.

[0182] The instrument panel 133 includes various instruments such as a speedometer and a navigation display arranged thereon, and, for example, the instrument panel 133 provides notifications to the driver by displaying messages on the display.

[0183] For example, the head-up display 134 is a display unit located above the windshield and displays messages notified to the driver, see reference. Figure 18 Explanation of virtual targets, etc.

[0184] The virtual image display unit 135 displays virtual images that add to the real world in an overlay manner that can be seen from the real world by the driver.

[0185] The electronic control unit (ECU) 151 can have various functions by executing programs for controlling the various blocks constituting the automatic driving control system 101. For example, in Figure 9 In the example, the electronic control unit 151 includes functions as a driving behavior analysis unit 152, a driver state determination unit 153, and an automatic driving controller 154.

[0186] The driving behavior analysis unit (operation behavior analyzer) 152 analyzes the driver's driving behavior (driving operations, behaviors, etc.) based on the operation detection results provided by the acceleration operation detection unit 124, braking operation detection unit 125, and steering operation detection unit 126. Then, the driving behavior analysis unit 152 provides the analysis results obtained by analyzing the driver's driving behavior to the driver state determination unit 153.

[0187] The driver state determination unit 153 performs driver state processing to determine the driver's awakening state based on monitoring results provided from the driver monitoring device 127 and analysis results provided from the driving behavior analysis unit 152. Then, the driver state determination unit 153 provides the determined driver state to the automatic driving controller 154. Although not shown in the figure, the input to the driver monitoring device 127 is considered to include not only recognition of gaze and face but also the driver's electrocardiogram, thermal imaging, body odor, respiration, skin temperature distribution, and respiratory status.

[0188] The autonomous driving controller 154 provides various instructions required for the autonomous driving of the vehicle to the engine activation device 128, brake activation device 129, steering activation device 130, light control unit 131 and attention attraction processing device 132 based on the determination results of the driver state provided by the driver state determination unit 153.

[0189] It should be noted that the various processes performed by the driver state determination unit 153 and the automatic driving controller 154 will be described in detail with reference to the flowcharts described later.

[0190] Figure 10 This is a flowchart illustrating the initial determination process performed by the automatic driving control system 101.

[0191] First, in step S101, the automatic driving control system 101 performs setting input processing using voice-based or operation-based input from the input device 118, indications on a display set in the instrument panel 133, etc., to input the settings necessary to initiate automatic driving. For example, in the setting input processing, as referred to... Figure 11 As illustrated in the flowchart, the process includes selecting a route to the destination and deciding whether to use autonomous driving in emergency evacuation mode.

[0192] In step S102, the autonomous driving controller 154 refers to map data stored in a storage unit (not shown) and determines whether an autonomous driving lane exists in the route selected in step S101.

[0193] In step S102, if the autonomous driving controller 154 determines that an autonomous driving lane exists, then the process of step S107 is performed; and if the autonomous driving controller 154 determines that no autonomous driving lane exists, then the process of step S103 is performed.

[0194] In step S103, the automatic driving controller 154 determines to initiate cruise and driving in a manual-only driving mode where the vehicle is driven solely by the driver, and terminates the initial determination process. Thereafter, the automatic driving control system 101 restricts the automatic driving performed by the automatic driving controller 154 and drives the vehicle manually by the driver.

[0195] In parallel with step S101, the autonomous driving controller 154 performs vehicle self-device diagnostic processing in step S104 for diagnosing the equipment necessary for performing autonomous driving. For example, in the vehicle self-device diagnostic processing, it performs detection such as... Figure 9 This is a confirmation of whether all the blocks constituting the automatic driving control system 101 are operating normally.

[0196] In step S105, the autonomous driving controller 154 determines whether it can autonomously drive the equipment based on the diagnostic results of the vehicle self-device diagnostic processing in step S104. For example, if all blocks constituting the autonomous driving control system 101 are operating normally, the autonomous driving controller 154 determines that it can autonomously drive the equipment; if at least one block is not operating normally, the autonomous driving controller 154 performs a risk diagnosis; and if the safe driving risk is not equal to or less than a predetermined level, the autonomous driving controller 154 determines that it cannot autonomously drive the equipment.

[0197] If the autopilot controller 154 determines in step S105 that the equipment cannot perform autopilot, then the process of step S103 is performed, and cruise and driving are started in manual dedicated driving mode in the same manner as described above, and the initial determination process is terminated.

[0198] In parallel with steps S101 and S104, in step S106, the autonomous driving controller 154 performs driver capability diagnosis processing to diagnose the driver's driving ability based on the driver state determination result provided by the driver state determination unit 153.

[0199] Following the processing in step S106, or if it is determined in step S105 that the equipment cannot perform automatic driving, the processing in step S107 is then executed. In step S107, the automatic driving controller 154 performs a comprehensive recovery capability assessment process to predict the recovery capability from automatic driving to manual driving (described later). Figure 12 (Processing of flowcharts), and conduct a comprehensive evaluation.

[0200] In step S108, the autonomous driving control unit 154 predicts and determines whether the driver has the ability to resume manual driving after the autonomous driving is terminated, based on the evaluation results of the comprehensive evaluation of recovery ability in step S107.

[0201] If the automatic driving controller 154 determines in step S108 that the driver has the ability to resume manual driving, then the processing in step S109 is executed.

[0202] In step S109, the autonomous driving controller 154 determines to initiate cruise and driving in an autonomous driving lane entry permitted mode, permitting autonomous driving within the autonomous driving lane selected in step S101, and terminates the initial determination process. Thereafter, in the autonomous driving control system 101, before the vehicle enters the autonomous driving lane, the aforementioned road environment R2 is a buffer zone road between the ordinary road and the fully autonomous driving lane. Within this buffer zone road, as a result of the aforementioned determination process, it is determined that recovery capability exists, and the vehicle engages in autonomous driving performed by the autonomous driving controller 154.

[0203] On the other hand, in step S108, the automatic driving controller 154 determines that the driver does not have the ability to return to manual driving, and then performs the processing in step S110.

[0204] In step S110, the automatic driving controller 154 determines whether the emergency evacuation mode was selected in the setting input processing of step S101. Figure 11 ).

[0205] If the emergency evacuation mode is selected in step S110, the process of step S103 is then executed, and cruise and driving are started in manual dedicated driving mode in the same manner as described above (and if the vehicle is already being driven, continue), and the initial determination process is terminated.

[0206] On the other hand, in step S110, it is determined that the emergency evacuation mode has been selected, and then the processing in step S111 is executed. In step S111, the automatic driving controller 154 performs the emergency evacuation mode activation process (described later). Figure 13 (Processing of the flowchart) to make the settings or changes required to activate the emergency evacuation mode.

[0207] In step S112, the autonomous driving controller 154 determines to initiate cruise and driving in emergency evacuation mode and terminates the initial determination process. Subsequently, in the autonomous driving control system 101, after the vehicle enters the autonomous driving lane, the autonomous driving performed by the autonomous driving controller 154 is abnormally executed. When the vehicle enters driving mode in emergency evacuation mode, the state transition and conditions are recorded and saved.

[0208] Subsequently, Figure 11 It is used for explanation Figure 10 The flowchart for setting the input process in step S101.

[0209] In step S121, when the driver inputs the destination location via voice or operation, the input device 118 receives the input and sets the destination location. Then, the input device 118 provides the location information about the destination location to the automatic driving controller 154.

[0210] In step S122, the autonomous driving controller 154 searches for a route from the current location to the destination location based on the location information about the destination location provided in step S121, referring to map data stored in a memory unit (not shown). Then, the autonomous driving controller 154 displays, for example, multiple route candidates obtained as search results on a display provided on the dashboard 133.

[0211] In step S123, when the driver selects a route from multiple candidate routes using voice or manual input, the input device 118 notifies the autonomous driving controller 154 of the received route. The autonomous driving controller 154 then selects the user-selected route as the vehicle's driving route.

[0212] In step S124, the autopilot controller 154 determines whether it has received an emergency evacuation command to specifically execute autopilot when the driver's driving ability is impaired. For example, the autopilot controller 154 displays a message on a display provided on the dashboard 133 asking whether to specifically execute autopilot when the driver's driving ability is impaired, and makes a determination based on user input in response to the message.

[0213] If it is determined in step S124 that no emergency evacuation instruction will be input, the setting input process is actually terminated. On the other hand, if it is determined in step S124 that an emergency evacuation instruction will be input, the autonomous driving control unit 154 sets an emergency evacuation sign in step S125, and then terminates the setting input process. In this embodiment, the destination location for driving and route selection is set through a display interface and command input. However, in the future, if a seamless and simple method is implemented, HMI can be implemented using voice guidance, voice input, gesture input, etc. Furthermore, it is also possible that, for example, the destination location can be automatically narrowed down from the previous plan before boarding the vehicle, and suggestions can be presented. However, the embodiment is not limited to this example.

[0214] Subsequently, Figure 12 It is used for explanation Figure 10 The flowchart for the comprehensive assessment of recovery capabilities in step S107.

[0215] In step S131, the autonomous driving controller 154 predicts that in Figure 11 The fully automated driving lane selected in step S123 ends, and the predicted time for switching to manual driving occurs. Note that this will be explained later. Figure 23 Provide a detailed explanation of the forecast time.

[0216] In step S132, the driver state determination unit 153 obtains the driver state from various data monitored by the driver monitoring device 127 and provides the driver state to the autonomous driving controller 154. For example, the driver state determination unit 153 obtains biometric information about the driver (such as the stability of the gaze, the level of alcohol intake, pulse, facial recognition fatigue measurement results, odor measurement results, driver response evaluation based on the human-machine interface, etc.) as the driver state.

[0217] In step S133, the autonomous driving controller 154, based on the driver's state provided in step S132, comprehensively calculates the driver's fatigue level, arousal level, etc., at the predicted time predicted in step S131. Then, as the evaluation result of the comprehensive recovery capability evaluation process, the autonomous driving control unit 154 obtains the driver's manual driving recovery capability predicted based on the calculated fatigue level and terminates the comprehensive recovery capability evaluation process.

[0218] Subsequently, Figure 13 It is used for explanation Figure 10 The flowchart for the emergency evacuation mode activation process in step S111.

[0219] In step S141, even if a driver's driving ability deficiency is observed, there is a possibility of an emergency situation. Therefore, the automatic driving controller 154 makes necessary settings to perform abnormal handling specifically to enable autonomous driving. For example, the automatic driving controller 154 sets a suppressed upper speed limit for specific emergency automatic driving, which is more suppressed than the upper speed limit in normal automatic driving.

[0220] In step S142, the automatic driving controller 154 records the anomaly handling in a non-erasable manner. By performing this recording, for example, it is possible to prevent driver misuse by continuously performing anomaly handling in terms of mechanism and operation.

[0221] In step S143, the autonomous driving controller 154 performs cruise preparation for a specific emergency autonomous driving situation. For example, in this scenario, the driver may be unable to take action during an emergency and at the end of autonomous driving in the lane. Therefore, the autonomous driving controller 154 performs cruise preparation, for example, by changing the limits during autonomous driving and prioritizing the route path in which the vehicle can be driven more safely even with low driver intervention. There is a risk that the driver may be unable to take appropriate action when some abnormal situation occurs during autonomous driving; therefore, when autonomous driving is engaged, the autonomous driving controller 154 performs cruise preparation to change the driving speed limit and braking characteristics.

[0222] In step S144, the autonomous driving controller 154 determines the road conditions, checks the route as needed, and then terminates the emergency evacuation mode initiation process.

[0223] For reference Figures 10 to 13 The automated driving control system 101 performs the processing required to initiate automated driving. Subsequently, when the vehicle enters the automated driving lane, automated driving is initiated. Then, in order to predict the time when automated driving will terminate and the vehicle will switch to manual driving, and to notify the driver of that time, the automated driving controller 154 can perform automated driving termination prediction processing to obtain a predicted time when automated driving will end.

[0224] Subsequently, Figure 14 This is a flowchart used to illustrate the prediction process for autonomous driving termination.

[0225] For example, in step S151, the autonomous driving controller 154 acquires data indicating the latest current position measured by the measuring device 122.

[0226] In step S152, the autonomous driving controller 154 performs autonomous position analysis processing to derive the current position from the autonomous driving movement based on the vehicle's acceleration and angular velocity detected by the accelerometer sensor (not shown) and the gyroscope sensor 123, and the distance traveled measured by the odometer (not shown). When the receiving sensitivity is good, this is not limited to autonomous position analysis processing, and the current position can also be obtained from GNSS (not shown) or similar sources.

[0227] In step S153, the autonomous driving control unit 154 detects the position deviation in the map data relative to the current position derived in steps S151 and S152, and performs position adjustment in order to identify the current position on the map data (registration analysis).

[0228] In step S154, the autonomous driving controller 154 inputs the end position of the fully autonomous driving lane during driving based on the selected route.

[0229] In step S155, the autonomous driving controller 154 determines whether the vehicle is approaching the end position of the fully autonomous driving lane entered in step S154 based on the current position identified in step S153.

[0230] In step S155, if the autonomous driving controller 154 determines that the vehicle has not approached the end position of the fully autonomous driving lane, the process of step S151 is executed again, and the same process is repeated thereafter.

[0231] On the other hand, if the autonomous driving controller 154 determines in step S155 that the vehicle is approaching the end position of the fully autonomous driving lane, then the processing in step S156 is executed. In step S156, the autonomous driving controller 154 initiates a termination preparation process for preparing for the termination of autonomous driving, including switching from autonomous driving to manual driving, and terminates the autonomous driving termination prediction process.

[0232] As described above, the autonomous driving controller 154 monitors detailed timing for initiating termination preparation processing to prepare for terminating autonomous driving based on current location data prior to the predicted arrival time of the fully autonomous driving lane. Then, when the time is less than a predetermined set time before the vehicle reaches the end position of the fully autonomous driving lane, the termination preparation processing is initiated.

[0233] Subsequently, Figure 15 This is a flowchart used to illustrate the termination preparation process.

[0234] In step S161, the driver state determination unit 153 notifies the driver, for example, by outputting an audible signal that the fully automated driving lane has ended. Alternatively, the driver can be clearly notified of the end of the fully automated driving lane by displaying a message on a monitor or by turning on the lights, using methods other than audible output.

[0235] In step S162, the driver state determination unit 153 attracts attention to inform the driver of the extent to which the fully automated driving lane ends so that the driver does not feel uncomfortable.

[0236] In step S163, the driver state determination unit 153 waits for the time when the driver needs to respond to the attention given in step S162, and performs the determination process.

[0237] In step S164, the driver status determination unit 153 obtains the latest driver status based on the monitoring results provided by the driver monitoring device 127 and the analysis results provided by the driving behavior analysis unit 152.

[0238] In step S165, the driver state determination unit 153 determines whether the attention attraction is repeated to a defined number of times to a degree that does not make the driver feel uncomfortable.

[0239] If the defined number of attention-attracting attempts is not reached in step S165, step S162 is performed again, and this process is repeated until the defined number of attempts is reached. Then, when the defined number of attention-attracting attempts is reached in step S165, step S166 is then performed.

[0240] In step S166, the driver state determination unit 153 determines, based on the latest driver state obtained in step S164, whether the driver's awakening recovery is sufficient and whether the driver can perform manual driving. If the driver state determination unit 153 determines in step S166 that the driver may not perform manual driving, then the processing in step S167 is executed.

[0241] In step S167, the driver state determination unit 153 performs a wake-up operation to induce the driver to wake up. For example, in order to induce the driver to wake up, a more unpleasant alarm (termination warning) may be output, or a more intense tactile feedback may be given, than the attention-grabbing alarm in step S162.

[0242] In step S168, the driver state determination unit 153 keeps the processing in a waiting state based on the time required for the driver to respond to the wake-up operation in step S167.

[0243] In step S169, the driver status determination unit 153 obtains the latest driver status based on the monitoring results provided by the driver monitoring device 127 and the analysis results provided by the driving behavior analysis unit 152.

[0244] In step S170, the driver state determination unit 153 determines whether the wake-up operation to wake up the driver has been repeated a defined number of times.

[0245] If it is determined in step S170 that the wake-up operation to induce the driver to wake up does not need to be repeated a defined number of times, then step S167 is performed again, and the process is repeated until the defined number of times has been reached. Then, if it is determined in step S170 that the wake-up operation to induce the driver to wake up has been repeated a defined number of times, then step S171 is performed.

[0246] In step S171, the driver state determination unit 153 determines whether the driving ability has been restored to a level that allows the driver to drive manually based on the latest driver state obtained in step S169.

[0247] If, in step S171, the driver status determination unit 153 determines that the driving ability has not recovered to a level that allows the driver to perform manual driving, then proceed to step S172. In step S172, the driver status determination unit 153 notifies the automatic driving control unit 154 that the driver cannot perform manual driving, and the automatic driving control unit 154 performs automatic evacuation procedures (described later). Figure 16 (Processing of the flowchart) and terminate the termination preparation process.

[0248] On the other hand, if the driver state determination unit 153 determines in step S171 that the driving ability has been restored to a level that allows the driver to drive manually, or if it determines in step S166 that the driver is able to drive manually, then the processing in step S173 is performed.

[0249] In step S173, the autonomous driving controller 154 allows a seamless transition from the autonomous driving lane to manual driving by the driver.

[0250] In step S174, the automatic driving controller 154 seamlessly continues to execute the process of switching to manual driving performed by the driver.

[0251] In step S175, the driver status determination unit 153 continuously monitors the driver's manual driving situation and the driver's response to the surrounding environment.

[0252] In step S176, the driver state determination unit 153 determines whether the driver has fully recovered and whether the manual driving ability has been continuously maintained. If the driver state determination unit 153 determines in step S176 that the manual driving ability has been continuously maintained, the preparation process is terminated.

[0253] On the other hand, if the driver state determination unit 153 determines in step S176 that the manual driving capability is not being continuously maintained, then the processing in step S177 is performed. More specifically, in this case, if a temporary decrease in the driver's arousal level is detected, the automatic driving controller 154 can further perform attention attraction in step S177, and can perform processing based on the driver's state to prompt the driver to stop the vehicle or take an earlier rest. Then, while waiting to stop and rest, the termination preparation process terminates. In this case, although not shown in the flowchart, since the system operation and the driver's state can continue after performing step S177 to further attract attention, step S175 is performed again. Furthermore, continuous driver monitoring can be performed once the vehicle transitions from automatic driving to manual driving and the sequence is terminated.

[0254] Subsequently, Figure 16 It is used for explanation Figure 15 The flowchart shows the automatic evacuation process performed in step S172.

[0255] In step S181, for example, as part of the process associated with switching to an evacuation lane, the automatic driving controller 154 flashes hazard warning lights to notify following vehicles that an automatic evacuation process is in progress.

[0256] For example, when each of the left and right turn indicator lights passes through... Figure 17When at least three light sources are arranged side by side as shown, the automatic driving controller 154 and indicator light control unit 131 control the hazard lights to flash in a sequential order in which the light sources are turned on in the radiation direction. By using the visual means of sequential illumination, an amplified illusion can be created, and a similar illusion effect can be given to the drivers of vehicles behind, thus allowing the drivers to take action and prepare in advance.

[0257] In step S182, the automatic driving controller 154 performs automatic driving to move the vehicle to the evacuation lane and temporarily prohibits manual driving except for the driver's soft braking. Therefore, this prevents the anticipated accident from occurring when the driver forces manual driving.

[0258] In step S183, the autopilot controller 154 applies a highly intense level of attention attraction to the point of causing the driver discomfort. For example, the autopilot controller 154 applies this intense attention attraction by giving instructions to the brake activation device 129 to rapidly decelerate or by applying tactile input of seat vibration. As described above, when evacuation driving using autopilot is initiated without the driver's manual driving being resumed, this highly intense attention attraction causes the driver discomfort, thus inducing a psychological process in the driver to consciously prioritize a seamless transition to manual cruise control to avoid discomfort, and is expected to have the effect of prioritizing a return to manual driving.

[0259] In step S184, the driver state determination unit 153 acquires the driver state based on the monitoring results provided by the driver monitoring device 127 and the analysis results provided by the driving behavior analysis unit 152 in response to a highly intense attentional attraction.

[0260] In step S185, the driver state determination unit 153 determines whether the driver's manual driving ability has been fully restored by a highly intense attentional pull.

[0261] If, in step S185, the driver state determination unit 153 determines that the driver's manual driving ability has been fully restored, then proceed to step S186. In step S186, the automatic driving controller 154 allows manual driving after a predetermined delay time, and for example, allows manual driving to leave the evacuation parking space, and terminates the automatic evacuation process.

[0262] On the other hand, if the driver state determination unit 153 determines in step S185 that the driver's manual driving ability has not been fully restored, then the process proceeds to step S187, and the automatic driving controller 154 performs automatic parking processing. In the automatic parking processing, for example, the automatic driving controller 154 uses automatic driving to navigate to the evacuation lane, searches for available parking spaces in the parking lot, and uses automatic guidance to park the vehicle in the desired parking space. For example, at the end of the evacuation parking area and the fully automated driving dedicated lane, evacuation vehicles can queue continuously; therefore, evacuation deceleration vehicles approach and evacuate from the main lane of the fully automated driving dedicated lane. Afterwards, the evacuation deceleration vehicles perform automatic cruise to the desired parking space and stop there.

[0263] In step S188, the automatic driving controller 154 sets a timer for delaying subsequent driving based on the time required for the driver to fully regain manual driving ability and terminates the automatic evacuation process. If a mechanism exists that locks the vehicle's continuous cruise as a penalty in the event of a decline in awareness and the vehicle automatically making an emergency evacuation stop, until the driver is ready to restart from the evacuation parking lot, psychological processes also play a role here for the driver to seamlessly choose manual continuous cruise by avoiding automatic evacuation. The tolerance level for this delay timer-based preventative function of the driver may be tolerated by some and not by others due to cultural factors. Therefore, the timer-based delay setting is preferably combined with a mechanism that allows for variable settings based on the vehicle's market. On the other hand, a driver may attempt to force manual driving due to insufficient driving ability to avoid the subsequent driving delay caused by such penalty; therefore, limited restrictions are preferably applicable to the transfer of authority to the driver associated with driving from a period of high attention engagement to automatic stop transition, thus enabling the transfer.

[0264] In this context, when determining whether the driver is in a state of alertness sufficient for normal driving, the illuminated sign is used as a target in the driver's line of sight, as referenced above. Figure 8 However, alternatively, a virtual target can also be used as the driver's line of sight. More specifically, a virtual target as the driver's line of sight can be displayed on the head-up display 134, and the driver's line of sight can be tracked to determine whether the driver has the ability to drive manually.

[0265] Subsequently, Figure 18 This is a flowchart illustrating the manual driving ability determination process used to determine whether a driver has the ability to drive manually.

[0266] In step S191, the driver state determination unit 153 resets the normal visual recognition counter AC (response counter), which counts the number of times the driver normally visually recognizes virtual targets (AC = 0).

[0267] In step S192, the driver state determination unit 153 sets up the display of a virtual target on the head-up display 134 and sets a virtual target counter nt (nt=1) for measuring the number of times the virtual target is displayed.

[0268] In step S193, the driver state determination unit 153 gently wakes up the driver by using, for example, seat vibration, steering impact, attention attraction warning, etc.

[0269] In step S194, the driver state determination unit 153 displays a virtual target for determining the driver's level of alertness on the head-up display 134 in a manner that is in the target direction θn when viewed from the driver's perspective.

[0270] In step S195, the driver state determination unit 153 maintains a processing wait based on the driver's response time to the virtual target displayed on the head-up display 134 in step S194.

[0271] In step S196, the driver state determination unit 153 tracks the driver's gaze based on the monitoring results provided by the driver monitoring device 127.

[0272] In step S197, the driver state determination unit 153 analyzes the visual tracking state after time tn, which indicates the tracking state of the driver's line of sight to the virtual target displayed in the target direction θn.

[0273] In step S198, the driver state determination unit 153 determines whether there is visual recognition of the driver in the target direction θn based on the tracking state analyzed in step S197.

[0274] If, in step S198, the driver state determination unit 153 determines that there is no visual recognition of the driver in the target direction θn, then the processing in step S199 is performed.

[0275] In step S199, the driver status determination unit 153 determines whether a preset predetermined permission confirmation number has been exceeded in order to confirm whether the driver has correctly visually recognized the virtual target.

[0276] If, in step S199, the driver state determination unit 153 determines that the predetermined number of authorized confirmations has been exceeded, then the processing in step S200 is executed. In step S200, since the driver is typically unable to visually recognize virtual targets and is not in an awake state, the driver state determination unit 153 initiates a driver wake-up process to awaken the driver. Figure 19 ), and terminate manual driving capability determination processing.

[0277] On the other hand, if the driver status determination unit 153 determines in step S199 that the predetermined permission confirmation number has not been exceeded, then the process of step S201 is performed. In step S201, the driver status determination unit 153 increments the virtual target counter nt (nt = nt + 1), and performs the process of step S193 again, and then repeats the same process.

[0278] On the other hand, if the driver state determination unit 153 determines in step S198 that there is visual recognition of the driver in the target direction θn, then the processing in step S202 is performed. In step S202, the driver state determination unit 153 increments the normal visual recognition counter AC (AC = AC + 1).

[0279] In step S203, the driver state determination unit 153 determines whether the number of visual recognition confirmations of the driver in the target direction θn has reached the predetermined number of repetitions (ACmin) that can determine whether the driver has been awakened (AC≥ACmin).

[0280] If, in step S203, the driver state determination unit 153 determines that the number of visual recognition confirmations of the driver in the target direction θn has not reached the predetermined number of repetitions, then the processing of step S201 is performed, and the virtual target counter nt is incremented, and then the processing of step S193 is performed again.

[0281] On the other hand, if the driver state determination unit 153 determines in step S203 that the number of visual recognition confirmations of the driver in the target direction θn has reached a predetermined number of repetitions, then the processing in step S204 is performed. In step S204, the driver state determination unit 153 notifies the autopilot controller 154 that the driver has indeed been awakened, and the autopilot controller 154 allows the switch to manual driving (transfer of operating authority) and terminates the manual driving capability determination process.

[0282] As described above, the autonomous driving control system 101 uses the driver's awake state as a confirmation means, and can feed the driver the infrastructure environment or virtual visual recognition targets in the vehicle as needed. The driver state determination unit 153 can determine the driver's reaction using at least one of the units including facial, gaze, and dynamic gaze analysis, and can determine whether the driver makes a determination in an awake state based on the fed information.

[0283] It should be noted that symbols such as numbers and characters illuminated and displayed around the road can also be used as feed content, and the driver provides operational feedback by making a conscious judgment based on the displayed content, and the driver state determination unit 153 determines the state of wakefulness. Alternatively, for example, the target can be a composite element such as numbers, symbols, signals, etc., and the driver can make a confirmation / thinking judgment, and the state of wakefulness can be determined based on whether the confirmation has been correctly performed.

[0284] Subsequently, Figure 19 This is a flowchart illustrating a driver alerting process that alerts a driver who is not normally able to visually recognize virtual targets.

[0285] In step S211, the driver state determination unit 153 attracts attention by, for example, outputting an audible alarm for wake-up and activating a tactile device.

[0286] In step S212, the driver state determination unit 153 maintains processing wait based on the reaction time required for the driver to respond to the attention attraction in step S211. After that, the driver state determination unit 153 monitors the driver's state changes based on the monitoring results provided by the driver monitoring device 127.

[0287] In step S213, the driver state determination unit 153 analyzes the stability of the driver's line of sight, pedal operation, steering operation, etc. Then, based on the analysis results, the driver state determination unit 153 comprehensively determines whether the driver has returned to an awake state.

[0288] In step S214, the driver state determination unit 153 determines whether the confirmation of whether the driver has returned to an awake state as a result of the comprehensive determination made in step S213 has been completed, or whether the confirmation has been performed a predetermined number of times.

[0289] If, in step S214, it is determined that the confirmation of whether the driver has returned to an awake state has not been completed, or if it is determined that the confirmation has not been performed a predetermined number of times, the process of step S211 is repeated, and then the same process is repeated.

[0290] On the other hand, if it is determined in step S214 that the confirmation of whether the driver has returned to a state of wakefulness has been completed, or if it is determined that the confirmation has been performed a predetermined number of times, then the processing in step S215 is executed.

[0291] In step S215, the driver status determination unit 153 notifies the automatic driving controller 154 to initiate automatic evacuation procedures. Figure 16 ), and terminate the driver wake-up process.

[0292] Incidentally, the autonomous driving control system 101 is preferably configured to allow the vehicle to initiate manual driving under various conditions and to handle situations where, during autonomous driving in the autonomous lane, automatic evacuation is preferably performed even at locations other than the end position of the autonomous driving lane. For example, perfect autonomous driving is difficult to achieve when unexpected abnormal weather prevents the expected autonomous driving from being performed, when road signs (such as white lines) necessary for autonomous driving are damaged due to an accident, or when parts constituting the autonomous driving control system 101 are damaged.

[0293] In response to this situation, when the autonomous driving control system 101 determines that it is difficult to continue autonomous driving, the autonomous driving control system 101 determines the driver's awake state, and when the driver is determined to have manual driving ability, the autonomous driving control system 101 switches from autonomous driving to manual driving, and if the driver does not have manual driving ability, the autonomous driving control system 101 takes action to carry out emergency evacuation.

[0294] Subsequently, Figure 20 This is a flowchart illustrating the emergency evacuation and transfer process that begins during normal driving.

[0295] In step S221, the autonomous driving controller 154 monitors safety risks when continuously performing autonomous driving. For example, the autonomous driving controller 154 confirms that... Figure 9 Whether all blocks of the automated driving control system 101 shown are operating normally and is confirmed in advance before reaching the location in question, in order to ascertain whether any anomalies have occurred in the road environment along the driving route by communicating with ITS (Intelligent Transportation System) points installed along the road via mobile communication devices such as 120.

[0296] In step S222, the autonomous driving controller 154 calculates the degree of risk as the extent of safety risks that may occur due to continued autonomous driving, based on the monitoring results in step S221.

[0297] In step S223, the autonomous driving controller 154 determines whether the risk level calculated in step S222 has increased to a predetermined value or above.

[0298] If the autonomous driving controller 154 determines in step S223 that the risk level calculated in step S222 has not increased to the predetermined value, it will repeat the process of step S221 and repeat the periodic monitoring.

[0299] On the other hand, if the autonomous driving controller 154 determines in step S223 that the risk level calculated in step S222 has increased to a predetermined value, then the processing in step S224 is performed.

[0300] In step S224, the autopilot controller 154 determines whether an emergency stop is required. If a driver awakening determination is made, the autopilot controller 154 immediately proceeds to sequence S232 to stop the vehicle if it is too late. If an immediate stop is not required, the driver can make the decision during manual driving.

[0301] If the autopilot controller 154 does not determine that an emergency stop is required in step S224, then the process proceeds to step S225. In step S225, the driver state determination unit 153 performs processing to confirm the driver's alertness (e.g., like...). Figure 18 The manual driving capability determination process is used for tracking the line of sight to virtual targets.

[0302] In step S226, based on the driver's arousal state confirmed in step S225, the driver state determination unit 153 determines whether the driver has normal driving ability. If the driver state determination unit 153 determines in step S226 that the driver does not have normal driving ability, the process in step S227 is performed.

[0303] In step S227, the autonomous driving controller 154 performs the process of waking up the driver (e.g., Figure 19 (Driver wake-up processing). In this case, the autopilot controller 154 is configured to take measures to prevent abnormal operations, in order to prevent accidents caused by actions performed hastily by the driver when the driver does not fully understand the situation (e.g., making a sharp turn or braking immediately after the driver wakes up).

[0304] In step S228, as in step S225, the driver state determination unit 153 performs processing to confirm the driver's awake state.

[0305] In step S229, as a result of confirming the driver's awake state in step S228, the driver state determination unit 153 determines whether the driver's driving ability has been restored. If the driver state determination unit 153 determines in step S229 that the driver's driving ability has been restored, then the process in step S230 is performed. If it is determined in step S226 that the driver has normal driving ability, then the process in step S230 is performed.

[0306] In step S230, the automatic driving controller 154 terminates automatic driving and performs normal manual priority driving.

[0307] In step S231, the automatic driving controller 154 confirms that the driver's alertness is ensured. Thereafter, the automatic driving control unit 154 cancels the measures set in step S227 for abnormal operation and terminates the emergency evacuation and transfer process.

[0308] On the other hand, if it is determined in step S224 that an emergency stop is required, or if it is determined in step S229 that the driver's driving ability has not been restored, then the process in step S232 is performed.

[0309] In step S232, as a process to begin the transition to the evacuation lane, for example, the automatic driving controller 154 flashes the direction indicator lights along the direction of the evacuation lane and flashes the hazard lights to notify the vehicles behind that an automatic evacuation process is being performed.

[0310] In step S233, the autopilot controller 154 processes the movement to the evacuation lane on the shoulder. If the vehicle cannot reach the evacuation lane, the autopilot controller 154 decelerates and then brings the vehicle to a stop. After the processing in step S233, the emergency evacuation process is terminated.

[0311] In the following text, we will refer to... Figure 21 A and 21B explain that the driver status determination unit 153 monitors the driver's condition. Figure 20 The active monitoring of the driver's response characteristics during the process of confirming the driver's alertness in steps S225 and S228.

[0312] Figure 21 A illustrates the operational delay of a driver under normal conditions. Figure 21 B illustrates the operational delay caused by a decrease in the driver's level of alertness.

[0313] For example, when a vehicle is driving on road sections where driving operations such as turning the steering gear are frequently required, driver alertness can be determined passively by checking the stability of steering corrections when the driver correctly turns the steering gear along the curve of the road. Conversely, when the driver does not need intervention due to continuous cruising on a straight and flat road, certain inputs corresponding to response requests from the vehicle system are required to determine the driver's alertness.

[0314] Therefore, the autonomous driving control system 101 provides the driver with a wake-up vibration and a warning sound, and measures the driver's response characteristics to prevent loss and delay in the driver's response to active input from the system when the level of awareness is low. Figure 21 As shown in B, this is consistent with... Figure 21 This contrasts with the delay that occurs in the waking state shown in Figure A.

[0315] As described above, the autonomous driving control system 101 monitors and evaluates the driver's response characteristics using an active method, and measures the response delay under these conditions. Based on the measured values, the vehicle's braking can be enhanced. More specifically, in the autonomous driving control system 101, when acting as a response to... Figure 21 If the measurement and monitoring results of the driver's response characteristics shown in A and 21B show a delay in response characteristics, then auxiliary intervention in vehicle braking is required.

[0316] Figure 22 This is a flowchart illustrating the brake assist intervention process that provides auxiliary intervention for vehicle braking.

[0317] In step S241, the driver state determination unit 153 monitors the driver state based on the monitoring results provided by the driver monitoring device 127 and the analysis results provided by the driving behavior analysis unit 152. The automatic driving controller 154 monitors the surrounding environment based on the input from various sensors.

[0318] In step S242, the driver state determination unit 153 measures the driver's response characteristics. More specifically, the driver state determination unit 153 invokes a reference... Figure 21 A and 21B describe the input of wake-up vibration and warning sound, and the driving behavior analysis unit 152 measures and monitors the response characteristics, operating stability, etc. detected by the acceleration operation detection unit 124, braking operation detection unit 125 and steering operation detection unit 126.

[0319] In step S243, the driver state determination unit 153 determines the driver's state as follows: Figure 21 Compared to the necessary time in the normal preferred state of wakefulness shown in A, is there a delay in response characteristics?

[0320] In step S243, if the driver state determination unit 153 determines that the driver is in a preferred awake state when there is no delay in the response characteristics, the process of step S241 is executed again, and then the same process is repeated.

[0321] On the other hand, if the driver state determination unit 153 determines in step S243 that a delay has occurred in the response characteristics, then the processing in step S244 is subsequently executed. In this case, the driver is not in a preferred awake state, so in step S244, the driver state determination unit 153 performs auxiliary braking intervention to compensate for the driver's reaction delay, and for example, the driver state determination unit 153 performs adjustments to the braking control parameters of the brake activation device 129 required to reduce braking time in response to the braking operation.

[0322] As described above, the automatic driving control system 101 can perform braking assistance intervention if there is a delay compared to the statistical values ​​obtained from learning the driver-specific characteristics in a normal preferred state of wakefulness, based on the transition from the driver's wakefulness state.

[0323] Next, we will refer to Figure 23 This describes the processing used to predict the timing of a switch to manual driving.

[0324] like Figure 23 As shown, there is a dedicated lane for autonomous driving from the departure location to the destination location, and the method used to predict the vehicle's arrival at the end location will be explained (e.g., in...). Figure 3 The probability of arrival at positions R7 and R13 (before) is predicted.

[0325] For example, in order to allow a vehicle to smoothly transition from an autonomous driving lane to a manual driving section, the driver needs to be awakened until the driver can fully and accurately assess the surrounding situation. To do this, the vehicle's position for a certain period of time before reaching the end of the section needs to be predicted, and the driver's awakening status needs to be determined until the vehicle reaches that position, and the driver needs to be awakened as needed.

[0326] For example, if there is a potential delay in recognizing environmental conditions due to wakefulness and the execution of other tasks, the autonomous driving controller 154 needs to take measures such as automatic evacuation and deceleration. More specifically, switching to a manual driving lane when the driver of the autonomous vehicle is not ready to resume driving at the predicted time of the end of the autonomous driving lane segment could lead to a very dangerous situation because the manual driving lane ahead may not necessarily have a suitable environment for autonomous driving.

[0327] Therefore, it is necessary to accurately and in advance determine the end position of the dedicated autonomous vehicle lane segment or the planned time for the vehicle to transition to the manual lane, and to have a preparation time for the driver to switch to manual driving. More specifically, it is necessary to accurately and in advance determine the time for the vehicle to transition from the end position of the dedicated autonomous vehicle lane segment to the manual driving segment at position R7 ( Figure 3 Before a certain time period Δt, or when the vehicle enters the position R13 where it transitions from the autonomous driving lane to the manual lane. Figure 3 The manual driving conversion preparation sequence is initiated during the time period Δt prior to this.

[0328] During the period before the vehicle reaches the end of the autonomous driving lane segment, the transition preparation sequence point can be designated using road markings and signs, and further fixed transit points can be specified by using DSRC (Digital Switched Reconnaissance Carrier) and other road infrastructure. Under this fixed, constant coordinate system, the vehicle can travel at speeds up to the maximum permissible driving speed Vmax (V... 最大 The vehicle passes through that section of the road and can later reach position R7 at time Δt, and therefore, this is the position reached before the change of position by a distance of travel (Vmax×Δt).

[0329] For example, when the time period is defined as five minutes, and the speed of 80 km / h is the maximum permissible speed for autonomous driving, the position 6.7 km before the actual end of the dedicated lane is the manual driving transition preparation position. However, depending on the level of road congestion, the actual driving speed is, for example, 20 km / h, which is one-quarter of the maximum speed, and it takes four times as long. Even if a notification is received 6.7 km before the actual end, in reality, the manual driving transition preparation would begin 20 minutes earlier because it takes four times as long.

[0330] Furthermore, due to traffic congestion and other factors, driving speeds on the road change dynamically, so even if a previously set wake-up destination location is established, it may vary significantly from the actual time. In such cases, if the actual time required for the driver's recovery is delayed too much, the driver may become relaxed due to the situation, and their alertness level may decrease, resulting in the opposite effect. Therefore, it is preferable to perform the manual driving preparation sequence at a slightly more accurate prior time. On routes where traffic congestion is common, this can be unpleasant for the driver; therefore, it is preferable to perform the manual driving preparation transition sequence a certain time Δt in advance, regardless of the overall traffic flow on the road.

[0331] Furthermore, in autonomous driving lanes, there may be exits at unspecified locations, and many such locations (R13) can be frequently set up within a single, continuous dedicated autonomous driving lane for entry into manual lanes. Figure 3Therefore, arranging exit signs for all of these, and applying manual driving preparation transition sequences not only from the perspective of infrastructure maintenance but also from the perspective of installation and long-term maintenance, may not be considered a very desirable configuration. Therefore, even if external information is obtained through some other means, the vehicle preferably includes a manually determined manual driving preparation transition sequence. By doing so, it is unnecessary to have signs or symbols at every exit on the road, and it is sufficient to set a single driving preparation transition sequence start sign at a single location at the final end of a single, continuous autonomous driving dedicated vehicle road segment, which greatly reduces the burden as infrastructure maintenance, thus reducing obstacles to this introduction. Each vehicle then preferably autonomously predicts its destination location based on its purpose, within the driving environment and while driving the vehicle, and autonomously initiates the manual driving preparation transition sequence from autonomous driving at the appropriate point, without relying on infrastructure installation signs.

[0332] For example, specific locations of road environments used daily, such as commuting routes, should be recorded and saved. It is essential to pre-define a prior preparation point for manual driving so that even if the driver becomes temporarily drowsy and environmental recognition is interrupted, a fixed location pre-specified and set by the driver can be used to initiate a wake-up in the same familiar environment, and successful situation recognition is expected again. In cases where the driver has pre-defined settings, the driver may try to set the installation location as close as possible to the manual driving transition point, as the driver's desire for an easy process is potentially dangerous. Therefore, for safe operation, the starting location of the manual driving preparation transition sequence is preferably limited to, for example, at least one kilometer prior to the manual driving transition point.

[0333] In recent years, a service has been provided to collect road traffic data intermittently from navigation systems such as Global Navigation Satellite Systems (GNSS) and numerous driving vehicles, centrally acquire road information, and provide this road information from a server to designated vehicles. Therefore, it is possible to obtain a calculable portion based on information held by the vehicles themselves and the extensive information acquired through centralized management of central road information.

[0334] In one use case where the position for preparing to switch to manual driving sequence is predetermined, the determination of whether the vehicle is approaching the position for preparing to switch to manual driving sequence is made by referring to driving history and reference map information, and the sequence can be initiated at the determined driving point.

[0335] On the other hand, the predicted arrival time at the destination at the start of driving varies due to various factors (such as road congestion depending on the time zone and a particular day of the week, various weather conditions along the route, accident occurrences and driving sentiment factors, road maintenance conditions, etc.), thus broadening the distribution of the early arrival prediction probability. As the arrival point approaches, the variation factors also decrease, allowing for more accurate predictions. What the system needs to operate appropriately is to initiate a manual driving transition preparation sequence a certain time period Δt beforehand. In this case, if there are no driving congestion factors and no information needed for predicting variation factors, the vehicle can drive at its maximum permissible speed. Therefore, after traveling through a segment of ΔS = Vmax × Δt, the vehicle reaches the end position of the dedicated autonomous driving lane at the fastest speed.

[0336] More specifically, if the latest arrival prediction probability distribution can be obtained at a location with ΔS, the more appropriate arrival time of the end position of the dedicated autonomous driving lane changes, and the updated arrival prediction time obtained again is defined as t. rae In the case of (updating the arrival estimate), it is possible to fully ensure the time that the driver can use the manual driving transition procedure, such that if the manual driving transition preparation sequence is initiated 2σ before the arrival of the predicted time, even if the changing factors remain unchanged, no major problems will occur.

[0337] When an increase in predicted driving speed is detected, for example, after traffic congestion has been resolved following a ΔS interval, the arrival time becomes earlier. Therefore, the manual driving transition preparation sequence is initiated earlier to compensate for the increase in speed. By initiating the sequence earlier, measures to avoid delays in the transition preparation sequence can be implemented even if the vehicle arrives slightly earlier than the predicted arrival time.

[0338] The aforementioned series of processes (information processing methods) can be executed by hardware or by software. When the series of processes are executed by software, the program constituting the software is installed from a program recording medium on which the program is recorded to a computer incorporating dedicated hardware, and for example, a general-purpose personal computer capable of performing various functions by installing various programs.

[0339] Figure 24 This illustrates the use of a program to perform the aforementioned series of processes and / or to be configured to achieve the above-mentioned relative to Figure 1 and Figure 9 An example of the configuration of the hardware of a computer in one or a combination of the aforementioned units.

[0340] In this computer, the CPU (Central Processing Unit) 202, ROM (Read-Only Memory) 203, RAM (Random Access Memory) 204, and EEPROM (Electrically Erasable Programmable Read-Only Memory) 205 are interconnected via bus 206. Bus 206 is further connected to input / output interface 207 and connected to the outside via input / output interface 207.

[0341] In the computer configured as described above, the CPU 202 performs the aforementioned series of processes by executing a program that is loaded into RAM 203 via bus 206 and stored in ROM 103 and EEPROM 205.

[0342] For example, a program executed by the computer (CPU202) can be provided from an external device connected via the input / output interface 207 using a packaged medium consisting of a disk (including floppy disk), optical disk (CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Universal Optical Disc) etc.), magneto-optical disk, or semiconductor memory, or via a wired or wireless transmission medium (e.g., a local area network, the Internet, and digital satellite broadcasting).

[0343] It should be noted that this technology can be configured as follows.

[0344] (1) A method for switching modes for operating a vehicle, the method comprising:

[0345] The circuitry of the information processing unit determines whether the mode for operating the vehicle should be switched from one of autonomous and manual driving modes to another of autonomous and manual driving modes.

[0346] When it is determined that the mode used to operate the vehicle has been switched, the driver's status is obtained; and

[0347] Based on the acquired driver status, the system switches the vehicle operation mode from one of autonomous driving mode and manual driving mode to the other via circuitry.

[0348] (2) The method according to feature (1) further includes:

[0349] Determine the driver's ability to drive the vehicle based on the acquired driver status; and

[0350] When the driver's ability is determined to be greater than or equal to a predetermined level, the mode used to operate the vehicle will be switched from autonomous driving mode to manual driving mode.

[0351] (3) The method according to feature (2) further includes:

[0352] When the driver's ability is determined to be below a predetermined level, a warning is provided to the driver to change the driver's state; and

[0353] After providing a warning to the driver, determine the driver's ability to drive the vehicle; and

[0354] When it is determined that the driver's ability after the warning is provided is greater than or equal to the predetermined level, the vehicle operation mode is switched from autonomous driving mode to manual driving mode.

[0355] (4) The method according to feature (3) further includes:

[0356] The vehicle is stopped when the driver's ability to drive is determined to be below a predetermined level.

[0357] (5) The method according to any one of features (1) to (4) further includes:

[0358] When the vehicle's operating mode is determined to switch from autonomous driving mode to manual driving mode, the driver will be notified that autonomous driving mode will be terminated.

[0359] (6) The method according to any one of features (1) to (5), wherein the determining step comprises:

[0360] Based on the distance to the end of the dedicated autonomous driving lane, the mode for operating the vehicle will be switched from autonomous driving mode to manual driving mode.

[0361] (7) The method according to any one of features (1) to (6) further includes:

[0362] The autonomous driving mode is activated based on the acquired driver status.

[0363] The steps for switching modes include switching the mode used to operate the vehicle from manual driving mode to autonomous driving mode when autonomous driving mode can be activated.

[0364] (8) The method according to feature (7) further includes:

[0365] Determine the driver's ability to drive the vehicle based on the acquired driver status; and

[0366] When the driver's ability is determined to be below a predetermined level, the mode used to operate the vehicle is prohibited from switching from manual driving mode to autonomous driving mode.

[0367] (9) The method according to any one of features (1) to (8) further includes:

[0368] When the vehicle is in autonomous driving mode, it monitors the driver's status and the surrounding environment.

[0369] (10) The method according to any one of features (1) to (9) further includes:

[0370] To display multiple pieces of content to the driver; and

[0371] Monitor the driver's reactions to multiple displays, including

[0372] The acquisition process includes obtaining the driver's status based on the monitored driver's reactions.

[0373] (11) A non-transitory computer-readable medium storing instructions, which, when executed by a computer, cause the computer to perform a method for switching a mode for operating a vehicle, the method comprising:

[0374] Determine whether the mode used to operate the vehicle needs to be switched from one of the autonomous and manual driving modes to another of the autonomous and manual driving modes;

[0375] When it is determined that the mode used to operate the vehicle needs to be switched, the driver's status is obtained; and

[0376] Based on the acquired driver status, the vehicle operation mode is switched from one of the autonomous driving mode and the manual driving mode to the other of the autonomous driving mode and the manual driving mode.

[0377] (12) The non-transitory computer-readable medium according to feature (11) further includes:

[0378] Determine the driver's ability to drive the vehicle based on the acquired driver status; and

[0379] When the driver's ability is determined to be greater than or equal to a predetermined level, the mode used to operate the vehicle will be switched from autonomous driving mode to manual driving mode.

[0380] (13) The non-transitory computer-readable medium according to feature (12) further includes:

[0381] When the driver's ability is determined to be below a predetermined level, a warning is provided to the driver to change the driver's state; and

[0382] After providing a warning to the driver, determine the driver's ability to drive the vehicle; and

[0383] When it is determined that the driver's ability is greater than or equal to a predetermined level after the warning has been issued, the mode used to operate the vehicle will be switched from autonomous driving mode to manual driving mode.

[0384] (14) The non-transitory computer-readable medium according to feature (13) further includes:

[0385] The vehicle is stopped when the driver's ability to drive is determined to be below a predetermined level.

[0386] (15) The non-transitory computer-readable medium according to any one of features (11) to (14) further comprises:

[0387] When the mode used to operate the vehicle is determined to switch from autonomous driving mode to manual driving mode, the driver will be notified that the autonomous driving mode will be terminated.

[0388] (16) A non-transitory computer-readable medium according to any one of features (11) to (15), wherein the determining step comprises:

[0389] Based on the distance to the end of the dedicated autonomous driving lane, the mode for operating the vehicle will be switched from autonomous driving mode to manual driving mode.

[0390] (17) The non-transitory computer-readable medium according to any one of features (11) to (16) further comprises:

[0391] The determination of whether autonomous driving mode can be activated is based on the acquired driver status.

[0392] The steps for switching modes include switching the mode used to operate the vehicle from manual driving mode to autonomous driving mode when autonomous driving mode can be activated.

[0393] (18) The non-transitory computer-readable medium according to feature (17) further includes:

[0394] Determine the driver's ability to drive the vehicle based on the acquired driver status; and

[0395] When the driver's ability is determined to be below a predetermined level, the mode used to operate the vehicle is prohibited from switching from manual driving mode to autonomous driving mode.

[0396] (19) The non-transitory computer-readable medium according to any one of features (11) to (18) further comprises:

[0397] When the vehicle is in autonomous driving mode, the driver's status and the surrounding environment are monitored.

[0398] (20) The non-transitory computer-readable medium according to any one of features (11) to (19) further includes:

[0399] To display multiple pieces of content to the driver; and

[0400] Monitor the driver's reactions to multiple displays, including

[0401] The acquisition process includes obtaining the driver's status based on the monitored driver's reactions.

[0402] (21) An information processing apparatus, comprising:

[0403] The circuit is configured as follows

[0404] Determine whether the mode used to operate the vehicle needs to be switched from one of the autonomous and manual driving modes to another of the autonomous and manual driving modes;

[0405] When it is determined that the mode used to operate the vehicle needs to be switched, the driver's status is obtained; and

[0406] Based on the acquired driver status, the mode used to operate the vehicle is switched from one of autonomous driving mode and manual driving mode to another of the autonomous driving mode and manual driving mode.

[0407] (22) The information processing apparatus according to feature (21), wherein the circuitry is configured to determine the driver's ability to drive the vehicle based on the acquired driver's state; and

[0408] When the driver's ability is determined to be greater than or equal to a predetermined level, the mode used to operate the vehicle will be switched from autonomous driving mode to manual driving mode.

[0409] (23) The information processing apparatus according to feature (22), wherein the circuitry is configured as follows:

[0410] When the driver's ability is determined to be below the predetermined level, a warning is given to the driver to change the driver's state;

[0411] After providing a warning to the driver, determine the driver's ability to drive the vehicle; and

[0412] When it is determined that the driver's ability is greater than or equal to a predetermined level after a warning is issued, the mode used to operate the vehicle will be switched from autonomous driving mode to manual driving mode.

[0413] (24) The information processing apparatus according to feature (23), wherein the circuitry is configured to stop the vehicle when the driver’s ability to drive the vehicle is determined to be less than a predetermined level.

[0414] (25) An information processing apparatus according to any one of features (21) to (24), wherein the circuit is configured to

[0415] When it is determined that the mode used to operate the vehicle needs to be switched from autonomous driving mode to manual driving mode, the driver will be notified that the autonomous driving mode will be terminated.

[0416] (26) The information processing apparatus according to any one of features (21) to (25), wherein the circuit is configured to

[0417] Based on the distance to the end of the dedicated autonomous driving lane, the mode for operating the vehicle will be switched from autonomous driving mode to manual driving mode.

[0418] (27) The information processing apparatus according to any one of features (21) to (26), wherein the circuit is configured to

[0419] Based on the obtained driver status, determine whether autonomous driving mode can be activated.

[0420] When autonomous driving mode can be activated, the mode used to operate the vehicle will be switched from manual driving mode to autonomous driving mode.

[0421] (28) The information processing apparatus according to feature (27), wherein the circuitry is configured to determine the driver's ability to drive the vehicle based on the acquired driver's state; and

[0422] When the driver's ability is determined to be below a predetermined level, the mode used to operate the vehicle is prohibited from switching from manual driving mode to autonomous driving mode.

[0423] (29) An information processing apparatus according to any one of features (21) to (28), wherein the circuit is configured to

[0424] When the vehicle is in autonomous driving mode, the driver's status and the surrounding environment are monitored.

[0425] (30) An information processing apparatus according to any one of features (21) to (29), wherein the circuit is configured to

[0426] This allows multiple pieces of content to be displayed to the driver.

[0427] Monitor the driver's reaction to multiple displays; and

[0428] The driver's status is obtained based on the monitored driver's reactions.

[0429] (31) An information processing apparatus, comprising:

[0430] A device for determining whether the mode for operating a vehicle should be switched from one of autonomous and manual driving modes to another of autonomous and manual driving modes.

[0431] A means for acquiring the state of the vehicle driver when it is determined to switch to a mode for operating the vehicle; and

[0432] A device for switching the mode of operating the vehicle from one of autonomous and manual driving modes to another of autonomous and manual driving modes based on the acquired driver state.

[0433] (32) The information processing apparatus according to feature (31) further includes:

[0434] A device for determining a driver's ability to drive a vehicle based on the acquired driver state, wherein

[0435] When the driver's ability is determined to be greater than or equal to a predetermined level, the switching device will switch the mode used to operate the vehicle from autonomous driving mode to manual driving mode.

[0436] (33) The information processing apparatus according to feature (32) further includes:

[0437] A device for providing a warning to the driver to change the driver's state when the driver's ability is determined to be below a predetermined level, wherein

[0438] The device for determining a driver's ability determines the driver's ability to drive the vehicle after a warning has been given to the driver, and

[0439] When it is determined that the driver's ability after the warning is provided is greater than or equal to a predetermined level, the switching device will switch the mode used to operate the vehicle from autonomous driving mode to manual driving mode.

[0440] (34) The information processing apparatus according to feature (33) further includes:

[0441] A device for stopping a vehicle when the driver’s ability to drive the vehicle is determined to be less than a predetermined level.

[0442] (35) The information processing apparatus according to any one of features (31) to (34) further includes:

[0443] A device for notifying the driver that autonomous driving mode will be terminated when it is determined that the mode for operating the vehicle is switching from autonomous driving mode to manual driving mode.

[0444] (36) The information processing apparatus according to any one of features (31) to (35), wherein

[0445] The device used to determine whether to switch the mode for operating the vehicle determines, based on the distance to the end of the dedicated autonomous driving lane, whether to switch from autonomous driving mode to manual driving mode.

[0446] (37) The information processing apparatus according to any one of features (31) to (36) further includes:

[0447] A device for determining whether an autonomous driving mode can be activated based on the acquired driver status, wherein

[0448] When autonomous driving mode can be activated, the switching device will switch the mode used to operate the vehicle from manual driving mode to autonomous driving mode.

[0449] (38) The information processing apparatus according to feature (37) further includes:

[0450] A device for determining a driver's ability to drive a vehicle based on the acquired driver state; and

[0451] A device for preventing the switching of the vehicle operation mode from manual driving mode to autonomous driving mode when the driver's ability is determined to be less than a predetermined level.

[0452] (39) The information processing apparatus according to any one of features (31) to (38) further includes:

[0453] A device for monitoring the driver's status and the surrounding environment when the mode for operating the vehicle is in autonomous driving mode.

[0454] (40) The information processing apparatus according to any one of features (31) to (39) further includes:

[0455] Device for displaying multiple contents to the driver; and

[0456] A device for monitoring the driver's reaction to multiple content displays, wherein

[0457] The device used to acquire the driver's status is based on the driver's reactions being monitored.

[0458] It should be noted that this technology can be configured as follows.

[0459] (1) An automatic driving control device, comprising:

[0460] The driving execution unit is configured to perform autonomous driving cruise control of the vehicle; and

[0461] The recovery determination unit is configured to determine whether the driver has the driving ability to resume manual driving from automatic driving.

[0462] If, before the driving execution unit initiates the vehicle's autonomous driving cruise, the recovery determination unit determines that the driver lacks driving ability, it will then prohibit the autonomous driving cruise.

[0463] (2) The automatic driving control device according to (1), wherein even when the driver’s driving ability declines, the recovery determination unit allows the driving execution unit to perform automatic driving cruise in emergency evacuation response mode.

[0464] (3) The autonomous driving control device according to (2), wherein the recovery determination unit includes a recording unit configured to record and save in an indelible manner the autonomous driving cruise is permitted in the emergency evacuation response mode when the driving execution unit permits autonomous driving cruise in the emergency evacuation response mode.

[0465] (4) The automatic driving control device according to any one of (1) to (3), wherein the recovery determination unit comprises:

[0466] The monitoring unit is configured to monitor the driver's driving ability using one or more schemes, including driver facial recognition, head posture recognition, gaze direction recognition, and statistical determination processing of the time-varying stability of head posture or gaze direction; and

[0467] The determination processing unit is configured to determine manual driving capability after a specific time period based on the monitoring results of the monitoring unit.

[0468] (5) An autonomous driving control method for an autonomous driving control device including a driving execution unit configured to perform autonomous driving cruise of a vehicle and a determination unit configured to determine whether a driver has the ability to resume driving from autonomous driving to manual driving, wherein the autonomous driving control method includes prohibiting the autonomous driving cruise if the resumption determination unit determines that the driver does not have the ability to drive before the driving execution unit initiates the autonomous driving cruise of the vehicle.

[0469] (6) A program that causes a computer to perform the following process, the computer being included in an autonomous driving control device, the autonomous driving control device including a driving execution unit configured to perform autonomous cruise control of a vehicle and a recovery determination unit configured to determine whether a driver has the ability to resume driving from autonomous driving to manual driving:

[0470] Before the driving execution unit initiates the vehicle's autonomous cruise control, if the recovery determination unit determines that the driver is incapable of driving, it will disable the autonomous cruise control.

[0471] This embodiment is not limited to the above embodiment, and can be modified in various ways without departing from the essence of this disclosure.

[0472] List of reference numerals

[0473] 11 Automatic driving control equipment

[0474] 12 Restore Determination Unit

[0475] 13 Driving Execution Units

[0476] 21 monitoring units

[0477] 22 output units

[0478] 23 Road Information Acquisition Unit

[0479] 24 input units

[0480] 25 recording units

[0481] 26. Determine the processing unit

[0482] 31 Environmental Recognition Unit

[0483] 32 driving processing units

[0484] 33 Steering wheel control unit

[0485] 34 speed control unit

[0486] 35 Hazard Flashing Light Control Unit

[0487] 41 cameras

Claims

1. A method for determining a mode of operating a vehicle, the method comprising: Monitor safety risks during continuous autonomous driving; Based on the monitoring results, calculate the degree of risk as a measure of the extent to which safety risks may occur due to continued autonomous driving; If it is determined that the calculated level of risk has increased to the predetermined value, determine whether an emergency stop is necessary; In cases where it is not determined that an emergency stop is necessary, procedures are performed to confirm the driver's alertness to determine whether the driver has the normal driving ability. If it is determined that the driver does not have normal driving ability, the driver is awakened and measures are set up to prevent abnormal operation, so as to prevent accidents caused by the driver's hasty operation when the driver does not fully understand the situation. As a result of confirming the driver's state of consciousness, it determines whether the driver's driving ability has been restored; Once it is determined that the driver's driving ability has been restored, terminate the automated driving function. If an emergency stop is determined to be necessary, or if the driver's ability to drive has not been restored, the process of switching to the evacuation lane should begin. The method also includes initiating an emergency stop sequence for the vehicle when operated using an emergency automatic stop unit that can be operated by a non-driver.

2. An information processing apparatus, comprising: The circuit is configured as follows: Monitor safety risks during continuous autonomous driving; Based on the monitoring results, calculate the degree of risk as a measure of the extent to which safety risks may occur due to continued autonomous driving; If it is determined that the calculated level of risk has increased to the predetermined value, determine whether an emergency stop is necessary; In cases where it is not determined that an emergency stop is necessary, procedures are performed to confirm the driver's alertness to determine whether the driver has the normal driving ability. If it is determined that the driver does not have normal driving ability, the driver is awakened and measures are set up to prevent abnormal operation, so as to prevent accidents caused by the driver's hasty operation when the driver does not fully understand the situation. As a result of confirming the driver's state of consciousness, it determines whether the driver's driving ability has been restored; Once it is determined that the driver's driving ability has been restored, terminate the automated driving function. If an emergency stop is determined to be necessary, or if the driver's ability to drive has not been restored, the process of switching to the evacuation lane should begin. The circuit is also configured to initiate an emergency stop sequence for the vehicle when operated using an emergency automatic stop unit that can be operated by a non-driver.

3. The information processing apparatus according to claim 2, further comprising: The circuit is configured as The driver's ability to drive the vehicle is determined based on the acquired driver status. as well as When the driver's ability is determined to be sufficient for manual driving, the vehicle operation mode is switched from automatic driving mode to manual driving mode.

4. The information processing apparatus according to claim 3, further comprising: The circuit is configured as When a driver's ability is determined to be problematic, the driver is notified to change the driver's status.

5. The information processing apparatus according to claim 4, further comprising that the notification is a warning; The circuit is configured as follows: After providing a warning to the driver, determine the driver's ability to drive the vehicle; and When the driver's ability is determined to be sufficient for manual driving after a warning has been given, the vehicle operation mode is switched from automatic driving mode to manual driving mode.

6. The information processing apparatus according to claim 4, further comprising that the notification is a notice; The circuit is configured as follows: Provide drivers with notifications indicating the end of the automated driving segment; Attract the driver's attention to signal the end of the autonomous driving section; as well as The time required for the driver to respond to the attention.

7. The information processing apparatus according to claim 5, further comprising: The circuit is configured as follows: The vehicle is brought to a stop when the driver's ability is determined to be below a predetermined level after a warning.

8. The information processing apparatus according to claim 2, further comprising: The circuit is configured as follows: When it is determined that the vehicle operation mode will be switched from automatic driving mode to manual driving mode, the driver will be notified that the automatic driving mode will be terminated.

9. The information processing apparatus according to claim 2, further comprising: Storage device, configured as The information based on the emergency stop sequence will be stored as an indelible record.

10. A non-transitory computer-readable medium storing instructions, said instructions, when executed by a computer, causing the computer to perform a method for determining a mode of operating a vehicle, said method comprising: Monitor safety risks during continuous autonomous driving; Based on the monitoring results, calculate the degree of risk as a measure of the extent to which safety risks may occur due to continued autonomous driving; If it is determined that the calculated level of risk has increased to the predetermined value, determine whether an emergency stop is necessary; In cases where it is not determined that an emergency stop is necessary, procedures are performed to confirm the driver's alertness to determine whether the driver has the normal driving ability. If it is determined that the driver does not have normal driving ability, the driver is awakened and measures are set up to prevent abnormal operation, so as to prevent accidents caused by the driver's hasty operation when the driver does not fully understand the situation. As a result of confirming the driver's state of consciousness, it determines whether the driver's driving ability has been restored; Once it is determined that the driver's driving ability has been restored, terminate the automated driving function. If an emergency stop is determined to be necessary, or if the driver's ability to drive has not been restored, the process of switching to the evacuation lane should begin. The method also includes initiating an emergency stop sequence for the vehicle when operated using an emergency automatic stop unit that can be operated by a non-driver.

11. An information processing apparatus, comprising: Devices used to monitor safety risks when continuously performing autonomous driving; A device for calculating the degree of risk as a result of continued autonomous driving, based on monitoring results; A device used to determine whether an emergency shutdown is necessary when the calculated level of risk has increased to a predetermined value; A device for determining the driver's alertness to ascertain whether the driver has normal driving ability in situations where an emergency stop is not required. A device for waking up the driver when it is determined that the driver does not have normal driving ability, and for setting up measures to prevent abnormal operation, so as to prevent accidents caused by the driver's hasty operation when the driver does not fully understand the situation. A device used to determine whether a driver's ability to drive has been restored, as a result of confirming the driver's state of consciousness. A device for terminating automatic driving when it is determined that the driver's ability to drive has been restored; A device for initiating a switch to an evacuation lane when it is determined that an emergency stop is required or when the driver's ability to drive has not been restored. as well as A device for initiating an emergency braking sequence of a vehicle when operated using an emergency automatic stop unit that can be operated by a non-driver.

Images