SEAT CONTROL DEVICE, SEAT CONTROL PROGRAM, STATE ESTIMATION DEVICE, AND STATE ESTIMATION PROGRAM

The seat control device and state estimation program adjust the driver's seat and weighting of information to keep the driver within the detection range, addressing the issue of disrupted monitoring during autonomous driving by maintaining effective condition monitoring.

JP7582075B2Active Publication Date: 2024-11-13DENSO CORP
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2021093879
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-06-03
Publication Date
2024-11-13
Estimated Expiration
2041-06-03

AI Technical Summary

Technical Problem

When drivers are not obligated to monitor their surroundings during autonomous driving, there is a risk of them changing their posture significantly, leading to difficulties in maintaining effective monitoring, which can disrupt the continuous monitoring of their condition.

Method used

A seat control device and state estimation program that adjust the reclining state of the driver's seat and change the weighting of head and facial information to ensure the driver remains within the detection range of the driver monitor, even when not monitoring the surroundings.

Benefits of technology

This approach prevents interruptions in driver monitoring during autonomous driving by ensuring the driver remains detectable, thereby maintaining effective condition monitoring and allowing for smoother transitions between driving modes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007582075000001
    Figure 0007582075000001
  • Figure 0007582075000002
    Figure 0007582075000002
  • Figure 0007582075000003
    Figure 0007582075000003
Patent Text Reader

Abstract

To provide a seat control device or the like capable of suppressing interruption of state monitoring of a driver in an automatic driving period with no obligation of surroundings monitoring.SOLUTION: An HCU 100 is used for an own vehicle Am mounted with a driver monitor 29 for monitoring a state of a driver. The HCU 100 functions as a seat control device for controlling a state of a driver's seat on which the driver sits. The HCU 100 grasps whether or not the own vehicle Am is traveling by autonomous travel control with no obligation of surroundings monitoring by the driver. When the own vehicle Am is traveling in autonomous traveling control, a reclining state of the driver's seat is controlled so that the driver is included in a detection range of the driver monitor 29.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The disclosure in this specification relates to a seat control device, a seat control program, a state estimation device, and a state estimation program. [Background technology]

[0002] Patent Document 1 describes a control device that performs all driving operations to enable autonomous driving without the driver having to monitor the surroundings. This control device judges whether the driver's condition is abnormal or not based on images and detection signals captured by an occupant monitoring device that monitors the driver's condition. If the control device judges that an abnormality has occurred that would make it difficult to continue autonomous driving, it performs vehicle control to stop the vehicle at an evacuation site such as a roadside or a parking area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-158090 A Summary of the Invention [Problem to be solved by the invention]

[0004] When a driver has a duty to monitor the surroundings, the position of the driver's head and the direction of the driver's face can generally be within a predetermined range. However, when a driver does not have a duty to monitor the surroundings, the driver may lose his / her driving posture or operate a mobile device. In this way, when the driver does not have a duty to monitor the surroundings, the driver may move more than when the driver has a duty to monitor the surroundings. As a result, there is a risk that it may be difficult to continue monitoring the driver's condition.

[0005] The present disclosure aims to provide a seat control device, a seat control program, a state estimation device, and a state estimation program that can suppress interruption of driver state monitoring during an autonomous driving period when there is no obligation to monitor the surroundings. [Means for solving the problem]

[0006] In order to achieve the above object, one aspect of the present invention is , D The seat control device is used in a host vehicle (Am) equipped with a driver monitor (29) that monitors the state of the driver, and controls the state of a driver's seat (DS) in which the driver sits. The seat control device includes a control grasping unit (82) that grasps whether the host vehicle is traveling by autonomous driving control in which the driver is not required to monitor the surroundings, and a reclining control unit (89) that controls the reclining state of the driver's seat so that the driver is included in the detection range of the driver monitor when the host vehicle is traveling by autonomous driving control. When a schedule for transition from autonomous driving control to driving assistance control in which the driver is required to monitor the surroundings is grasped, the reclining control unit is a seat control device that sets stricter judgment criteria for deciding to implement reclining state control than when the schedule for transition is not grasped.

[0007] Also, one disclosed embodiment is , D The seat control program is used in a host vehicle (Am) equipped with a driver monitor (29) that monitors the state of the driver, and controls the state of the driver's seat in which the driver is seated, and causes at least one processing unit (11) to execute processing including: determining whether or not the host vehicle is traveling by autonomous driving control in which the driver is not required to monitor the surroundings (S101); if the host vehicle is traveling by autonomous driving control, controlling the reclining state of the driver's seat so that the driver is included in the detection range of the driver monitor (S105, S109); and, if a schedule for transition from autonomous driving control to driving assistance control in which the driver is required to monitor the surroundings is determined, setting stricter judgment criteria for deciding to implement reclining state control than when the schedule for transition is not determined (S104).

[0008] In these aspects, the reclining state of the driver's seat is controlled so that the driver is included in the detection range of the driver monitor. Therefore, even if the driver who is released from the duty of monitoring the surroundings tries to lose his / her driving posture, the driver can remain in the detection range of the driver monitor by the reclining control. As a result, it is possible to suppress the interruption of the driver's state monitoring during the automated driving period when there is no duty of monitoring the surroundings.

[0009] In order to achieve the above object, one disclosed aspect is a state estimation device for estimating a state of a driver, which is used in a host vehicle (Am) capable of traveling by autonomous driving control with no obligation for the driver to monitor the surroundings, and which estimates the state of the driver, the state estimation device including: a monitoring state grasping unit (85) that grasps head information related to the shape of the head captured in the driver's captured image and face information related to parts of the face captured in the driver's captured image based on a driver captured image obtained by photographing a detection range defined to include at least the head of the driver seated in the driver's seat; and a driver state estimation unit (86) that estimates the state of the driver by combining the head information and the face information when the host vehicle is traveling by autonomous driving control, and the driver state estimation unit changes weighting of the head information and the face information in estimating the state of the driver depending on whether the host vehicle is traveling by driving assistance control with the obligation for the driver to monitor the surroundings or the host vehicle is traveling by autonomous driving control. When the vehicle is traveling by autonomous driving control, the weight of head information in estimating the driver's state is increased compared to when the vehicle is traveling by driving assistance control. This is a state estimation device.

[0010] Another disclosed aspect is a state estimation program for estimating a driver's state, which is used in a host vehicle (Am) capable of traveling by autonomous driving control with no obligation for the driver to monitor the surroundings, and which estimates the state of the driver, and which grasps head information related to the shape of the head captured in the driver's captured image and face information related to the facial features captured in the driver's captured image based on a driver captured image obtained by photographing a detection range defined to include at least the head of the driver seated in the driver's seat (S32), and when the host vehicle is traveling by autonomous driving control, estimates the state of the driver by combining the head information and face information (S33), and changes the weighting of the head information and face information in estimating the state of the driver depending on whether the host vehicle is traveling by driving assistance control with the obligation for the driver to monitor the surroundings or the autonomous driving control (S21 to S23); When the vehicle is traveling under autonomous driving control, the weighting of head information in estimating the driver's state is increased compared to when the vehicle is traveling under driving assistance control. The state estimation program causes at least one processing unit (11) to execute a process including the steps of:

[0011] In these aspects, when driving by autonomous driving control, the driver's state is estimated by combining head information and face information. Therefore, even if the driver who is freed from the duty of monitoring the surroundings looks down to operate a mobile terminal or the like, the face information that is difficult to grasp can be supplemented by the head information. As a result, it is possible to suppress interruption of the driver's state monitoring during the autonomous driving period when there is no duty of monitoring the surroundings.

[0012] The reference numbers in parentheses above and in the claims merely indicate an example of a corresponding relationship with a specific configuration in an embodiment described later, and do not limit the technical scope in any way. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing an overall view of an in-vehicle network including an HCU according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram showing details of an HCU. [Diagram 3] 13 is a flowchart showing details of a weighting change process. [Figure 4] 10 is a flowchart showing details of a state estimation process. [Diagram 5] 6 is a time chart showing an example of information presentation and reclining control performed by the HCU. [Figure 6] 10 is a flowchart showing details of a seat control process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] 1 and 2, an HCU (Human Machine Interface Control Unit) according to an embodiment of the present disclosure is an interface control device used in a vehicle (hereinafter, the host vehicle Am). The HCU 100 constitutes an HMI (Human Machine Interface) system 10 of the host vehicle Am together with a plurality of display devices, an audio device 24, an operation device 26, and the like. The HMI system 10 has an input interface function that accepts operations by a passenger such as a driver of the host vehicle Am, and an output interface function that presents information to the driver.

[0015] The HCU 100 is communicatively connected to a communication bus 99 of an in-vehicle network 1 mounted on the vehicle Am. The HCU 100 is one of a plurality of nodes provided in the in-vehicle network 1. The surroundings monitoring sensor 30, the locator 35, the cruise control ECU 40, the driving assistance ECU (Electronic Control Unit) 50a, the automatic driving ECU 50b, and the like are connected as nodes to the communication bus 99 of the in-vehicle network 1. Furthermore, the seat actuator 28, the driver monitor 29, and the like are connected to the communication bus 99. These nodes connected to the communication bus 99 can communicate with each other. Specific nodes among these devices and each ECU, etc. may be directly electrically connected to each other and can communicate with each other without going through the communication bus 99.

[0016] The periphery monitoring sensor 30 is an autonomous sensor that monitors the surrounding environment of the host vehicle Am. The periphery monitoring sensor 30 includes, for example, one or more of a camera unit 31, a millimeter wave radar 32, a lidar 33, and a sonar 34. The periphery monitoring sensor 30 can detect moving objects and stationary objects within a detection range around the host vehicle. The periphery monitoring sensor 30 provides detection information of objects around the host vehicle to the driving assistance ECU 50a, the autonomous driving ECU 50b, etc.

[0017] The locator 35 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, etc. The locator 35 further includes a map database that stores three-dimensional map data and two-dimensional map data. The locator 35 reads out map data of the surroundings of the current position from the map database, and provides the map data together with the position information and direction information of the host vehicle Am as locator information to the driving assistance ECU 50a, the autonomous driving ECU 50b, etc.

[0018] The driving control ECU 40 is an electronic control device mainly including a microcontroller. The driving control ECU 40 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The driving control ECU 40 continuously controls the brake force of each wheel, the output control of the in-vehicle power source, and the steering angle control based on one of an operation command based on the driver's driving operation, a control command from the driving assistance ECU 50a, and a control command from the automatic driving ECU 50b.

[0019] The driving assistance ECU 50a and the automatic driving ECU 50b configure an automatic driving system 50 of the host vehicle Am. By installing the automatic driving system 50, the host vehicle Am becomes an automatic driving vehicle equipped with an automatic driving function, and is capable of traveling by the automatic driving function.

[0020] The driving assistance ECU 50a realizes a driving assistance function that assists the driver in driving operations in the automated driving system 50. The driving assistance ECU 50a enables advanced driving assistance of about level 2 or partial automated driving in the automated driving levels defined by the Society of Automotive Engineers. The automated driving performed by the driving assistance ECU 50a is an automated driving with a surrounding monitoring obligation that requires the driver to visually monitor the surroundings of the vehicle.

[0021] The driving assistance ECU 50a is a computer mainly including a control circuit equipped with a processing unit, a RAM, a storage unit, an input / output interface, and a bus connecting these. The driving assistance ECU 50a realizes driving assistance functions such as ACC (Adaptive Cruise Control), LTC (Lane Trace Control), and LCA (Lane Change Assist) by executing a program in the processing unit. As an example, the driving assistance ECU 50a performs driving assistance control to drive the host vehicle Am along the host vehicle lane Lns by cooperation of the ACC and LTC functions.

[0022] The autonomous driving ECU 50b is an in-vehicle ECU that realizes an autonomous driving function that can take over driving operations from the driver. The autonomous driving ECU 50b can perform autonomous driving of level 3 or higher, where the system is the main controller. The autonomous driving performed by the autonomous driving ECU 50b does not require monitoring of the surroundings of the vehicle, that is, it is eyes-off autonomous driving with no obligation to monitor the surroundings.

[0023] The autonomous driving ECU 50b is a computer that mainly includes a control circuit equipped with a processing unit 51, a RAM 52, a storage unit 53, an input / output interface 54, and a bus connecting these. The autonomous driving ECU 50b has a higher computing power than the driving assistance ECU 50a, and can at least perform driving control equivalent to ACC, LTA, and LCA. When the autonomous driving ECU 50b detects congestion around the vehicle in a pre-set autonomous driving possible area (hereinafter, AD area, see FIG. 5), it executes autonomous driving control (hereinafter, level 3 during congestion) that is performed only when driving in congestion.

[0024] In the above-described autonomous driving system 50, the driving control state of the autonomous driving function is switched among a plurality of states including at least driving assistance control by the driving assistance ECU 50a with the obligation to monitor the surroundings, and autonomous driving control by the autonomous driving ECU 50b with no obligation to monitor the surroundings. In the following description, autonomous driving control of level 2 or lower by the driving assistance ECU 50a is referred to as "driving assistance control," and autonomous driving control of level 3 or higher by the autonomous driving ECU 50b is referred to as "autonomous driving control."

[0025] During the autonomous driving period in which the host vehicle Am is driven by the autonomous driving control by the autonomous driving ECU 50b, the driver may be permitted to perform a specific action other than driving that is prescribed in advance (hereinafter, a second task). The second task is legally permitted to the driver until a request for execution of a driving operation performed in cooperation with the autonomous driving ECU 50b and the HCU 100, that is, a request for a driving handover occurs. For example, actions such as watching entertainment content such as video content, operating a device such as a smartphone, and eating are assumed as second tasks.

[0026] The seat actuator 28 is a drive device that adjusts the seat position of the driver's seat DS. The seat actuator 28 has the function of moving the position of the seating surface of the driver's seat DS in the front-rear and left-right directions, and the function of changing the angle of the backrest of the driver's seat DS relative to the seating surface (hereinafter referred to as the reclining angle). In addition, the seat actuator 28 has position sensors that detect the position of the seating surface and the reclining angle.

[0027] The seat actuator 28 adjusts the position of the seating surface and the reclining angle based on the operation by the driver and a control signal input from the HCU 100. In addition, the seat actuator 28 provides information on the position of the seating surface and the reclining angle detected by each position sensor to the HCU 100, etc. as seat information indicating the state of the driver's seat DS.

[0028] The driver monitor 29 is mounted on the vehicle Am and monitors the state of the driver of the vehicle Am. The driver monitor 29 includes a near-infrared light source, a near-infrared camera, and a control unit for controlling them. The driver monitor 29 is installed on the upper side of the backrest of the driver's seat DS, specifically, on the upper surface of the steering column or the upper surface of the instrument panel, with the near-infrared camera facing the headrest. The near-infrared camera is integrally configured with the meter display 21 or the center information display (hereinafter, CID) 22, which will be described later, and is provided on either of the screens.

[0029] The driver monitor 29 uses a near-infrared camera to capture an image of the driver's head illuminated with near-infrared light by a near-infrared light source. The detection range (image capture range) of the near-infrared camera is predefined to be a range with the headrest portion at the approximate center so as to include at least the head of the driver seated in the driver's seat DS. The image captured by the near-infrared camera (hereinafter, the driver captured image) is subjected to image processing or image analysis by the control unit. The control unit may be capable of extracting information such as the position of the driver's eye point and the line of sight from the driver captured image. The driver monitor 29 provides at least one of the driver captured image and the extracted information from the image to the HCU 100, the autonomous driving ECU 50b, etc. as driver status information.

[0030] Next, the multiple display devices, the operation device 26, and the HCU 100 included in the HMI system 10 will be described in detail in turn.

[0031] The display device presents information to the driver's vision by displaying images or the like. The display devices include a meter display 21, a CID 22, and a head-up display (hereinafter, HUD) 23. The CID 22 has a touch panel function and detects touch operations on the display screen by the driver or the like. The audio device 24 has multiple speakers installed in the vehicle cabin in an arrangement surrounding the driver's seat DS, and reproduces notification sounds, voice messages, or the like through the speakers in the vehicle cabin.

[0032] The operation device 26 is an input unit that accepts user operations by a driver or the like. User operations related to, for example, activation and deactivation of an automatic driving function are input to the operation device 26. As an example, a driver input instructing a transition from driving assistance control to autonomous driving control (hereinafter, level 3 transition operation) is input to the operation device 26. The operation device 26 includes a steer switch provided on the spokes of the steering wheel, an operation lever provided on the steering column, and a voice input device that recognizes the contents of the driver's speech.

[0033] The HCU 100 comprehensively controls the presentation of information using a plurality of display devices and the audio device 24. In addition, the HCU 100 further includes a function of a seat control device that cooperates with a seat actuator 28 to control the state of a driver's seat DS in which the driver sits, and a function of a state estimation device that cooperates with a driver monitor 29 to estimate the state of the driver.

[0034] The HCU 100 mainly includes a control circuit including a processing unit 11, a RAM 12, a storage unit 13, an input / output interface 14, and a bus connecting these. The processing unit 11 is hardware for arithmetic processing coupled to the RAM 12. The processing unit 11 includes at least one arithmetic core such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing unit 11 may further include an FPGA (Field-Programmable Gate Array), an NPU (Neural network Processing Unit), and an IP core having other dedicated functions. The RAM 12 may include a video RAM for generating video data. The processing unit 11 accesses the RAM 12 to execute various processes for presentation control processing. The storage unit 13 includes a non-volatile storage medium. The storage unit 13 stores various programs (such as a seat control program and a state estimation program) executed by the processing unit 11.

[0035] The HCU 100 configures a plurality of functional units by executing each program stored in the storage unit 13 with the processing unit 11. The HCU 100 configures functional units such as an information acquisition unit 81, an information linking unit 82, a driver information grasping unit 85, a driver state estimation unit 86, a presentation control unit 88, and a reclining control unit 89 (see FIG. 2).

[0036] The information acquisition unit 81 acquires vehicle information (e.g., vehicle speed information, etc.) indicating the state of the host vehicle Am from the communication bus 99. In addition, the information acquisition unit 81 acquires operation information indicating the content of a user operation from the CID 22, the operation device 26, etc.

[0037] The information linking unit 82 links with the autonomous driving ECU 50b and enables information sharing between the autonomous driving system 50 and the HCU 100. The information linking unit 82 provides the autonomous driving ECU 50b with operation information grasped by the information acquisition unit 81, driver estimation information (described later) estimated by the driver state estimation unit 86, and the like.

[0038] The information linking unit 82 grasps the operating state of the automatic driving by the automatic driving system 50 by acquiring control status information indicating the state of the automatic driving function. Based on the control status information, the information linking unit 82 grasps whether the driving control being performed is driving assistance control or autonomous driving control, in other words, whether the driver has an obligation to monitor the surroundings in the driving control performed by the automatic driving function. In addition, the information linking unit 82 grasps the schedule for transition from driving assistance control to automatic driving control when the host vehicle Am is driven by driving assistance control, and grasps the schedule for transition from autonomous driving control to driving assistance control when the host vehicle Am is driven by autonomous driving control.

[0039] The driver information ascertaining unit 85 acquires driver status information including at least one of a captured image of the driver and extracted information from the driver monitor 29. The driver information ascertaining unit 85 ascertains the operating state of the driver monitor 29 based on the driver status information. As an example, when the upper body of the driver is out of the detection range of the driver monitor 29, the driver information ascertaining unit 85 ascertains whether the driver monitor 29 can detect the driver, in other words, whether the driver detection in the driver monitor 29 has been interrupted. When the driver information ascertaining unit 85 ascertains that the driver detection by the driver monitor 29 has been interrupted, it notifies the presentation control unit 88 of the interruption of the driver monitoring.

[0040] The driver information grasping unit 85 grasps head information and face information based on the driver's captured image based on the driver status information. The head information is information related to the shape of the head shown in the driver's captured image, and is information extracted mainly from the outline shape of the head. The face information is information related to the facial features (e.g., eyes, nose, mouth, etc.) shown in the driver's captured image, and is information extracted mainly from the positional relationship of each feature.

[0041] The head information and face information may be generated by either the driver information ascertaining unit 85 or the driver monitor 29. For example, when a driver captured image is provided from the driver monitor 29 to the driver information ascertaining unit 85, all of the head information and face information may be prepared by the driver information ascertaining unit 85. In addition, at least a part of the head information and face information may be generated by a control unit of the driver monitor 29 and provided to the driver information ascertaining unit 85 as driver status information.

[0042] The driver state estimation unit 86 grasps the state and behavior of the driver based on the information grasped by the driver information grasping unit 85. The driver state estimation unit 86 grasps whether the driver is monitoring the surroundings of the vehicle Am, whether the driving posture is unstable, etc. The driver state estimation unit 86 grasps the contents of the second task performed by the driver during the autonomous driving period. The driver state estimation unit 86 may further grasp whether the steering wheel is being gripped, and abnormalities in the driver's physical condition, etc.

[0043] The driver state estimation unit 86 cooperates with the information linking unit 82 and the driver information grasping unit 85 to perform a weighting change process (FIG. 3) and a state estimation process (FIG. 4). The weighting change process and the state estimation process are started by the HCU 100 that has completed the initial process after the power supply of the host vehicle Am is turned on, and are repeatedly performed until the power supply of the host vehicle Am is turned off.

[0044] In the weighting change process, the driver state estimation unit 86 changes the weighting of the head information and face information in the driver state estimation depending on whether the host vehicle Am is traveling under driving assistance control or under autonomous driving control. In the weighting change process, it is determined whether or not level 3 autonomous driving control is being executed based on the control status information (S21).

[0045] When level 3 autonomous driving control is being executed in the automatic driving system 50 (S21: YES), the driver state estimation unit 86 selects the setting for level 3 (S22). In the setting for level 3, the weighting of the head information is increased, and the proportion of the head information in the driver's state estimation is increased compared to the setting for level 2. In other words, the influence of the head information on the driver's state estimation is greater when autonomous driving control is executed than when driving assistance control is executed.

[0046] On the other hand, if level 2 driving assistance control is being executed in the autonomous driving system 50 (S21: NO), the driver state estimation unit 86 selects the setting for level 2 (S23). In the setting for level 2, the weighting of head information is reduced, and the weighting of face information in the driver state estimation is increased compared to the setting for level 3.

[0047] In the state estimation process, the driver state estimation unit 86 estimates the state of the driver while the host vehicle Am is traveling by autonomous driving control or a driving assistance function. The driver state estimation unit 86 continues to grasp the driver state by repeating the state estimation process. In the state estimation process, the driver state estimation unit 86 reads the weighting settings of the head information and face information based on the weighting change process (S31). The step of reading the weighting settings does not have to be performed every time, and may be performed only once at a predetermined time.

[0048] The driver state estimation unit 86 cooperates with the driver information grasping unit 85 to grasp face information and head information based on the latest captured image of the driver (S32). The driver state estimation unit 86 combines the face information and the head information with a ratio reflecting the loaded weighting setting to estimate the driver's state (S33). Specifically, the driver state estimation unit 86 estimates the driver's state such as the driver's line of sight, whether or not the driver is operating a mobile terminal, whether or not the driver is sleeping, whether or not the driver has consumed alcohol, etc. The estimation result of the driver state by the driver state estimation unit 86 is provided to the presentation control unit 88, the autonomous driving ECU 50b, etc. as driver estimation information.

[0049] The presentation control unit 88 performs integrated control of the provision of information to the driver using each display device and the audio device 24. The presentation control unit 88 provides content and presents information in accordance with the operating state of the autonomous driving, based on the control status information acquired by the information linking unit 82 and the driver status information acquired by the driver information grasping unit 85, etc. (see FIG. 5 ).

[0050] Specifically, when approaching traffic congestion is detected while the host vehicle Am is traveling in an AD area under level 2 driving assistance control, the presentation control unit 88 issues a level 3 approach notification using the HUD 23, the meter display 21, etc. The level 3 approach notification is a notification that indicates to the driver that it is likely that level 3 automated driving (level 3 in traffic congestion) will be available, and for example, a message such as "Traffic jam ahead, 2nd task available" is displayed.

[0051] Furthermore, when it is determined that the host vehicle Am is entering a congestion period, the presentation control unit 88 issues a level 3 possible notification. The level 3 possible notification is a notification that indicates to the driver that level 3 autonomous driving control has become available. In the level 3 possible notification, for example, a message such as "2nd task is now available" is displayed.

[0052] After the level 3 possible notification, when the information acquisition unit 81 grasps the driver's input of a level 3 transition operation, the presentation control unit 88 issues a level 3 start notification. The level 3 start notification is a notification that indicates to the driver that level 3 autonomous driving control has started. In the level 3 start notification, for example, a message such as "Pay attention to information from the vehicle. Autonomous driving will be canceled due to surrounding road conditions" is displayed. In addition, when the information linking unit 82 grasps that autonomous driving control is being performed by the autonomous driving ECU 50b, the presentation control unit 88 enables playback of video content, etc. by the CID 22.

[0053] When the driver monitor 29 cannot detect the driver after the autonomous driving control is started, the presentation control unit 88 makes a posture improvement request based on a driver monitoring interruption notification acquired from the driver information grasping unit 85. For example, when the driver's head is out of the detection range of the driver monitor 29 due to a breakdown in driving posture (see frame-out in FIG. 5), the posture improvement request is implemented. The posture improvement request is an interruption notification that indicates to the driver that the detection by the driver monitor 29 has been interrupted, and is a notification that requests the driver to improve the driving posture so that the head returns to the detection range of the driver monitor 29. In the posture improvement request, for example, a message such as "Please take a correct driving posture" is displayed on the CID 22 and is played in the vehicle cabin by the audio device 24. When the driver improves the driving posture in response to the posture improvement request and the driver's head returns to the detection range of the driver monitor 29 (see frame-in in FIG. 5), the presentation control unit 88 ends the posture improvement request.

[0054] When the autonomous driving ECU 50b predicts the end of autonomous driving control due to the end of a congested section, etc., the presentation control unit 88 makes a driving handover request. The driving handover request is a notification to the driver that the autonomous driving control of level 3 is scheduled to end and that the driver needs to take over control of the driving operation. In the driving handover request, a message such as "Autonomous driving will be canceled. Please hold the steering wheel" is displayed on at least one of the HUD 23 and the meter display 21. Furthermore, in accordance with the driving handover request, playback of video content by the CID 22 is stopped.

[0055] When the level 3 autonomous driving control (level 3 in traffic jam) is ended, the presentation control unit 88 issues a level 3 end notification. The level 3 end notification is a notification that notifies the driver of the transition from autonomous driving control to driving assistance control. In the level 3 end notification, a message indicating the end of autonomous driving, such as "Autonomous driving canceled", is displayed on at least one of the HUD 23 and the meter display 21.

[0056] The reclining control unit 89 grasps the position of the seating surface of the driver's seat DS and the reclining angle of the backrest based on the seat information acquired from the reclining control unit 89. In addition, the reclining control unit 89 can adjust the position of the seating surface of the driver's seat DS and the reclining angle of the backrest by outputting a drive signal (control command) to the seat actuator 28. The reclining control unit 89 controls at least the reclining state of the driver's seat DS so that the driver is included in the detection range of the driver monitor 29 during the automatic driving period when the host vehicle Am is traveling by the autonomous driving control. The reclining control unit 89 adjusts the reclining angle and the position of the seating surface so that the driver is present in a predetermined position where the state monitoring by the driver monitor 29 is good. The predetermined position is the driver's position where the driver's head and face are located near the center of the detection range.

[0057] The reclining control unit 89 performs reclining control of the driver's seat DS to avoid interruption of driver monitoring during the period from the time of Level 3 start notification, at which the start of autonomous driving control is grasped, to the time of the start of a driving change request (see the dotted range in FIG. 5). The reclining control unit 89 starts reclining control when the driver monitor 29 does not detect the driver, or the driver state estimation unit 86 cannot estimate the driver state. As an example, after the driver's head goes out of frame (see FIG. 5), the reclining control is performed when the driver's driving posture is not improved even in response to a posture improvement request and the driver monitor 29 continues to be unable to detect the driver.

[0058] When the reclining control unit 89 detects a schedule for transition from autonomous driving control to driving assistance control, it performs reclining control in accordance with the timing at which a driving change request (see FIG. 5) is initiated. As one example, the reclining control unit 89 performs control to return the reclining state to the state before the start of autonomous driving control. In detail, the reclining control unit 89 stores the positions of the seating surface and backrest of the driver's seat DS during the driving assistance period, and performs control to return the position of the seating surface and the reclining angle to the stored positions.

[0059] Next, details of the seat control process for realizing the reclining control described above will be described below based on Fig. 6 and with reference to Fig. 2 and Fig. 5. The seat control process is started by the HCU 100 based on the start of traveling under the driving assist control, for example.

[0060] In S101, the information linking unit 82 determines whether the host vehicle Am is traveling under autonomous driving control without the obligation to monitor the surroundings, and determines whether autonomous driving control is to be started. If it is determined in S101 that traveling under autonomous driving control is to be started, the process proceeds to S102.

[0061] In S102, the reclining control unit 89 stores the reclining position etc. before the start of the autonomous driving control based on the seat information acquired from the seat actuator 28, and proceeds to S103. Note that the process of storing the reclining position etc. may be performed during the period from when the Level 3 possible notification is issued to when the Level 3 start notification is issued.

[0062] In S103, the information linking unit 82 determines whether or not there is a plan to end the autonomous driving control at level 3. If it is determined in S103 that there is a plan to end the autonomous driving control, in other words, if it is determined that there is a plan to transition to driving assistance control at level 2 or lower or manual driving, the process proceeds to S104.

[0063] In S104, the driver state estimation unit 86 judges whether the driving posture is good or bad. The conditions for judging the driving posture in S104 may be stricter than the conditions for judging the driving posture in S106 described later. If it is judged in S104 that the driver's head is well positioned within the detection range of the driver monitor 29, the seat control process ends. On the other hand, if most of the driver's head is out of the detection range of the driver monitor 29 or the driver has his face facing downward, it is judged that the driving posture is inappropriate, and the process proceeds to S105.

[0064] In S105, the reclining control unit 89 performs reclining control. In S105, the information stored in S102 is read out, and each position of the driver's seat DS is adjusted to the state before the start of the autonomous driving control of level 3. When the reclining control is completed, the seat control process ends.

[0065] On the other hand, if it is determined in S103 that there is no plan to switch to driving assistance control or manual driving and that level 3 autonomous driving control will continue, the process proceeds to S106. In S106, the driving posture is also determined to be good or bad. In S106, the driver information grasping unit 85 grasps the continuation state of driver monitoring by the driver monitor 29 and judges whether the driver's head is framed out of the detection range of the driver monitor 29. If it is determined in S106 that the driver's head is located within the detection range, the process proceeds to S110. On the other hand, if it is determined in S106 that most of the driver's head is out of the detection range, the process proceeds to S107. In S107, the presentation control unit 88 starts a posture improvement request indicating that the driver monitor 29 is not able to recognize the driver.

[0066] In S108, it is determined whether the driver has improved his / her driving posture in response to the posture improvement request. If it is determined in S108 that the driver's driving posture has improved and the driver's head is located approximately in the center of the detection range, the process proceeds to S110. On the other hand, if it is determined in S108 that the driving posture has not improved and the driver monitoring continues to be suspended, the process proceeds to S109.

[0067] In S109, the reclining control unit 89 controls the reclining state of the driver's seat DS so that the driver's head is included in the detection range of the driver monitor 29. The reclining control in S109 may be a control for returning the position of the seating surface and the reclining angle to each position stored in S102, similar to the reclining control in S105. Alternatively, the reclining control unit 89 may refer to the position information of the driver's head grasped by the driver state estimation unit 86 and perform reclining control to position the driver's head in the center of the detection range.

[0068] Here, if the driver monitoring continues to be suspended even after the reclining control in S109, the autonomous driving ECU 50b determines that it is difficult to continue the level 3 autonomous driving control, and starts transitioning to MRM (Minimal Risk Maneuver) control. The MRM control is an automatic evacuation control that sets an evacuation location within the lane in which the vehicle is traveling or the side of the road (shoulder) of the road in which the vehicle is traveling as an evacuation location, and stops the host vehicle Am at the evacuation location.

[0069] In S110, the information linking unit 82 determines whether or not the autonomous driving control at level 3 is to be ended. If it is determined in S110 that the autonomous driving control is to be continued, the process returns to S103. On the other hand, if it is determined in S110 that the autonomous driving control is to be ended, the seat control process for this time is ended.

[0070] In the present embodiment described so far, the reclining state of the driver's seat DS is controlled so that the driver is included in the detection range of the driver monitor 29. Therefore, even if the driver, who is released from the obligation to monitor the surroundings, tries to lose his / her driving posture, the driver can remain within the detection range of the driver monitor 29 by the reclining control. As a result, it is possible to suppress interruption of the monitoring of the driver's state during the automated driving period when there is no obligation to monitor the surroundings.

[0071] As a result, it is possible to avoid a situation in which autonomous driving control is forcibly terminated and transition to MRM control is initiated due to an inappropriate driving posture. In other words, it becomes easier to continue level 3 autonomous driving. Therefore, the reclining control of this embodiment can contribute to improving the convenience of autonomous driving.

[0072] Additionally, in this embodiment, when the start of autonomous driving control is detected, the reclining state of the driver's seat DS is controlled so that the driver is in a predetermined position where the state can be monitored well by the driver monitor 29. As a result of the above, interruptions to driver monitoring after the start of Level 3 autonomous driving can be further suppressed.

[0073] In addition, in this embodiment, when a schedule for transition from autonomous driving control to driving assistance control is grasped, reclining control is performed to return the reclining state to the state before the autonomous driving control started. As a result, when taking over driving, the driver can smoothly start monitoring the surroundings around the vehicle.

[0074] Furthermore, in this embodiment, if the driver monitor 29 continues to be unable to detect the driver even after a posture improvement request is issued, the reclining control unit 89 starts the reclining control. As a result, it is possible to prevent the driver from feeling uneasy or uncomfortable due to the abrupt execution of the reclining control.

[0075] In addition, in this embodiment, when the host vehicle Am is traveling by autonomous driving control, the driver's state is estimated by combining head information and face information. Therefore, even if the driver who is freed from the duty of monitoring the surroundings looks down to operate a mobile terminal or the like, the face information that is difficult to grasp can be supplemented by the head information. As a result, it is possible to suppress interruption of the driver's state monitoring during the automatic driving period when there is no duty of monitoring the surroundings.

[0076] In addition, the driver state estimation unit 86 of this embodiment changes the weighting of head information and face information in estimating the driver's state between when the host vehicle Am is traveling by driving assistance control with a periphery monitoring obligation and when the host vehicle Am is traveling by autonomous driving control. Therefore, the state of the driver who is monitoring the periphery and the state of the driver who is not monitoring the periphery can be estimated by processing suitable for each state. Therefore, the estimation accuracy of the driver state can be improved.

[0077] Furthermore, in this embodiment, when the host vehicle Am is traveling by autonomous driving control, the weight of head information in the driver state estimation is higher than when the host vehicle Am is traveling by driving assistance control. Therefore, the accuracy of the driver state estimation can be ensured even during the autonomous driving period when it is difficult to extract face information.

[0078] Additionally, in this embodiment, when the driver's state estimation is interrupted, the reclining state of the driver's seat DS is controlled so that the head is positioned within the detection range of the driver monitor 29. As described above, by combining the complementation of face information by head information with the reclining control, interruptions in the driver's state monitoring can be further suppressed.

[0079] In the above embodiment, the information linking unit 82 corresponds to a "control grasping unit", the driver information grasping unit 85 corresponds to a "monitoring state grasping unit", the presentation control unit 88 corresponds to a "notification control unit", and the HCU 100 corresponds to a "seat control device" and a "state estimation device". In addition, the posture improvement request corresponds to an "interruption notification".

[0080] (Other embodiments) Although one embodiment of the present disclosure has been described above, the present disclosure should not be construed as being limited to the above embodiment, and can be applied to various embodiments and combinations within the scope not departing from the gist of the present disclosure.

[0081] In the reclining control of the above embodiment, both the position of the seating surface and the reclining angle are adjusted. In contrast, in the first modified example of the above embodiment, only the reclining angle is adjusted in the reclining control. In addition, the steering position and the like may be adjusted together with the reclining angle as a driving position control linked to the reclining control.

[0082] In the reclining control of the above embodiment, an adjustment is performed to return the state of the driver's seat DS to the position stored when switching to the autonomous driving control. In contrast, in the second modification of the above embodiment, a reclining control is performed to limit the adjustment ranges of the seat position and the reclining angle so that the driver's head does not go out of the detection range.

[0083] In the modified examples 3 and 4 of the above embodiment, the process of storing the position when moving to level 3 is omitted. In the modified example 3, a reference state is set in advance such that the driver is in a predetermined position where the state monitoring is good, and the reclining control unit 89 performs reclining control to adjust the reclining angle, etc. to the reference state. In the modified example 4, the reclining control is performed to refer to information extracted from the captured image of the driver and adjust the reclining angle, etc. to position the driver's face position in the center of the detection range.

[0084] In the fifth modified example of the above embodiment, the posture improvement request is omitted. In the fifth modified example, the reclining control is immediately started when the driver monitor 29 stops monitoring the driver. In the sixth modified example of the above embodiment, the reclining control is performed in parallel with the posture improvement request.

[0085] In the above embodiment, the driver's head information is also used for estimating the driver's state during the driving assistance period. On the other hand, in the seventh modification of the above embodiment, the driver's state is estimated during the driving assistance period using only the driver's face information. Also, the driver's state is estimated during the autonomous driving period using only the driver's head information.

[0086] The HCU 100 of the above embodiment can estimate the state of the driver even during driving assistance control of automated driving level 1 or manual driving. The driver state estimation unit 86 may select a weighting setting for level 2 or a weighting setting different from that for level 2 during driving assistance control of level 1 or manual driving.

[0087] In the above embodiment, the judgment criteria for judging whether the driving posture is good or bad when the level 3 is scheduled to end (see S104 in FIG. 6) are stricter than the judgment criteria for judging whether the driving posture is good or bad during the level 3 continuation period (see S106 in FIG. 6). However, the judgment criteria used when the level 3 is scheduled to end and the judgment criteria used during the level 3 continuation period may be substantially the same. Furthermore, the judgment criteria used during the level 3 continuation period may be stricter than the judgment criteria used when the level 3 is scheduled to end.

[0088] In the above embodiment, the functions of the seat control device and the state estimation device implemented in the HCU 100 may be implemented in an in-vehicle ECU other than the HCU 100. For example, the function of the seat control device may be implemented in the driving assistance ECU 50a, the automatic driving ECU 50b, or a control unit of the seat actuator 28. The function of the state estimation device may be implemented in the driving assistance ECU 50a, the automatic driving ECU 50b, or a control unit of the driver monitor 29. Furthermore, the function of the state estimation device may be implemented in a deadman determination device provided separately from the driver monitor 29.

[0089] The autonomous driving ECU 50b may be capable of implementing autonomous driving control in a driving control mode different from Level 3 during congestion, such as area limited control (hereinafter, Area Level 3) that is implemented within a specific area only. Furthermore, the autonomous driving ECU 50b may be capable of implementing autonomous driving control of Level 4 or higher.

[0090] In the eighth modification of the above embodiment, the functions of the autonomous driving ECU 50b and the HCU 100 are provided by a single in-vehicle ECU. In the eighth modification, the in-vehicle ECU that functions as both the autonomous driving ECU 50b and the HCU 100 corresponds to the "seat control device" and the "state estimation device."

[0091] In the above embodiment, each function provided by the HCU itself can be provided by software and hardware that executes the software, software alone, hardware alone, or a combination of these. Furthermore, when such functions are provided by electronic circuits as hardware, each function can also be provided by digital circuits including multiple logic circuits, or analog circuits.

[0092] Each processing unit in the above-mentioned embodiment may be individually mounted on a printed circuit board, or may be mounted on an ASIC (Application Specific Integrated Circuit) and an FPGA. The form of a storage medium (non-transitory tangible storage medium) for storing various programs and the like may also be changed as appropriate. Furthermore, the storage medium is not limited to a configuration provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to a control circuit such as an autonomous driving ECU or HCU. The storage medium may also be an optical disk or a hard disk drive that is a source from which a program is copied to the autonomous driving ECU or HCU.

[0093] The vehicle equipped with the above-mentioned automatic driving system and HMI system is not limited to a general private passenger car, and may be a rental car vehicle, a manned taxi vehicle, a ride-sharing vehicle, a freight vehicle, a bus, etc. In addition, the vehicle equipped with the automatic driving system and HMI system may be a right-hand drive vehicle or a left-hand drive vehicle. Furthermore, the traffic environment in which the vehicle travels may be a traffic environment based on left-hand traffic or a traffic environment based on right-hand traffic. The seat control and driver state estimation according to the present disclosure may be appropriately optimized according to the road traffic laws of each country and region, and further the steering wheel position of the vehicle, etc.

[0094] The control unit and the method described herein may be implemented by a special-purpose computer having a processor programmed to execute one or more functions embodied in a computer program. Alternatively, the device and the method described herein may be implemented by a special-purpose hardware logic circuit. Alternatively, the device and the method described herein may be implemented by one or more special-purpose computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. The computer program may also be stored in a computer-readable non-transitory tangible recording medium as instructions to be executed by the computer. [Explanation of symbols]

[0095] Am: vehicle; DS: driver's seat; 11: processing unit; 29: driver monitor; 82: information linking unit (control grasping unit); 85: driver information grasping unit (monitoring state grasping unit); 86: driver state estimating unit; 88: presentation control unit (alert control unit); 89: reclining control unit; 100: HCU (seat control device, state estimating device)

Claims

1. A seat control device for use in a host vehicle (Am) equipped with a driver monitor (29) for monitoring a state of a driver, the seat control device controlling a state of a driver's seat (DS) in which the driver is seated, A control recognition unit (82) that recognizes whether the vehicle is traveling under autonomous driving control without the driver having to monitor the surroundings; a reclining control unit (89) that controls a reclining state of the driver's seat so that the driver is included in a detection range of the driver monitor when the host vehicle is traveling under the autonomous traveling control; The reclining control unit is a seat control device that sets stricter judgment criteria for deciding to implement reclining state control when a planned transition from autonomous driving control to driving assistance control in which the driver is obligated to monitor the surroundings is detected than when the planned transition is not detected.

2. The seat control device according to claim 1 , wherein the reclining control unit, when the planned transition from the autonomous driving control to the driving assistance control is grasped, performs control to return the reclining state to the state before the autonomous driving control was started.

3. a monitoring state grasping unit (85) for grasping whether the driver monitor is able to detect the driver; and a notification control unit (88) that, when the driver monitor cannot detect the driver, issues an interruption notification to the driver indicating that the detection by the driver monitor has been interrupted.

3. The seat control device according to claim 1, wherein the reclining control unit starts the control of the reclining when the state in which the driver monitor is unable to detect the driver continues after the interruption notification.

4. The seat control device according to any one of claims 1 to 3, wherein the reclining control unit controls the reclining state when the start of the autonomous driving control is detected so that the driver is in a predetermined position where the state monitoring by the driver monitor is good.

5. A seat control program for use in a host vehicle (Am) equipped with a driver monitor (29) for monitoring a state of a driver, the seat control program controlling a state of a driver's seat in which the driver is seated, It is determined whether the vehicle is traveling under autonomous driving control without the driver having to monitor the surroundings (S101); When the host vehicle is traveling under the autonomous traveling control, the reclining state of the driver's seat is controlled so that the driver is included in the detection range of the driver monitor (S105, S109). When a schedule for transition from the autonomous driving control to a driving assistance control in which the driver is obligated to monitor the surroundings is grasped, a judgment criterion for determining whether to execute the reclining state control is set to be stricter than when the schedule for transition is not grasped (S104). A seat control program that causes at least one processing unit (11) to execute a process including the steps of:

6. A state estimation device for estimating a state of a driver of a vehicle (Am) that can travel by autonomous driving control without a driver having to monitor the surroundings, comprising: a monitoring state grasping unit (85) for grasping head information related to the shape of the head shown in the driver's captured image and face information related to facial features shown in the driver's captured image based on the driver's captured image obtained by photographing a detection range defined to include at least the head of the driver seated in the driver's seat; a driver state estimation unit (86) that estimates a state of the driver by combining the head information and the face information when the host vehicle is traveling under the autonomous traveling control, The driver state estimation unit a weighting of the head information and the face information in the estimation of the state of the driver is changed depending on whether the host vehicle is traveling under driving assistance control in which the driver is obligated to monitor the surroundings or the host vehicle is traveling under autonomous driving control; A state estimation device that places a higher weight on the head information in estimating the driver's state when the vehicle is traveling under the autonomous driving control than when the vehicle is traveling under the driving assistance control.

7. 7. The state estimation device according to claim 6, further comprising a reclining control unit (89) that controls a reclining state of the driver's seat so that the head is positioned within the detection range when the driver's state estimation is interrupted.

8. A state estimation program for estimating a state of a driver of a vehicle (Am) that can travel by autonomous driving control without a driver having to monitor the surroundings, comprising: Based on a driver captured image obtained by capturing an image of a detection range defined to include at least the head of the driver seated in the driver's seat, head information related to the shape of the head captured in the driver captured image and face information related to the facial features captured in the driver captured image are grasped (S32); When the vehicle is traveling under the autonomous travel control, the head information and the face information are combined to estimate the state of the driver (S33); A weighting of the head information and the face information in the driver's state estimation is changed between a case where the vehicle is traveling under driving assistance control with a surrounding monitoring obligation by the driver and a case where the vehicle is traveling under autonomous driving control (S21 to S23). When the host vehicle is traveling under the autonomous driving control, the weight of the head information in the estimation of the state of the driver is increased compared to when the host vehicle is traveling under the driving assistance control. A state estimation program that causes at least one processing unit (11) to execute a process including the steps of:

Citation Information

Patent Citations

  • Driver condition determination device and method

    JP2005108033A

  • Vehicle control system, vehicle control method, and vehicle control program

    JP2017197011A

  • Vehicle control device

    JP2019087874A

  • Vehicle control device

    JP2020128167A

  • Vehicle control system

    JP2020158090A