Driving assistance system, vehicle, recording medium on which computer program is recorded, and driving assistance method

Abstract

The present application provides a driving assistance system or the like capable of appropriately guiding a driving posture in accordance with both a traveling state of a vehicle and a state of a driver who drives the vehicle. In particular, a driving assistance control device 100 has a configuration for guiding a driver state when a driver drives a vehicle M to an appropriate state on the basis of the current driving scene in the vehicle M or both a future predicted driving scene and state of the driver in the vehicle M.

Description

Driving assistance system, vehicle, recording medium having computer program recorded thereon, and driving assistance method

[0001] The present disclosure relates to a driving assistance system, a vehicle, a recording medium having a computer program recorded thereon, and a driving assistance method.

[0002] In recent years, research has been conducted into technologies that provide appropriate driving guidance for the driver to suppress car sickness in the vehicle, including in other passengers, by controlling the driver's posture and stabilizing the vehicle's driving line while the vehicle is traveling.

[0003] In particular, recently, devices have become known that predict the movement of a vehicle and stimulate the driver to generate muscle tension that results in a posture that corresponds to the movement of the vehicle (e.g., Patent Document 1), or devices that control the driver's seat to guide the driver into an appropriate driving posture (e.g., Patent Document 2).

[0004] International Publication No. 2020 / 095082 Japanese Patent Application Laid-Open No. 2009-274673

[0005] However, devices such as those described in Patent Documents 1 and 2 do not detect the driver's condition and do not perform various controls on the driving posture to suit the driver's condition, so there are cases where they are unable to provide appropriate guidance for the driving posture.

[0006] The present disclosure has been made in consideration of the above problems, and the purpose of the present disclosure is to provide a driving assistance system that can guide a driver to an appropriate driving posture according to the driving state of the vehicle and the state of the driver operating the vehicle.

[0007] In order to solve the above problem, a driving assistance system according to a first aspect of the present disclosure is a driving assistance system that controls a force feedback device to provide a stimulating force to the driver's body by controlling the amount of stimulation, and assists the driver in driving a vehicle, the driving assistance system comprising one or more processors and one or more memories communicably connected to the one or more processors, wherein the one or more processors: execute a driving scene detection process to detect a current driving scene in the vehicle or a driving scene predicted to be future in the vehicle; execute a driver state detection process to detect the state of the driver; repeatedly execute output control for the stimulation of the force feedback device as a stimulation output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state; and execute a registration process to register the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process and the state of the driver before and after stimulation output in memory, in correspondence with the driving scene and the driver.

[0008] In addition, in order to solve the above problem, a vehicle according to a second aspect of the present disclosure is a vehicle equipped with a driving assistance system that controls a force feedback device to provide a stimulating force to the driver's body by controlling the amount of stimulation, and assists the driver in driving the vehicle, wherein the driving assistance system executes a driving scene detection process to detect a current driving scene in the vehicle or a driving scene predicted to be future in the vehicle, executes a driver state detection process to detect the state of the driver, repeatedly executes output control of the stimulation from the force feedback device as a stimulation output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state, and executes a registration process to register in memory the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process, and the state of the driver before and after the stimulation output, in correspondence with the driving scene and the driver.

[0009] In addition, in order to solve the above problem, a recording medium having recorded thereon a computer program according to a third aspect of the present disclosure is a recording medium having recorded thereon a computer program for a driving assistance system that controls a force feedback device for controlling the amount of stimulation to provide a force feedback to the driver's body and assists the driver in driving a vehicle, and is configured to cause a computer to: execute a driving scene detection process that detects the current driving scene in the vehicle or a driving scene predicted to be future in the vehicle; execute a driver state detection process that detects the state of the driver; repeatedly execute output control for the stimulation of the force feedback device as a stimulation output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state; and execute a registration process that registers in memory the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process, and the state of the driver before and after stimulation output, in correspondence with the driving scene and the driver.

[0010] In addition, in order to solve the above problems, a driving assistance method according to a fourth aspect of the present disclosure is a driving assistance method that controls a force feedback device to provide a stimulating force to the driver's body by controlling the amount of stimulation, and assists the driver in driving a vehicle, and is configured to: execute a driving scene detection process that detects a current driving scene in the vehicle or a driving scene predicted to be future in the vehicle; execute a driver state detection process that detects the state of the driver; repeatedly execute output control for the stimulation of the force feedback device as a stimulation output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state; and execute a registration process that registers in memory the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process, and the state of the driver before and after stimulation output, in correspondence with the driving scene and the driver.

[0011] The driving assistance system of the present disclosure can appropriately induce a driving posture in accordance with the vehicle's running state and the state of the driver driving the vehicle.

[0012] Therefore, the driving assistance system of the present disclosure can not only suppress car sickness for many drivers, but also improve their driving skills and lead to safer driving.

[0013] FIG. 1 is a schematic diagram showing a configuration example of a vehicle according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram showing a configuration example of a seat-type haptic device according to an embodiment. FIG. 3 is a schematic diagram showing a configuration example of a wearable haptic device according to an embodiment. FIG. 4 is an example of a system configuration diagram showing a configuration of a driving assistance control device mounted on a vehicle according to an embodiment. FIG. 5 is a diagram for explaining driving assistance control processing including feedback control of a stimulus output control processing executed by a driving assistance control device according to an embodiment. FIG. 6 is a diagram for explaining feedback control executed in a driving assistance control device according to an embodiment. FIG. 7 is a flowchart showing the operation of a driving assistance control processing including feedback control of a stimulus output control processing executed by a driving assistance control device according to an embodiment.

[0014] [A] Features of an embodiment of the present disclosure (1) An embodiment of the present disclosure is a driving assistance system that controls a force feedback device to provide a stimulating force to a driver's body by controlling the amount of stimulation, and assists the driver in driving a vehicle, the system comprising one or more processors and one or more memories communicably connected to the one or more processors, wherein the one or more processors: execute a driving scene detection process to detect a current driving scene in the vehicle or a driving scene predicted to be future in the vehicle; execute a driver state detection process to detect the state of the driver; repeatedly execute output control for the stimulation of the force feedback device as a stimulation output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state; and execute a registration process to register the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process and the state of the driver before and after stimulation output in memory, in correspondence with the driving scene and the driver.

[0015] In addition, the embodiments of the present disclosure can also be realized by a vehicle equipped with a driving assistance system that executes each of the above processes, a recording medium on which a computer program for executing each of the above processes is recorded, or a driving assistance method that executes each of the above processes.

[0016] With this configuration, the driving assistance system of the present disclosure can guide the driver's posture, facial direction, head position and direction, etc., to an appropriate state when driving the vehicle based on the detected vehicle state and driver state, respectively.

[0017] In other words, the driving assistance system of the present disclosure can appropriately guide the driver's state in accordance with both the vehicle's running state and the state of the driver driving the vehicle.

[0018] For example, the driving assistance system of the present disclosure can appropriately change the amount of stimulation to control the driver's driving posture even if the driver becomes accustomed to the stimulation, so that the system can continuously guide the driver to an appropriate driving posture.

[0019] Furthermore, for example, the driving assistance system of the present disclosure can appropriately change the amount of stimulation even when drivers with different driving characteristics or different physiques are driving, thereby guiding each driver to an appropriate driving posture.

[0020] Therefore, the driving assistance system of the present disclosure can not only suppress car sickness for many drivers, but also improve their driving skills and lead to safer driving.

[0021] The "force feedback device" refers to, for example, a driver's seat, a seat belt, or other in-vehicle equipment that comes into contact with the driver, or a wearable device such as goggles, clothing, or a belt. However, it is preferable that the force feedback device be linked to a control device that controls the vehicle.

[0022] A "driving scene" refers to the movement of a vehicle accompanying a driver's driving actions, such as turning right or left, parking, changing lanes, or merging, and may also be a concept that includes the location where the vehicle movement occurs, such as an intersection, a merging point, or a highway.

[0023] The "state of the driver" refers to, for example, the driver's posture and behavior (including the turning state of the upper body or head only, the movement of the eyes, or changes in head position).

[0024] "Detecting the driver's state" refers to, for example, analyzing the state using an in-vehicle camera installed in the vehicle, or detecting the seating state of the driver in the driver's seat.

[0025] (2) Furthermore, an embodiment of the present disclosure may perform a judgment process to determine whether the detected driver's state is an ideal state that is predefined based on the driving scene, and if the judgment process determines that the state is not ideal, adjust the amount of stimulation provided to the driver by the force feedback device, and control the output of the force feedback device based on the adjusted amount of stimulation. This stimulus output control process may be repeatedly performed together with the driver state detection process and the judgment process.

[0026] With this configuration, the driving assistance system of the present disclosure can easily guide the driver's state to an appropriate state, provided that the state in which the driver drives the vehicle, including the appropriate driver posture, facial direction, and head position and direction, is predefined as an ideal state.

[0027] The "ideal state" includes, for example, the appropriate driver posture, facial direction, and head position and direction.

[0028] (3) Furthermore, an embodiment of the present disclosure may be configured such that, when the ideal state is stored in the memory in association with the driving scene for each driver or for each attribute of the driver, if the determination process determines that the state is not ideal, the processor executes the stimulus output control process based on the ideal state of the driver to be assisted or the ideal state stored in association with the attribute of the driver, and the detected driving scene.

[0029] With this configuration, the driving assistance system of the present disclosure can control the driver's state in the current driving scene based on past driving scenes or the state of a driver with the same tendency, thereby improving the reproducibility of the driver's state.

[0030] Therefore, the driving assistance system of the present disclosure can easily guide the driver's state to an ideal state, and for many drivers, it can not only suppress car sickness but also improve their driving skills and lead to safer driving.

[0031] The "driver attributes" include, for example, factors related to the driver, such as gender, age, personality, physical information (height, bone structure, etc.), driving style, and sense of driving burden.

[0032] Furthermore, the "ideal state stored in association with the driver's attributes" includes not only the ideal state when the driver's attributes are the same, but also the ideal state when the attributes are considered to be the same.

[0033] In particular, "when attributes are considered to be the same" includes not only completely the same but also similar attributes. For example, "when driver attributes are the same or considered to be the same" includes not only the same age, but also the same "generation," or a standard age or an age within a certain range (e.g., ±5 years).

[0034] (4) Furthermore, an embodiment of the present disclosure may have a configuration in which the processor executes a vehicle state detection process to detect the state of the vehicle, and executes the driving scene detection process based on the state of the vehicle.

[0035] With this configuration, the driving assistance system of the present disclosure can detect the driving scene using vehicle conditions that have a significant impact on the driving scene, thereby generating the driving scene more accurately.

[0036] [B] Details of the Preferred Embodiments of the Present Disclosure Hereinafter, details of preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that in this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0037] [B1] Vehicle First, the vehicle M of this embodiment will be described with reference to FIGS.

[0038] Note that Figure 1 is a schematic diagram showing an example of the configuration of a vehicle M of this embodiment, and Figure 2 is an example of a system configuration diagram showing the configuration of a driving assistance control device 100 installed in the vehicle M of this embodiment.

[0039] (Basic Configuration of Vehicle) As shown in FIG. 1 , the vehicle M has a driving force source 9 that generates a driving torque for the vehicle M, and is configured to transmit the driving torque output from the driving force source 9 to the wheels 3.

[0040] In particular, the vehicle M is configured as a four-wheel drive vehicle that transmits drive torque to a left front wheel 3LF, a right front wheel 3RF, a left rear wheel 3LR, and a right rear wheel 3RR.

[0041] The driving force source 9 may be an internal combustion engine such as a gasoline engine or a diesel engine, a driving motor, or may include both an internal combustion engine and a driving motor.

[0042] The vehicle M may be, for example, an electric vehicle equipped with two drive motors, a front-wheel drive motor and a rear-wheel drive motor, or may be an electric vehicle equipped with drive motors corresponding to each wheel 3.

[0043] In addition, if the vehicle M is an electric vehicle or a hybrid electric vehicle, the vehicle M is equipped with a secondary battery that stores power to be supplied to the drive motor, or a generator such as a motor or fuel cell that generates power to charge the battery.

[0044] As shown in FIG. 1, the vehicle M is equipped with a driving force source 9, an electric steering device 15, and brake devices 17LF, 17RF, 17LR, and 17RR (hereinafter, collectively referred to as "brake devices") as devices used to control the operation of the vehicle.

[0045] The drive power source 9 outputs a drive torque that is transmitted to the front drive shaft 5F and the rear drive shaft 5R via a transmission (not shown) and a front wheel differential mechanism 7F and a rear wheel differential mechanism 7R.

[0046] The drive of the drive power source 9 and the transmission is controlled by a vehicle drive control unit 40 that includes one or more electronic control units (ECUs).

[0047] An electric steering device 15 including an electric motor and a gear mechanism (not shown) is provided on the front wheel drive shaft 5F, and the electric steering device 15 is controlled by the vehicle drive control unit 40 to adjust the steering angle of the left front wheel 3LF and the right front wheel 3RF.

[0048] During manual driving, the vehicle drive control unit 40 controls the electric steering device 15 based on the steering angle of the steering wheel 13 by the driver.

[0049] The brake devices 17LF, 17RF, 17LR, and 17RR apply braking forces to the front, rear, left, and right drive wheels 3LF, 3RF, 3LR, and 3RR, respectively.

[0050] The brake devices 17 are configured as hydraulic brake devices, for example, and the hydraulic pressure supplied to each brake device 17 is controlled by the vehicle drive control unit 40 to generate a predetermined braking force.

[0051] In addition, when the vehicle M is an electric vehicle or a hybrid electric vehicle, the brake device 17 is used in combination with a regenerative brake using a drive motor.

[0052] The vehicle drive control unit 40 includes one or more electronic control devices that control the drive of the drive power source 9, the electric steering device 15, and the brake device 17, and has the function of controlling the drive of the transmission that changes the speed of the output from the drive power source 9 and transmits it to the wheels 3 as necessary.

[0053] (Exterior Cameras) The exterior cameras 31 are cameras for acquiring information about the surrounding environment of the vehicle M, and are composed of front cameras 31LF and 31RF and a rear camera 31R.

[0054] In particular, the front imaging cameras 31LF, 31RF and the rear imaging camera 31R are equipped with imaging elements such as CCDs (Charged-Coupled Devices) or CMOSs ​​(Complementary Metal-Oxide-Semiconductors).

[0055] The front photographing cameras 31LF, 31RF and the rear photographing camera 31R photograph the front or rear of the vehicle M to generate image data, and provide the generated image data to the driving assistance control device 100.

[0056] The front photographing cameras 31LF, 31RF are configured as stereo cameras including a pair of left and right cameras, and the rear photographing camera 31R is configured as a so-called monocular camera, but each may be either a stereo camera or a monocular camera.

[0057] Furthermore, instead of or in addition to the front photographing cameras 31LF, 31RF and the rear photographing camera 31R, for example, cameras mounted on the side mirrors 11L, 11R to photograph the left rear or right rear may be provided.

[0058] (Ambient Environment Sensor) The ambient environment sensor 32 is a sensor that detects people or obstacles around the vehicle M. For example, the ambient environment sensor 32 has one or more sensors selected from the group consisting of a high-frequency radar sensor, an ultrasonic sensor, and a LiDAR.

[0059] In particular, the surrounding environment sensor 32 has the function of detecting any objects present around the vehicle M, such as other vehicles or bicycles, buildings, utility poles, traffic signs, traffic lights, natural objects, or other obstacles.

[0060] The surrounding environment sensor 32 then transmits a sensor signal including the detected data to the driving assistance control device 100 .

[0061] (Vehicle Operation / Behavior Sensor) The vehicle operation / behavior sensor 35 is composed of at least one sensor that detects the operation state and behavior of the vehicle M.

[0062] For example, the vehicle operation / behavior sensor 35 has at least one of a vehicle speed sensor, an acceleration sensor, and an angular velocity sensor, and detects information on the behavior of the vehicle M, such as vehicle speed, longitudinal acceleration, lateral acceleration, and yaw rate.

[0063] Furthermore, for example, the vehicle operation / behavior sensor 35 includes at least one of an accelerator position sensor, a brake stroke sensor, a brake pressure sensor, a steering angle sensor, an engine rotation speed sensor, a brake lamp switch, and a blinker switch.

[0064] The vehicle operation / behavior sensor 35 detects information on the vehicle operation state, such as the steering angle of the steering wheel or steered wheels, accelerator opening, brake operation amount, on / off of the brake lamp switch, and on / off of the blinker switch.

[0065] Furthermore, the vehicle operation / behavior sensor 35 provides the detected behavior, speed, steering wheel position, and other information of the vehicle M to the driving assistance control device 100.

[0066] (GNSS Antenna) The GNSS antenna 37 receives satellite signals from satellites such as GPS (Global Positioning System) satellites, and transmits position information of the vehicle M on map data contained in the received satellite signals to the driving assistance control device 100.

[0067] In addition, instead of the GNSS antenna 37, an antenna that receives satellite signals from other satellite systems that identify the position of the vehicle M may be provided.

[0068] (Map Data Storage Unit) The map data storage unit 33 is a storage medium that is configured from a storage device such as a memory element, a magnetic disk, an optical disk, or a flash memory, and stores map data.

[0069] For example, a RAM (Random Access Memory) or a ROM (Read Only Memory) is used as a storage element. A HDD (Hard Disk Drive) is used as a magnetic disk. A CD (Compact Disc) or a DVD (Digital Versatile Disc) is used as an optical disk. A SSD (Solid State Drive) or a USB (Universal Serial Bus) memory is used as a flash memory.

[0070] Further, the map data storage unit 33 provides the corresponding map data (including data related to the map) to the navigation system 39 under the control of the navigation system 39 .

[0071] The map data storage unit 33 of this embodiment may be a storage medium that stores map data for guiding the vehicle M to a predetermined destination.

[0072] (Navigation System) Under the control of the driving assistance control device 100, the navigation system 39 works in conjunction with the map data storage unit 33 to identify the current position of the vehicle M based on the position information acquired from the GNSS antenna 37, and provides the identified current position to the HMI 43.

[0073] (HMI) The HMI 43 is driven by the driving assistance control device 100 and has a function of notifying the driver of various information by means of image display, audio output, etc., for example, a display device and a speaker (not shown) provided in the instrument panel.

[0074] In particular, the HMI 43 works in conjunction with the navigation system 39 to display various types of information (image information and sound information including map information) generated by the navigation system 39 for providing driving assistance to the driver as images or as audio output.

[0075] The display device may be a HUD (head-up display) that displays information on the front window superimposed on the scenery around the vehicle M.

[0076] (In-Vehicle Camera) The in-vehicle camera 41 includes an imaging element such as a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor).

[0077] The in-vehicle camera 41 captures an image of the driver's upper body, including the face, generates image data, and provides the generated image data to the driving assistance control device 100. The in-vehicle camera 41 is also configured to be able to detect the driver's line of sight.

[0078] (Seating Sensor) The seating sensor 42 is a sensor incorporated in the driver's seat, and is a sensor for detecting the seating state of the driver and estimating the driver's behavior and the driver's fatigue level.

[0079] Specifically, the seating sensor 42 is provided on each of the seat surface and the back surface of the driver's seat 60 and includes a sensor sheet having a plurality of measurement points.

[0080] The seating sensor 42 then provides the driving assistance control device 100 with pressure data relating to each measurement point.

[0081] (Vehicle Drive Control Unit) The vehicle drive control unit 40 has at least one control system that controls the drive of the vehicle M.

[0082] The vehicle drive control unit 40 has an engine control system or a motor control system that controls the driving force of the vehicle M, a steering wheel, an electric steering device 15 that controls the steering angle of the steering wheels, and a brake device 17 that controls the braking force of the vehicle M.

[0083] The vehicle drive control unit 40 may also have a transmission system that changes the speed of the output from the engine or drive motor and transmits it to the drive wheels.

[0084] [B2] Force Feedback Device [B2.1] Sheet-Type Force Feedback Device Next, the sheet-type force feedback device 50 of this embodiment will be described with reference to FIG. 2 together with FIG.

[0085] FIG. 2 is an example of a configuration diagram showing the configuration of the sheet-type force feedback device 50 of this embodiment.

[0086] The sheet-type force feedback device 50 is a device for guiding the direction of the driver's body under the control of the driving assistance control device 100.

[0087] The seat-type force feedback device 50 is a device provided in a driver's seat 60 that is equipped with a seat belt 61 that has a structure that allows it to be wound up by the seat belt winding device (not shown).

[0088] The sheet-type force-feedback presentation device 50 is a device that provides stimulation to a predetermined part of the driver.

[0089] In particular, the seat-type force feedback device 50 has a configuration for controlling the amount of stimulation to provide a force sensation to the driver's body through stimulation, and a configuration for performing posture control to make the driver's posture suitable for the driving conditions of the vehicle M.

[0090] The seat belt 61 is provided so that it can be worn by the driver, with one end fixed to an anchor (not shown) and the other end connected to a seat belt retractor (not shown) provided in the pillar or seat back so that it can be retracted and pulled out.

[0091] Specifically, the sheet-type force feedback device 50 includes a pressure actuator 51 having a back pressure actuator 52 and a belly pressure actuator 55 that are driven and controlled by the driving assistance control device 100 .

[0092] The back pressure actuator 52 is provided on the back of the seat, inside the seat trim of the back support portion where the driver's back abuts.

[0093] The rear pressure actuator 52 includes a left rear pressure drive portion 53 and a right rear pressure drive portion 54 at approximately the same height as the waist belt portion when the driver fastens the seat belt 61.

[0094] The left rear side pressing drive unit 53 and the right rear side pressing drive unit 54 are driven by a motor or by supplying and discharging fluid pressure, and when pressed, the rear of the driver's seat 60 protrudes forward of the vehicle and presses the left and right sides of the driver's back.

[0095] When the driver's seat is not pressed, the back of the driver's seat is kept flat and no pressure is applied to the left or right side of the driver's back.

[0096] The abdominal surface pressing actuator 55 is provided inside the waist belt portion of the seat belt 61 and includes a left abdominal surface side pressing drive portion 56 and a right abdominal surface side pressing drive portion 58 .

[0097] The left abdominal surface side pressure drive unit 56 and the right abdominal surface side pressure drive unit 58 are formed in the shape of a bag through which fluid pressure is supplied and discharged, or are displaced in the thickness direction of the waist belt portion by angularly displacing, for example, a cam-shaped object using a motor or the like.

[0098] When driven in a pressure-operated state, the seat protrudes toward the rear of the vehicle and presses against the left and right sides of the driver's abdominal surface, whereas when driven in a non-pressing state, the seat protrudes toward the rear of the vehicle and does not press against the left and right sides of the abdominal surface.

[0099] Furthermore, in the present embodiment, the sheet-type force feedback device 50 is used as the force feedback device for controlling the orientation of the driver's body, but a belt-type wearable device may also be used.

[0100] [B2.2] Wearable Force Feedback Device Next, the wearable force feedback device 70 of this embodiment will be described with reference to FIG. 1 and FIG.

[0101] FIG. 3 is an example of a configuration diagram showing the configuration of the wearable force feedback device 70 of this embodiment.

[0102] The wearable force feedback device 70 of this embodiment is a device that guides the head direction when worn by a driver under the control of the driving assistance control device 100.

[0103] As shown in FIG. 3, the wearable force feedback device 70 of this embodiment includes a stimulus generator 71 that induces the driver to move their head in a direction that rotates it left and right.

[0104] That is, the stimulus generating unit 71 is a device that uses different types of motion-induced illusion phenomena to induce movement in a predetermined direction in the driver.

[0105] The stimulus generating unit 71 is attached to the driver's head and has a function of guiding the head direction.

[0106] The stimulus generating unit 71 is a device (hanger device) that uses the hanger reflex to induce the driver to move their head in a direction that rotates it from side to side.

[0107] The stimulus generator 71 applies pressure and skin deformation to the driver's head as input stimuli.

[0108] Specifically, as shown in FIG. 3, the stimulus generating unit 71 has a wearable shape that can be worn on the head to generate pressure and skin deformation, and has a plurality of pneumatic actuators 72 attached at predetermined positions.

[0109] Each pneumatic actuator 72 is configured to expand when filled with air, which is a fluid, and to apply pressure to the corresponding location to slightly move the driver's skin toward the surface, thereby inducing a rotational movement of the wearing part.

[0110] In addition, the wearable force feedback device 70 may have a stimulus generating unit that induces movement of the head in a direction different from the left and right directions by utilizing visually induced motion illusion phenomena such as visually induced self-motion sensation (vection).

[0111] For example, in this case, the wearable force feedback device 70 induces movement in the direction of tilting the head left and right (rotational direction around the pitch axis) and up and down (rotational direction around the roll axis), which are directions different from the left and right direction of the head.

[0112] The wearable force-feedback presentation device 70 may also have a function of controlling the driver's line of sight by providing a visual stimulus.

[0113] Furthermore, in this embodiment, the wearable force feedback device 70 is described as being attached to the driver's head, but it may also be in the form of goggles like glasses, a wearable garment such as a vest (sleeveless jacket), or a belt.

[0114] [B3] Driving assistance control device Next, an example of the configuration of the driving assistance control device 100 for the vehicle M of this embodiment will be described with reference to Fig. 4. Fig. 4 is an example of a system configuration diagram showing the configuration of the driving assistance control device 100 mounted on the vehicle M of this embodiment.

[0115] The driving assistance control device 100 functions as a device that assists the driving of the vehicle M by having one or more processors such as CPUs execute computer programs, and is particularly configured to perform processing related to parking assistance in parking areas.

[0116] In addition, the driving assistance control device 100 of this embodiment is not limited to a device installed in the vehicle M, but may also be an information terminal device such as a smartphone or wearable device that can be linked to the vehicle M and communicate with a management server not shown.

[0117] In addition, the driving assistance control device 100 is connected to an exterior camera 31, a surrounding environment sensor 32, and a vehicle operation / behavior sensor 35 either directly or via a communication means such as a CAN (Controller Area Network) or a LIN (Local Internet).

[0118] The driving assistance control device 100 is connected to a map data storage unit 33, a GNSS (Global Navigation Satellite System) antenna 37, and a navigation system 39 via communication means.

[0119] Furthermore, the driving assistance control device 100 is connected to a vehicle drive control unit 40, an in-vehicle camera 41, an HMI (Human Machine Interface) 43, a seat-type force feedback device 50, and a wearable force feedback device 70 via communication means.

[0120] In the vehicle M, the vehicle operation / behavior sensor 35, the GNSS antenna 37, the map data storage unit 33, the HMI 43, and the vehicle drive control unit 40 are each directly connected to the driving assistance control device 100. However, these may also be indirectly connected to the driving assistance control device 100 via communication means such as CAN or LIN.

[0121] 4, the driving assistance control device 100 includes a processing unit 110, an operation input unit 120, a storage unit 140, a communication unit 170, and an information storage medium 180. However, the driving assistance control device 100 may be configured to omit some of these components.

[0122] Furthermore, part or all of the driving assistance control device 100 may be configured as updatable firmware or the like, or may be a program module or the like that is executed by instructions from a CPU or the like.

[0123] The processing unit 110 drives hardware such as various processors (CPU, DSP, etc.) based on the application stored in the information storage medium 180 to execute each process of this embodiment.

[0124] It should be noted that any type of application may be stored in the information storage medium 180. Furthermore, the processing unit 110 of this embodiment may read out programs and data stored in the information storage medium 180, temporarily store the read out programs and data in the storage unit 140, and perform various processes based on the programs and data.

[0125] The processing unit 110 also executes various processes using a main memory unit 141 in the storage unit 140 as a work area, or realizes these processes using application programs.

[0126] Specifically, the processing unit 110 is composed of a communication control unit 111, an operation reception processing unit 112, a driving scene detection processing unit 113, a driver state detection processing unit 114, a driver posture control unit 115, and a driver posture control information management unit 116. However, the processing unit 110 may be configured to omit some of these units.

[0127] The communication control unit 111 controls the communication unit 170, establishes a communication line with a server device (not shown) via a network N such as a mobile communication network or the Internet, and transmits and receives data (predetermined information digitized).

[0128] The operation reception processing unit 112 recognizes input information input by the driver from the operation input unit 120 , and outputs the recognized information to the driver attitude control unit 115 .

[0129] The driving scene detection processing unit 113 executes a driving scene detection process to detect the current driving scene of the vehicle M or a driving scene predicted to occur in the future for the vehicle M.

[0130] The driver state detection processing unit 114 executes a driver state detection process to detect the state of the driver while driving the vehicle M (for example, the direction of the head, the direction of the line of sight, and the direction the body is facing).

[0131] The driver posture control unit 115 repeatedly controls the output of the stimulus amount in the seat-type force feedback device 50 and the wearable force feedback device 70 based on the detected driving scene and the driver's state as a stimulus output control process.

[0132] That is, when the driver posture control unit 115 detects a driving scene, it repeats the driver state detection process and the stimulus output control process to perform feedback control on the driver state.

[0133] The driver posture control information management unit 116 executes a registration process to register the amount of stimulation of the finally adjusted sheet-type force feedback device 50 and wearable force feedback device 70, the duration thereof, and the driver's state before and after the stimulation output in correspondence with the driving scene and the driver.

[0134] The operation input unit 120 is a device used when the driver inputs given information, and is configured to output the information input by the driver (hereinafter referred to as "input information") to the processing unit 110.

[0135] For example, the operation input unit 120 is configured with a lever, a button, a dial-type operation device, a microphone, a touch panel display, a keyboard, a camera for receiving input by a mouse or gestures, and the like.

[0136] The operation input unit 120 also includes a detection unit (not shown) that detects input information (input signal) from the driver.

[0137] In particular, when the operation input unit 120 is configured with a microphone, the detection unit constitutes a voice recognition device that receives input from the voice of a passenger such as a driver. Also, when the operation input unit 120 is configured with a camera, the detection unit constitutes an image recognition device that receives input of a gesture captured by the camera.

[0138] The storage unit 140 serves as a work area for the processing unit 110 and the like, and its functions are realized by hardware such as RAM (VRAM).

[0139] The storage unit 140 of this embodiment includes a main storage unit 141 used as a work area, a data storage unit 142, and a driver attitude control information storage unit 143. However, the storage unit 140 may be configured to omit some of these units.

[0140] In particular, the data storage unit 142 stores computer programs used to execute each process, table data, and the like.

[0141] In addition, the driver posture control information storage unit 143 stores driver posture control information for each driver and each driving scene for the vehicle M, for controlling the posture of the corresponding driver using the seat-type force feedback device 50.

[0142] In particular, the driver posture control information specifies, for each driver and each driving scene, the finally adjusted stimulation location, stimulation amount, and stimulation period, as well as the driver's state before and after the stimulation output, when force feedback is presented to the driver at a given timing.

[0143] Specifically, the driver posture control information storage unit 143 stores driver posture control information that specifies, for each driver and for each driving scene, information (prediction formula) that indicates the relationship between the amount of stimulation at each stimulation point and the respective reaction amount.

[0144] For example, the driver posture control information is determined by a regression equation (regression model) as predictive information showing the relationship between the amount of stimulation at each stimulation point and the respective reaction amount (amount of head or body rotation) for each driver and each driving scene.

[0145] When the driver posture control information is used in the stimulus output control process and registered again in the driver posture control information storage unit 143, it may be newly generated based on the newly determined stimulus amount at each stimulus location and the respective response amount, or it may be updated using them.

[0146] In addition, when new driver attitude control information is generated, each time it is newly registered, data updating is performed to leave the previously registered driver attitude control information as history information.

[0147] In addition, the driver attitude control information may be a prediction formula that shows the relationship between the amount of stimulation at multiple stimulation points and the respective reaction amounts, or may include the amount of stimulation and the stimulation period for each stimulation point (hereinafter referred to as the ``stimulation period'').

[0148] On the other hand, the driver posture control information storage unit 143 may store driver posture control information that specifies the amount of stimulation for each stimulation location, or the amount of stimulation and stimulation period, for each driver and for each driver's state (gaze, head, body, etc.).

[0149] For example, in this case, when changing the driver's body from "forward" to "right front," the driver posture control information specifies "body: forward" before the stimulation output, "body: right front" after the stimulation output, the stimulation location "pressure drive on the back and right side of the abdomen," and "5 out of 10."

[0150] The driver attitude control information may be defined for each attribute of the driver and for each state of the driver (such as line of sight, head and body).

[0151] The communication unit 170 performs various controls for communication with the outside (for example, a server device not shown).

[0152] The functions of the communication unit 170 are realized by hardware such as various processors or communication ASICs, programs, and the like.

[0153] The communication unit 170 of this embodiment communicates with a server device (not shown) when acquiring driver attitude control information stored in the server device (not shown).

[0154] The information storage medium 180 is readable by a computer, and may store various data including various apps, an OS (operating system), and an ID corresponding to each driving assistance control device 100.

[0155] That is, the information storage medium 180 stores applications for causing a computer to function as each part of this embodiment (applications for causing a computer to execute the processing of each part), an ID for communicating with a server device not shown, and the like.

[0156] For example, the information storage medium 180 is configured by a memory element or a storage device such as a magnetic disk, an optical disk, or a flash memory.

[0157] [B4] Driving Assistance Control Processing of the Present Embodiment [B4.1] Overview Next, the driving assistance control processing including feedback control of the stimulus output control processing executed in the driving assistance control device 100 of the present embodiment will be described with reference to FIG.

[0158] FIG. 5 is a diagram for explaining the driving assistance control process including the driving condition setting process of this embodiment.

[0159] The driving assistance control device 100 of this embodiment is configured to accurately guide the driver's posture to an appropriate posture by presenting force sensations appropriate for each driver, and to execute driving assistance control processing to assist the driver in driving and improve their skills.

[0160] For example, if the direction of travel of the vehicle M does not match the driver's state (i.e., posture), there is a high possibility that the accumulation of fatigue such as "car sickness" will be accelerated.

[0161] Also, for example, when vehicle M makes a "right turn" at an intersection, vehicle M must visually confirm the direction in order to avoid an accident with a pedestrian crossing the crosswalk to the right of the direction of travel of vehicle M at the intersection.

[0162] Therefore, the driving assistance control device 100 of this embodiment has a configuration for guiding the driver to an appropriate state when driving the vehicle M based on the driving scene and the state of the driver.

[0163] That is, as shown in FIG. 5, the driving assistance control device 100 of this embodiment has a configuration for guiding the driver to an appropriate state (head direction, line of sight direction, and body orientation) according to the driving scene of the vehicle M and the state of the driver driving the vehicle M.

[0164] Specifically, the driving assistance control device 100 is configured to execute a driving scene detection process to detect the current driving scene in the vehicle M or a driving scene predicted to be generated in the future in the vehicle M ([1] in Figure 5).

[0165] The driving assistance control device 100 also has a configuration for executing a driver state detection process for detecting the driver state ([2] in FIG. 5).

[0166] Furthermore, the driving assistance control device 100 is configured to repeatedly execute output control for stimulating the force feedback devices 50 and 60 as a stimulus output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state ([3a] and [3b] in Figure 5).

[0167] In particular, the driving assistance control device 100 has a configuration in which, when a driving scene is detected, the driving assistance control device 100 repeats a driver state detection process and a stimulus output control process to perform feedback control on the driver state.

[0168] The driving assistance control device 100 is configured to execute a registration process that registers the final adjusted stimulus amount and the driver's state before and after the stimulus output in association with the driving scene and the driver ([4] in Figure 5).

[0169] FIG. 5 shows an example of a driving assistance control process for guiding the driver to an ideal state when the driving scene involves turning right.

[0170] FIG. 5 also shows an example of guiding the driver's state to an ideal state when the driver's state is not appropriate in terms of the head direction (right front), line of sight direction (right front), and body orientation (forward) based on the area to be checked (the crosswalk on the right side of the page).

[0171] In particular, FIG. 5 shows an example in which, when the driver is in a state where the crosswalk cannot be seen, the driver's state is guided to an ideal state so that the crosswalk can be seen, and an example in which the stimulus output control process and the driver state detection process are repeatedly executed.

[0172] With this configuration, the driving assistance control device 100 of this embodiment can compare the amount of stimulation with the actual driver's state and appropriately change the amount of stimulation, thereby continuously guiding the driver to an appropriate driving posture.

[0173] Furthermore, the driving assistance control device 100 of this embodiment can appropriately change the amount of stimulation to control the driver's driving posture, even if the driver becomes accustomed to the stimulation, or even if a driver with different driving characteristics or a driver with a different physique is driving.

[0174] Therefore, the driving assistance control device 100 of this embodiment can guide each driver to an appropriate driving posture, and therefore not only can it suppress car sickness for many drivers, but it can also provide driving assistance and improve driving skills, leading to safer driving.

[0175] [B4.2] Driving Scene Detection Processing Next, the driving scene detection processing executed in the driving assistance control device 100 of this embodiment will be described.

[0176] The driving scene detection processing unit 113 performs driving scene detection processing to detect the current driving scene in vehicle M (hereinafter referred to as the "current driving scene") or a driving scene predicted to occur in the vehicle in the future (hereinafter referred to as the "predicted driving scene").

[0177] In particular, the driving scene detection processing unit 113 performs a vehicle state detection process to detect the state of the vehicle M, and detects the current driving scene and the predicted driving scene based on the detected state of the vehicle M (hereinafter referred to as the "vehicle state").

[0178] In other words, the driving scene detection processing unit 113 has a configuration that can accurately identify the driving scene based on the state of vehicle equipment such as the steering wheel and turn indicators (blinkers), as well as the vehicle state such as the driving state and operating state.

[0179] In addition, the driving scene detection processing unit 113 may detect the current driving scene or the predicted driving scene in conjunction with at least one of the vehicle equipment, the exterior camera 31, the surrounding environment sensor 32, the vehicle operation / behavior sensor 35, and the navigation system 39.

[0180] For example, the driving scene detection processing unit 113 detects as the current driving scene a scene in which a series of behaviors of the vehicle M occurring during the driver's driving actions, such as turning right or left, parking or stopping, changing lanes, or merging.

[0181] Furthermore, for example, the driving scene detection processing unit 113 detects, as predicted driving scenes, scenes in which a series of behaviors of the vehicle M associated with the driver's driving actions, such as turning right or left, parking or stopping, changing lanes, or merging, may occur in the future.

[0182] The driving scene detection processing unit 113 may detect the current driving scene and the predicted driving scene including the location where the vehicle movement occurred, such as an intersection, a merging point, or a highway.

[0183] Specifically, the driving scene detection processing unit 113 identifies the driving environment around the vehicle M, such as the road shape (straight or curved), the presence or location of traffic lights and intersections, the presence or absence of pedestrians, and the number and lane on which the vehicle is traveling, based on information from the exterior camera 31, the surrounding environment sensor 32, or both.

[0184] In addition, the driving scene detection processing unit 113 identifies the driving status of the vehicle equipment, such as the position of the steering wheel, whether the left turn indicator or the right turn indicator is on or off, the vehicle speed, whether the brakes are operating, and whether the wipers are on or off, based on information from the vehicle operation / behavior sensor 35.

[0185] Furthermore, the driving scene detection processing unit 113 identifies the behavior of the vehicle M, such as yaw, pitching, roll, bounce, or vibration, based on information from the vehicle operation / behavior sensor 35.

[0186] In addition to the above, the driving scene detection processing unit 113 identifies the surrounding conditions and the predicted route of the vehicle M based on navigation information such as the destination, route, and current position transmitted from the navigation system 39.

[0187] Then, the driving scene detection processing unit 113 identifies the current driving scene or the predicted driving scene based on the driving environment of the vehicle M identified as described above, the driving status of the vehicle M's vehicle equipment, the behavior of the vehicle M, the situation around the vehicle M and the predicted route, and a combination of these.

[0188] For example, when the driving scene detection processing unit 113 determines that the vehicle M is entering an intersection based on the driving environment, it determines that the vehicle M will turn right based on whether the turn signal is on or off and the vehicle's route.

[0189] At this time, the driving scene detection processing unit 113 also identifies the surrounding conditions, such as whether there are traffic lights and crosswalks at the intersection, and whether there are pedestrians crossing the crosswalk.

[0190] When detecting a driving scene, the driving scene detection processing unit 113 may detect the driving scene including areas that the driver should check, such as crosswalks.

[0191] [B4.3] Driver State Detection Process Next, the driver state detection process executed in the driving assistance control device 100 of this embodiment will be described.

[0192] The driver state detection processing unit 114 is configured to acquire image data from the in-vehicle camera 41 and execute a driver state detection process to detect the driver's state (hereinafter referred to as the "driver state"), such as the driver's posture and driver's movements.

[0193] In particular, the driver state detection processing unit 114 repeatedly executes the driver state detection process at given timings while the stimulus output control process is being executed.

[0194] Specifically, the driver state detection processing unit 114 acquires image data from the in-vehicle camera 41 as the driver's posture based on the image data, and detects the direction of the driver's head, changes in its position, and the direction in which the body is turning.

[0195] Furthermore, the driver state detection processing unit 114 detects the driver's behavior, such as steering operation, line of sight direction and movement, based on the image data.

[0196] Then, while the stimulus output control process is being executed, the driver state detection processing unit 114 repeatedly detects the driver state, such as the driver's posture and driver movements, based on image data acquired from the in-vehicle camera 41 at given timings.

[0197] On the other hand, the driver state detection processing unit 114 may be configured to execute a driver state detection process to detect the driver state, such as the driver's posture and driver behavior, based on data from the seating sensor 42.

[0198] In this case, the driver state detection processing unit 114 detects the driver's seat contact pressure distribution based on data from the seat sensor 42 (e.g., pressure data related to each measurement point), and detects the driver's posture and movement based on the detected seat contact pressure distribution.

[0199] In particular, the driver state detection processing unit 114 detects the turning direction of the driver's body as the driver's posture, and detects the operation status of the brake pedal, accelerator pedal, etc.

[0200] Then, the driver state detection processing unit 114 repeatedly detects the driver's posture and movement based on the seating contact pressure distribution at given timings while the stimulus output control processing is being executed.

[0201] [B4.4] Stimulus Output Control Processing Including Feedback Control Next, the stimulus output control processing including feedback control executed in the driving assistance control device 100 of this embodiment will be described with reference to FIGS. 6A and 6B.

[0202] 6A and 6B are diagrams for explaining the feedback control executed in the driving assistance control device 100 of this embodiment.

[0203] (Basic principle of stimulus output control processing) The driver posture control unit 115 adjusts the amount of stimulus based on the detected driving scene and driver state, and repeatedly executes output control for stimulating the seat-type force feedback device 50, etc., as a stimulus output control processing.

[0204] In particular, the driver posture control unit 115 determines whether the driver's state in the detected driving scene is an ideal state.

[0205] If the driver posture control unit 115 determines that the driver state is not ideal, it presents stimuli to the driver via the seat-type force feedback device 50 or the like to bring the driver into an appropriate driver state.

[0206] The driver posture control unit 115 then works in conjunction with the driving scene detection process to repeatedly execute condition judgment processing, stimulus amount determination processing, and stimulus control processing, and executes feedback control (guiding the driver to the ideal state) as stimulus output control processing until the driver's state reaches the ideal state.

[0207] Specifically, first, as a condition specification process, the driver posture control unit 115 specifies the conditions (hereinafter referred to as "ideal driver state conditions") for an ideal driver state (hereinafter referred to as "ideal driver state").

[0208] Next, as a condition determination process, the driver posture control unit 115 determines whether or not the detected driver state (hereinafter referred to as the "detected driver state") satisfies the ideal driver state conditions.

[0209] Then, as part of the stimulation amount determination process, if the driver posture control unit 115 determines that the detected driver state does not meet the ideal driver state conditions, it determines the areas to be stimulated for the driver (hereinafter referred to as ``stimulation areas'') and the amount of stimulation for each stimulation area.

[0210] Next, as a stimulus control process, the driver posture control unit 115 controls the sheet-type force sense presentation device 50, the wearable force sense presentation device 70, or both devices based on the determined stimulus locations and the amount of stimulus at each stimulus location.

[0211] Furthermore, the driver posture control unit 115 again executes the condition determination process to determine whether or not the detected driver state after execution of the stimulus control process satisfies the ideal driver state condition.

[0212] At this time, if the driver posture control unit 115 determines that the detected driver state meets the ideal driver state conditions, it terminates the stimulus output control process; if not, it executes the stimulus amount determination process and stimulus control process again.

[0213] The driver posture control unit 115 then repeatedly executes the condition determination process, the stimulus amount determination process, and the stimulus control process in conjunction with the driver state detection process until the detected driver state meets the ideal driver state conditions.

[0214] (Condition identification process) As a condition identification process, the driver posture control unit 115 identifies the conditions (hereinafter referred to as "ideal driver state conditions") for an ideal driver state (hereinafter referred to as "ideal driver state") based on the driving scene detected by the driving scene detection process.

[0215] Specifically, the driver posture control unit 115 executes a condition identification process to identify the ideal driver state conditions using table data predetermined in the data storage unit 142, in which the ideal driver state conditions are specified for each driving scene.

[0216] In particular, the driver posture control unit 115 uses table data that specifies the head direction, line of sight direction, and body turning direction that indicate each element of the driver's state for each driving scene.

[0217] For example, when the driving scene detected is "turning right at an intersection (with a crosswalk)," the driver posture control unit 115 identifies the head direction (right front), gaze direction (right front and right direction), and body direction (forward) when turning right as the ideal driver state conditions.

[0218] In addition, when the detected driving scene includes an area that the driver should check, such as a crosswalk, as shown in Figure 5, the driver posture control unit 115 may specify, as the ideal driver state condition, a driver state in which the area that the driver should check can be seen.

[0219] (Condition Determination Process) As the condition determination process, the driver posture control unit 115 determines whether or not the detected driver state detected by the driver state detection process satisfies the ideal driver state condition identified by the condition identification process.

[0220] That is, the driver posture control unit 115 has a configuration for executing a determination process for determining whether or not the detected driver state is an ideal state that is defined in advance based on the driving scene.

[0221] Then, the driver posture control unit 115 performs condition identification processing by using the driver's head direction, line of sight direction, and body turning direction as the detected driver state and conditions defined as the ideal driver state conditions (specifically, the ideal driver state).

[0222] Specifically, the driver posture control unit 115 determines whether the detected driver state (head direction, gaze direction, body orientation, etc.) detected by the driver state detection process matches the ideal driver state conditions (head direction, gaze direction, body orientation, etc.).

[0223] At this time, if the detected driver state matches the ideal driver state conditions, the driver posture control unit 115 determines that the detected driver state meets the ideal driver state conditions, and stops the stimulus output control process without controlling the seat-type force feedback device 50.

[0224] Furthermore, if the detected driver state does not match the ideal driver state condition, the driver posture control unit 115 starts execution of a stimulus amount determination process (i.e., feedback control).

[0225] For example, assume that a driving scene is detected as "turning right at an intersection (with a crosswalk)" and the ideal driver state is identified as the head direction (right front), gaze direction (right front and right direction), and body direction (forward) when turning right.

[0226] In this case, if the detected driver state includes the head direction (right front), gaze direction (right front or right direction), and body turning direction (forward), the driver posture control unit 115 determines that the ideal driver state conditions are met.

[0227] On the other hand, if the detected driver state includes the head direction (left front), gaze direction (left direction), and body direction (forward) when turning right, the driver posture control unit 115 determines that the ideal driver state conditions are not met.

[0228] (Stimulus amount determination process and stimulus control process) If the detected driver state does not meet the ideal driver state conditions, the driver posture control unit 115 performs a stimulus amount determination process for the target driver using the driver posture control information stored in the driver posture control information storage unit 143.

[0229] Specifically, the driver posture control unit 115 detects driver posture control information of the target driver from the driver posture control information storage unit 143, with the detected driver state being the driver state before stimulation and the ideal driver state condition being the driver state after stimulation.

[0230] When the driver posture control unit 115 detects the corresponding driver posture control information, it determines the stimulation location and the stimulation amount specified in the driver posture control information as the stimulation location and the stimulation amount to be stimulated for the driver.

[0231] In addition, when the driver posture control unit 115 determines the stimulation location and amount of stimulation to be given to the driver, it controls the sheet-type force feedback device 50, the wearable force feedback device 70, or both devices based on the determined stimulation location and amount of stimulation.

[0232] For example, suppose a driving scene is detected as "turning right at an intersection (with a crosswalk)" and it is determined that the ideal driver state conditions (head direction (right front), gaze direction (right front and right direction), and body direction (forward)) are not met.

[0233] In addition, assume that the detected driver state is head direction (left front), gaze direction (left direction), and body direction (forward), and the following information is specified as driver posture control information when changing the driver's head from "left front" to "right front."

[0234] (1) Before stimulation output: "Head: left front" and after stimulation output: "Head: right front" (2) Stimulation location: "Air pressure actuators on the right frontal region and the left occipital region" (3) Stimulation amount: "Right side: 5 out of 10 / left side: 2 out of 10"

[0235] In this case, the driver posture control unit 115 determines the amount of stimulation to the right front head and left occipital region of the wearable force feedback device 70 to be 5 out of 10 or 2 out of 10 in order to orient the head forward to the right.

[0236] Then, the driver posture control unit 115 controls the wearable force sense presentation device 70 according to the determined content.

[0237] On the other hand, the driver posture control unit 115 may determine the stimulation location and the stimulation amount for the driver based on the attributes of the driver.

[0238] In this case, the driver posture control unit 115 identifies the attributes of the driver in advance based on previously acquired information (physical information such as height or questionnaire results regarding personal characteristics).

[0239] Then, the driver posture control unit 115 searches for driver posture control information from the driver posture control information storage unit 143 based on the attributes of the driver who is the target of driving assistance, and based on the corresponding driver posture control information, determines the stimulation points to be stimulated for the driver and the amount of stimulation.

[0240] That is, the driver posture control unit 115 detects driver posture control information of a driver having the same attributes as the identified attributes from the driver posture control information storage unit 143, with the detected driver state being the driver state before stimulation and the ideal driver state condition being the driver state after stimulation.

[0241] Furthermore, when the driver posture control unit 115 detects the corresponding driver posture control information, it determines the stimulation location and stimulation amount specified in the driver posture control information as the stimulation location and stimulation amount for the driver.

[0242] The driver's attributes include, for example, factors related to the driver, such as gender, age, personality, physical information (height, bone structure, etc.), driving style, and sense of driving burden.

[0243] In addition, the driver posture control unit 115 may detect from the driver posture control information storage unit 143 not only the driver posture control information of a driver having the same attribute as the identified attribute, but also the driver posture control information of a driver having an attribute that is considered to be the same as the identified attribute.

[0244] That is, the driver attitude control unit 115 may detect driver attitude control information of not only the completely identical driver but also a similar driver.

[0245] For example, the driver posture control unit 115 may detect driver posture control information from the driver posture control information storage unit 143 that has attributes not only of the same age, but also of the same "generation," or of a standard age or an age within a certain range (such as ±5 years).

[0246] The physical information used as attributes may be information automatically collected in advance by an in-vehicle camera 41, or the driver's attributes may be identified based on the results of a questionnaire entered in advance by the driver via the operation input unit.

[0247] The questionnaire includes a questionnaire about personal attributes such as age and gender, as well as a questionnaire about driving style such as the Driving Style Questionnaire (DSQ) or a questionnaire about driving load sensitivity such as the Workload Sensitivity Questionnaire (WSQ).

[0248] In particular, the driving style questionnaire includes driving skills, driving tendencies, anxiety tendencies, the driver's attitude toward driving, preferences, and way of thinking, and the questionnaire regarding mental burdens felt in driving situations identifies various driving burdens felt by each driver, quantitatively evaluated.

[0249] (Feedback Control) The driver posture control unit 115 executes a condition determination process to again determine whether the detected driver state detected by the driver state detection process executed after executing the stimulus control process satisfies the ideal driver state conditions.

[0250] At this time, if the detected driver state meets the ideal driver state conditions, the driver posture control unit 115 terminates the stimulus output control process, and if the detected driver state does not meet the ideal driver state conditions, it executes the stimulus amount determination process again.

[0251] In other words, the driver posture control unit 115 is configured to adjust the amount of stimulation given to the driver by the seat-type force feedback device 50 or the like when the condition determination process determines that the driver's state is not ideal.

[0252] The driver posture control unit 115 is configured to repeatedly execute a stimulus output control process that controls the output of the sheet-type force feedback device 50, etc., based on the adjusted stimulus amount (hereinafter referred to as the "adjusted stimulus amount"), along with a driver state detection process and a condition determination process.

[0253] In particular, when executing the stimulus output control process, the driver posture control unit 115 adjusts the stimulus location, the stimulus amount, or both when presenting a force sense as the adjusted stimulus amount.

[0254] For example, assume that the driver posture control unit 115 controls the output of equal amounts of stimulation at stimulation points on the right front and left rear of the head of the wearable force feedback device 70 for the head in order to guide the direction of the driver's head to the right front, as shown in Figure 6A.

[0255] In this case, if it is determined again by the condition determination process that the ideal state is not met, the driver posture control unit 115 changes the output of the amount of stimulation to the wearable force sense presentation device 70 .

[0256] In other words, in this case, the driver posture control unit 115 performs output control to change the stimulation location on the left rear of the wearable force feedback device 70 to a location closer to the left rear, as shown in Figure 6B, and to reduce the amount of stimulation to half of the original amount.

[0257] On the other hand, when the driver posture control unit 115 executes the stimulus amount determination process again, it causes the driver state detection processing unit 114 to identify the difference in the amount of movement from the previous time (i.e., the amount of reaction) when detecting the driver state.

[0258] Then, the driver posture control unit 115 repeatedly executes the condition determination process, the stimulus amount determination process, and the stimulus control process in conjunction with the driver state detection process until the detected driver state finally meets the ideal driver state conditions.

[0259] [B4.5] Registration Processing Next, the registration processing executed in the driving assistance control device 100 of this embodiment will be described.

[0260] The driver posture control information management unit 116 is configured to execute a registration process that registers the final adjusted amount of stimulus, the amount of reaction when each stimulus control process is executed, and the driver state before and after stimulus output in correspondence with the driving scene and the driver.

[0261] That is, the driver posture control information management unit 116 registers the results of the feedback control in association with the driving scene and the driver in order to reflect the results in improving the accuracy of the driving assistance control processing, which guides the driver to an ideal state accompanied by force feedback presentation from the next time onwards.

[0262] Specifically, when it is determined that the detected driver state satisfies the ideal driver state conditions, the driver attitude control information management unit 116 identifies the stimulated location controlled during the feedback control.

[0263] In addition, the driver posture control information management unit 116 identifies the total amount of stimulation finally adjusted from the start of the feedback control at each of the identified stimulation locations, and the reaction amount of each part of the driver (head, gaze, and body) when each stimulation control process is executed.

[0264] Then, the driver attitude control information management unit 116 associates information on each stimulation location, the controlled stimulation amount, and the driver's reaction amount to each stimulation location with the driver, and registers this information as driver attitude control information.

[0265] In particular, when registering driver posture control information for each driver, the driver posture control information management unit 116 identifies information (prediction formula) that indicates the relationship between the amount of stimulation at each stimulation point and the respective reaction amount, and registers the identified information as new driver posture control information for each driver.

[0266] For example, the driver attitude control information management unit 116 identifies and registers a regression formula (regression model) as information on a prediction formula that indicates the relationship between the amount of stimulation at each stimulation location and the amount of each reaction.

[0267] Furthermore, the driver attitude control information management unit 116 may register newly registered driver attitude control information as the driver attitude control information of each driver, while leaving past driver attitude control information as history information.

[0268] In particular, the driver posture control information management unit 116 registers the first detected driver state during feedback control as the driver state before stimulus output, and the ideal driver state condition as the driver state after stimulus output in the driver posture control information storage unit 143.

[0269] At this time, the driver attitude control information management unit 116 may register the newly registered driver attitude control information as the driver attitude control information for each driver.

[0270] In addition, in order to improve the accuracy of feedback control, when registering driver posture control information, the driver posture control information management unit 116 may register information on the total stimulation amount and each reaction amount at each stimulation point as information indicating a relationship such as a regression equation (regression model).

[0271] Furthermore, the driver attitude control information management unit 116 may register newly registered driver attitude control information as the driver attitude control information of each driver, while leaving past driver attitude control information as history information.

[0272] In addition, the driver posture control information management unit 116 may register the newly registered driver posture control information in the driver posture control information storage unit 143 for each predetermined attribute, instead of for each driver, or in addition.

[0273] In addition, the driver posture control information management unit 116 may register the final adjusted amount of stimulus, the amount of response when each stimulus control process is executed, and the driver's state before and after stimulus output in association with different driving scenes.

[0274] For example, if feedback control is always achieved with a small amount of stimulation at a stimulation point in a certain direction, the driver attitude control information management unit 116 may determine the amount of stimulation based on other driving scenes.

[0275] [B5] Operation of this Embodiment Next, a driving assistance control process including feedback control of the stimulus output control process executed by the driving assistance control device 100 of this embodiment will be described with reference to FIG.

[0276] FIG. 7 is a flowchart showing the operation of the driving assistance control process including the feedback control of the stimulus output control process executed by the driving assistance control device 100 of this embodiment.

[0277] In this operation, it is assumed that the driver attitude control information for each driving scene and each driver has already been registered in the driver attitude control information storage unit 143 .

[0278] First, when the driving scene detection processing unit 113 detects a change in the predetermined driver's driving situation and the driving environment of the vehicle M (step S101), it executes a driving scene detection process to detect the driving scene of the vehicle M (step S102).

[0279] For example, the driving scene detection processing unit 113 detects changes in the driving situation or driving environment, such as when the navigation system 39 detects that a right or left turn will occur at the next intersection, or when the vehicle operation / behavior sensor 35 detects operation of the right turn indicator.

[0280] Next, the driver state detection processing unit 114 acquires image data from the in-vehicle camera 41 and executes a driver state detection process to detect the driver state, such as the driver's posture and driver behavior (step S103).

[0281] For example, the driver state detection processing unit 114 detects the direction of the driver's head, the direction of the driver's line of sight, and the orientation of the driver's body as the driver's posture.

[0282] Next, the driver posture control unit 115 uses table data or the like to identify ideal driver state conditions based on the driving scene detected by the driving scene detection process (step S104).

[0283] Next, the driver posture control unit 115 executes a condition determination process to determine whether or not the detected driver state satisfies the ideal driver state conditions (step S105).

[0284] That is, the driver posture control unit 115 determines whether or not the detected driver state (head direction, etc.) detected by the driver state detection process matches the ideal driver state condition (head direction, etc.).

[0285] At this time, if the driver posture control unit 115 determines that the detected driver state matches the ideal driver state conditions, it determines that the detected driver state meets the ideal driver state conditions and terminates this operation.

[0286] On the other hand, if the driver posture control unit 115 determines that the detected driver state does not match the ideal driver state condition, it starts the execution of the stimulus amount determination process and starts feedback control (step S106).

[0287] First, the driver posture control unit 115 searches the driver posture control information storage unit 143 for the driver whose detected driver state is the driver state before the stimulus and whose ideal driver state condition is the driver state after the stimulus (step S107).

[0288] Next, the driver posture control unit 115 executes a stimulation amount determination process to determine the stimulation location and stimulation amount specified in the corresponding driver posture control information as the stimulation location and stimulation amount for the driver (step S108).

[0289] Next, the driver posture control unit 115 controls the sheet-type haptic device 50, the wearable haptic device 70, or both, based on the determined stimulation location and stimulation amount (step S109).

[0290] Next, the driver state detection processing unit 114 again executes the driver state detection process to detect the driver state, such as the driver's posture and driver's movements (step S110).

[0291] Next, the driver posture control unit 115 executes the condition determination process again to determine whether the re-detected driver state satisfies the ideal driver state condition (step S111).

[0292] At this time, if the driver posture control unit 115 determines that the detected driver state does not match the ideal driver state condition, the process returns to step S108.

[0293] On the other hand, if the driver posture control unit 115 determines that the detected driver state matches the ideal driver state conditions, it terminates the feedback control, registers new driver posture control information such as the final adjusted amount of stimulation (step S112), and terminates this operation.

[0294] That is, during feedback control, the driver posture control information management unit 116 associates the final amount of stimulus and the amount of response when each stimulus control process is executed, as well as the driver state before and after stimulus output with the driving scene and the driver, and registers them as driver posture control information.

[0295] [B6] Modifications [B1.6.1] Modification 1 Next, a case where machine-learned model data is used for driver attitude control information in Modification 1 of this embodiment will be described.

[0296] The driver attitude control information as model data may be information generated based on machine learning such as a support vector machine or a neural network (including deep learning) that uses multiple pieces of driver attitude control information as training data.

[0297] In other words, in this case, the driver attitude control information may be formed as model data generated for each driver or each driver attribute, and for each driver state or each driving scene, regarding the stimulation location and stimulation amount (which may include the stimulation period).

[0298] It is preferable that the driver attitude control information is learned based on new information each time a force feedback is presented to the driver.

[0299] Specifically, the driver posture control information as model data uses as input information the stimulation location and stimulation amount (which may include stimulation time) provided by the previous force sensation presentation, the driver's posture after the force sensation presentation (the amount of change associated with a change in the direction of the head or body), and the difference from the ideal state.

[0300] When the above input information is input, the driver attitude control information outputs the stimulation location and the stimulation amount (which may include the stimulation time).

[0301] The driver attitude control information may be further learned based on the output of the new stimulation location and stimulation amount, and the input information input when these are output.

[0302] In addition, the driver attitude control information may be learned based on the same driving scene, or based on the same driver or attributes of the same driver, or based on related driving scenes or different driving scenes, or attributes of related drivers.

[0303] For example, in this case, if weakening the amount of stimulation provided by the force sense in a driving scene when turning right can lead to the ideal state sooner, it can be weakened in the same way in a driving scene when turning left.

[0304] [B1.6.2] Modification 2 Next, a case where driver attitude control information is provided from a server device (not shown) in Modification 2 of this embodiment will be described.

[0305] The driver posture control unit 115 of this modified example may be connected to a server device (not shown) via the communication unit 170 under the control of the communication control unit 111, and may acquire driver posture control information from the server device.

[0306] In this case, the driver posture control information management unit 116 may also register the finally adjusted amount of stimulation, the amount of reaction when each stimulation control process is executed, and the driver state before and after stimulation output in a server device (not shown) in association with the driving scene and the driver.

[0307] [B1.6.3] Modification 3 Next, a case where the driver state is guided in consideration of the driver's fatigue level in Modification 2 of this embodiment will be described.

[0308] In this case, a seating sensor (not shown) is provided on the driver's seat 60, and the driver posture control unit 115 detects the seating contact pressure distribution of the driver, and detects the driver's fatigue level as the driver state based on the detected seating contact pressure distribution.

[0309] Then, the driver posture control unit 115 uses the detected driver state including the degree of fatigue in the same manner as described above to guide the driver state to the ideal state.

[0310] For example, in this case, when executing the stimulus output control process, the driver posture control unit 115 can change the amount of stimulus according to the level of fatigue, such as increasing the amount of stimulus more than usual if the level of fatigue is higher than usual.

[0311] [C] Others The embodiments of the present disclosure are not limited to those described in the above embodiments, and various modifications are possible. For example, terms cited in the description of the specification or drawings as broadly defined or synonymous can be replaced with broadly defined or synonymous terms in other descriptions of the specification or drawings.

[0312] The embodiments of the present disclosure include configurations that are substantially the same as the configurations described in the above embodiments (for example, configurations with the same function, method, and result, or configurations with the same purpose and effect). The embodiments of the present disclosure also include configurations in which non-essential parts of the configurations described in the above embodiments are replaced. The embodiments of the present disclosure also include configurations that achieve the same effects or purposes as the configurations described in the above embodiments. The embodiments of the present disclosure also include configurations in which publicly known technology is added to the configurations described in the above embodiments.

[0313] Although the embodiments of the present disclosure have been described in detail as above, it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel features and effects of the present invention. Therefore, all such modifications are intended to be included within the scope of the embodiments of the present disclosure.

[0314] M: Vehicle 13: Steering wheel 15: Electric steering device 31: Exterior camera 32: Surrounding environment sensor 33: Map data storage unit 35: Vehicle operation / behavior sensor 37: GNSS antenna 39: Navigation system 40: Vehicle drive control unit 41: In-vehicle camera 42: Seating sensor 43: HMI 50: Seat-type force feedback device 51: Pressure actuator 52: Back pressure actuator 53: Left back side pressure drive unit 54: Right back side pressure drive unit 55: Ventral pressure actuator 56: Left ventral side pressure drive unit 57: Left ventral side pressure drive unit 60: Driver's seat 61: Seat belt 61a: Waist belt unit 70: Wearable force feedback device 71: Stimulus generation unit 72 : Pneumatic actuator 100 : Driving assistance control device 110 : Processing unit 111 : Communication control unit 112 : Operation reception processing unit 113 : Driving scene detection processing unit 114 : Driver state detection processing unit 115 : Driver posture control unit 116 : Driver posture control information management unit 120 : Operation input unit 140 : Storage unit 141 : Main storage unit 142 : Data storage unit 143 : Driver posture control information storage unit 170 : Communication unit 180 : Information storage medium 307 : Communication unit 360 : Driver seat

Claims

1. A driving assistance system that controls a force sense presentation device to control the amount of stimulation to provide a force sense through stimulation to the driver's body and assists the driver in driving a vehicle, the driving assistance system comprising: one or more processors; and one or more memories communicatively connected to the one or more processors, wherein the one or more processors: execute a driving scene detection process to detect a current driving scene in the vehicle or a driving scene predicted to be future in the vehicle; execute a driver state detection process to detect the state of the driver; repeatedly execute output control for the stimulation of the force sense presentation device as a stimulation output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state; and execute a registration process to register in memory the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process, in correspondence with the driving scene and the driver.

2. A driving assistance system as described in claim 1, which executes a judgment process to judge whether the detected driver's state is an ideal state defined in advance based on the driving scene, and if the judgment process determines that the driver's state is not ideal, adjusts the amount of stimulation provided to the driver by the force feedback device, and controls the output of the force feedback device based on the adjusted amount of stimulation, and repeatedly executes the stimulus output control process together with the driver state detection process and the judgment process.

3. A driving assistance system as described in claim 2, wherein when the ideal state is stored in the memory in association with the driving scene for each driver or for each attribute of the driver, if the processor determines that the driving scene is not ideal by the judgment process, it executes the stimulus output control process based on the ideal state of the driver to be assisted or the ideal state stored in association with the attributes of the driver, and the detected driving scene.

4. The driving assistance system according to claim 1, wherein the processor executes a vehicle state detection process to detect a state of the vehicle, and executes the driving scene detection process based on the state of the vehicle.

5. A vehicle equipped with a driving assistance system that controls a force sense presentation device to control the amount of stimulation to provide a force sense through stimulation to the driver's body and assists the driver in driving the vehicle, wherein the driving assistance system executes a driving scene detection process to detect a current driving scene in the vehicle or a driving scene predicted to be future in the vehicle, executes a driver state detection process to detect the state of the driver, and repeatedly executes output control of the force sense presentation device for the stimulation while adjusting the amount of stimulation based on the detected driving scene and the driver's state as a stimulation output control process, and executes a registration process to register in memory the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process, in correspondence with the driving scene and the driver.

6. A recording medium having recorded thereon a computer program for a driving assistance system that controls a force sense presentation device for controlling the amount of stimulation to provide a force sense through stimulation to the driver's body and assists the driver in driving a vehicle, the recording medium having recorded thereon a computer program that causes a computer to: execute a driving scene detection process to detect a current driving scene in the vehicle or a driving scene predicted to be future in the vehicle; execute a driver state detection process to detect the state of the driver; repeatedly execute output control for the stimulation of the force sense presentation device as a stimulation output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state; and execute a registration process to register in memory the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process, and the state of the driver before and after the stimulation output, in correspondence with the driving scene and the driver.

7. A driving assistance method for controlling a force sense presentation device to control the amount of stimulation to provide a force sense through stimulation to the driver's body and assisting the driver in driving a vehicle, comprising: executing a driving scene detection process to detect a current driving scene in the vehicle or a driving scene predicted to be future in the vehicle; executing a driver state detection process to detect the state of the driver; repeatedly executing output control for the stimulation of the force sense presentation device as a stimulation output control process while adjusting the amount of stimulation based on the detected driving scene and the driver's state; and executing a registration process to register in memory the amount of stimulation finally adjusted by repeatedly executing the driver state detection process and the stimulation output control process, and the state of the driver before and after the stimulation output, in correspondence with the driving scene and the driver.

Images