Autopilot assistance device

Abstract

The present invention provides an automatic driving assistance device that can specifically and sequentially calculate the automatic driving control parameters of a vehicle for making the emotions of occupants such as drivers and / or passengers approach an ideal state. The automatic driving assistance device includes: an occupant emotion learning unit that constructs an occupant emotion model for estimating the emotions of occupants from the driving state of the vehicle based on information on the driving state of the vehicle and the emotions of the occupants; and a control parameter setting unit that calculates an ideal driving state of the vehicle in which the emotions of the occupants approach the target emotions based on the occupant emotion model, and sets control parameters for automatic driving based on the ideal driving state. The control parameter setting unit inputs a plurality of input values related to the driving state of the vehicle into the occupant emotion model respectively, and sets the control parameters by taking, as the ideal driving state of the vehicle, the input value in which the emotions of the occupants are closer to the target emotions than the current emotions among the input values input into the occupant emotion model.

Description

Technical Field

[0001] The present invention relates to an automatic driving assistance device. Background Art

[0002] In an autonomous vehicle, the following technology has been proposed: by reflecting the driving characteristics of occupants such as the driver and / or passengers in the vehicle driving control, negative emotions such as uneasiness and / or discomfort are suppressed. For example, Patent Document 1 discloses a vehicle driving assistance system including: a personal server that learns a personal driver model unique to the driver based on the driving data of the driver; and an in-vehicle controller provided in the vehicle of the driver and performing predetermined vehicle control processing. The personal server includes a recommendation engine that instructs the in-vehicle controller to perform a recommendation process, and the recommendation engine analyzes the current emotional state of the driver based on the voice data of the driver included in the driving data, and determines the recommendation process according to the analyzed emotional state based on the personal driver model.

[0003] In addition, Patent Document 2 discloses an electronic control device that uses biological information obtained from a biological sensor that measures the biological information of the driver or a passenger in the vehicle to estimate the emotion of the driver or the passenger, and controls the driving of the vehicle based on the estimated emotion.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2018-169704

[0007] Patent Document 2: Japanese Unexamined Patent Application Publication No. 2017-136922 Summary of the Invention

[0008] Technical Problem

[0009] However, although Patent Document 1 describes a case where the recommendation engine derives an appropriate recommendation process and selects and outputs an appropriate recommendation signal from a cabin space recommendation signal, a driving route recommendation signal, and an information prompt recommendation signal according to the situation, the step of specifically calculating the control parameter is not described. In Patent Document 2, the step of specifically calculating the parameter for controlling the driving of the vehicle based on the estimated emotion is also not described. In order to perform the automatic driving control of the vehicle, it is necessary to sequentially calculate the control parameters, and a method for specifically and sequentially calculating the control parameters is required.

[0010] The present invention has been made in view of the above problems, and an object of the present invention is to provide an automatic driving assistance device that can specifically and sequentially calculate the automatic driving control parameters of a vehicle for making the emotions of occupants such as the driver and / or passengers approach an ideal state.

[0011] Technical solution

[0012] To solve the above problems, according to one aspect of the present invention, there is provided an autonomous driving assistance device, comprising: an occupant emotion learning unit that constructs an occupant emotion model for inferring the emotion of an occupant from the driving state of the vehicle based on information on the driving state of the vehicle and the emotion of the occupant; and a control parameter setting unit that calculates an ideal driving state of the vehicle in which the emotion of the occupant approaches a target emotion based on the occupant emotion model, and sets control parameters for autonomous driving control based on the ideal driving state. The control parameter setting unit inputs a plurality of input values related to the driving state of the vehicle into the occupant emotion model respectively, and sets the control parameters by taking, as the ideal driving state of the vehicle, the input value in which the emotion of the occupant is closer to the target emotion than the current emotion among the input values input into the occupant emotion model.

[0013] In addition, the occupant emotion model may be an occupant emotion model that infers the emotion of the occupant based on information on the surrounding environment of the vehicle and information on the driving state of the vehicle, and the control parameter setting unit may calculate the ideal driving state of the vehicle in the surrounding environment corresponding to the current surrounding environment of the vehicle and set the control parameters.

[0014] In addition, the occupant emotion learning unit may store a data set associating information on the inferred emotion of the occupant, information on the surrounding environment of the vehicle, and information on the driving state of the vehicle. The control parameter setting unit may extract, from the stored data set, the driving state of the vehicle that is close to the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle, and generate a plurality of input values based on the extracted information on the driving state of the vehicle.

[0015] In addition, the control parameter setting unit may set a plurality of input values between the values of a predetermined data item of the extracted driving state of the vehicle and the values of a predetermined data item of the current driving state of the vehicle.

[0016] In addition, the predetermined data item of the driving state of the vehicle may include a plurality of data items, and at least one of the plurality of data items can be set as a priority item by the user. When there is a priority item set by the user, the control parameter setting unit may fix the value of the priority item among the data items of the plurality of input values input into the occupant emotion model as the value extracted as the driving state of the vehicle that is close to the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle, and generate a plurality of input values for the other data items.

[0017] In addition, when the emotion of the occupant during the execution of the automatic driving control with the control parameter obtained by fixing the value of the fixed priority item deteriorates compared to the emotion of the occupant calculated using the occupant emotion model, the control parameter setting unit may prompt the user to fix at least one data item that has a great impact on the emotion of the occupant as a value extracted as the driving state of the vehicle approaching the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle.

[0018] In addition, the predetermined data items of the driving state of the vehicle may include multiple data items. The occupant emotion learning unit may extract at least one data item that has a great impact on the emotion of the occupant based on the stored data set. The control parameter setting unit may fix at least one data item that has a great impact on the emotion of the occupant as a value extracted as the driving state of the vehicle approaching the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle, thereby obtaining multiple input values, and use the multiple input values to set the control parameter.

[0019] In addition, the control parameter setting unit may set the number of multiple input values input to the occupant emotion model based on at least one of the processing speed of the arithmetic processing device performing the operation and the update frequency of the control parameter.

[0020] Technical Effects

[0021] As described above, according to the present invention, it is possible to specifically calculate the automatic driving control parameters of the vehicle for making the emotions of occupants such as the driver and / or passengers approach an ideal state, and it is possible to implement the automatic driving control that can make the emotions of the occupants approach an ideal state. Brief Description of the Drawings

[0022] Figure 1 is a block diagram showing a configuration example of an automatic driving assistance device according to an embodiment of the present invention.

[0023] Figure 2 is an explanatory diagram showing an example of an occupant emotion model.

[0024] Figure 3 is an explanatory diagram showing an algorithm for setting the control parameter of the vehicle using the occupant emotion model.

[0025] Figure 4 is a flowchart showing an operation example of the automatic driving assistance device according to this embodiment.

[0026] Figure 5 is a flowchart showing the occupant emotion model learning process performed by the automatic driving assistance device according to this embodiment.

[0027] Figure 6It is a flowchart showing the driving control process based on the occupant's emotion by the autonomous driving assistance device of this embodiment.

[0028] Symbol Explanation

[0029] 10: Autonomous driving assistance device

[0030] 35: Vehicle driving control device

[0031] 50: Electronic control device

[0032] 51: Driving mode setting unit

[0033] 53: Surrounding environment determination unit

[0034] 55: Occupant emotion estimation unit

[0035] 57: Occupant emotion learning unit

[0036] 59: Emotion deterioration determination unit

[0037] 61: Driving emotion database

[0038] 63: Occupant emotion model

[0039] 65: Control parameter setting unit Detailed Embodiment

[0040] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that in this specification and the drawings, for components having substantially the same functional configuration, repeated descriptions are omitted by assigning the same reference numerals.

[0041] <1. Configuration Example of Autonomous Driving Assistance Device>

[0042] First, a configuration example of the autonomous driving assistance device according to the embodiment of the present invention will be described. Figure 1 It is a block diagram showing a configuration example of the autonomous driving assistance device 10 of this embodiment.

[0043] The autonomous driving assistance device 10 is configured to be mounted on a vehicle, detect information on the vehicle's occupants, the vehicle's driving state, and the vehicle's surrounding environment, and use the detected various information to execute control for assisting vehicle driving. The autonomous driving assistance device 10 includes: an occupant detection unit 41, a vehicle information detection unit 43, an input unit 45, a surrounding environment detection unit 47, a biometric information detection unit 49, an electronic control device 50, a communication device 31, and a vehicle driving control device 35.

[0044] (1-1. Occupant Detection Unit)

[0045] The occupant detection unit 41 is provided inside the vehicle and detects occupants of the vehicle such as the driver and / or passengers. The electronic control unit 50 is configured to be able to acquire the information detected by the occupant detection unit 41. The occupant detection unit 41 can at least detect the presence of an occupant in the vehicle and can also identify each occupant. In the present embodiment, the occupant detection unit 41 is configured to include a camera that captures the interior of the vehicle and an image processing device that determines each occupant based on the captured data obtained by the camera. The image processing device calculates the feature amounts of the faces of the persons by performing image processing on the captured data and determines each person. The occupant detection unit 41 can also determine the position where the detected occupant is sitting. The electronic control unit 50 uses the acquired information of the occupants for the learning of the emotions of each occupant.

[0046] (1-2. Vehicle information detection unit)

[0047] The vehicle information detection unit 43 detects information on the driving state of the vehicle. The driving state of the vehicle includes the operating state and behavior of the vehicle. The electronic control unit 50 is configured to be able to acquire the information detected by the vehicle information detection unit 43. The vehicle information detection unit 43 detects information on the behavior of the vehicle such as the vehicle speed, acceleration, and yaw rate. The vehicle information detection unit 43 can include at least one of, for example, an engine speed sensor, a vehicle speed sensor, an acceleration sensor, and an angular velocity sensor. In addition, the vehicle information detection unit 43 detects information on the operating state of the vehicle such as the accelerator operation amount, the brake operation amount, and the steering wheel steering angle. The vehicle information detection unit 43 can include at least one of, for example, an accelerator position sensor, a brake stroke sensor, and a steering angle sensor. The electronic control unit 50 uses the acquired vehicle information for the learning of the emotions of each occupant.

[0048] (1-3. Input unit)

[0049] The input unit 45 receives input operations from the driver and / or passengers and other users. In the present embodiment, the input unit 45 receives an input operation for switching the driving mode to the manual driving mode or the autonomous driving mode. In addition, when the driving mode is set to the autonomous driving mode, the input unit 45 receives an input operation for setting the emotion of the occupant as the target when setting the control parameters using the occupant emotion model 63. As will be described later, in the present embodiment, positive emotions and negative emotions are each defined as four levels, and the emotions of the occupants include the intermediate neutral emotion and are defined as a total of nine levels. Therefore, the driver or other occupants set any one of the nine levels of emotions as the target emotion.

[0050] The input unit 45 is not particularly limited and may be an appropriate input device such as a touch panel, a dial switch, or a button switch. Alternatively, the input unit 45 may be a device that receives input based on sound or gesture.

[0051] (1-4. Surrounding Environment Detection Unit)

[0052] The surrounding environment detection unit 47 detects information on the surrounding environment of the vehicle. The electronic control device 50 is configured to be able to acquire the information detected by the surrounding environment detection unit 47. The surrounding environment detection unit 47 detects information on people and / or other vehicles, bicycles, buildings, other obstacles, etc. existing around the vehicle as the surrounding environment of the vehicle. In addition, the surrounding environment detection unit 47 detects the weather and / or road surface conditions, sunlight conditions, etc. of the driving position or driving area of the vehicle. The surrounding environment detection unit 47 includes at least one of detectors such as a camera that captures the surroundings of the vehicle, a radar that detects objects around the vehicle, and a LiDAR that detects the distance and / or azimuth to an object around the vehicle. In addition, the surrounding environment detection unit 47 may include a communication device that obtains information from a device outside the vehicle, such as vehicle-to-vehicle communication or vehicle-to-roadside communication. Further, the surrounding environment detection unit 47 may also include a detector that detects information related to road surface friction. The electronic control device 50 uses the acquired information on the surrounding environment to determine the surrounding environment.

[0053] (1-5. Biological Information Detection Unit)

[0054] The biological information detection unit 49 is composed of one or more detection devices that detect information for estimating the emotions and / or feelings of the occupant. The electronic control device 50 is configured to be able to acquire the information detected by the biological information detection unit 49. The camera and image processing device that make up the occupant detection unit 41 can function as the biological information detection unit 49. For example, the image processing device can detect biological information such as the heart rate and / or pulse, body temperature, etc. of the occupant based on the change in the color of the facial image obtained through the camera. In addition to this, the biological information detection unit 49 may include, for example: a radio Doppler sensor for detecting the heart rate of the occupant; a non-wearable pulse sensor for detecting the pulse of the occupant; electrodes embedded in the steering wheel for measuring the heart rate or electrocardiogram of the driver; a pressure measurer embedded in the driver's seat for measuring the seat pressure distribution during the period when the occupant is sitting on the seat; a device for detecting the change in the position of the seat belt for measuring the heart rate or respiration of the occupant; a TOF (Time Of Flight) sensor for detecting information on the position (biological position) of the occupant; or at least one of a temperature recorder for measuring the surface temperature of the skin of the occupant. In addition, the occupant detection unit 41 may include a wearable detector such as a wearable device worn by the occupant to detect the biological information of the occupant.

[0055] (1-6. Communication Device)

[0056] The communication device 31 is an interface through which the electronic control device 50 transmits and receives information to and from an external server 20. For example, the communication device 31 can be a communication interface capable of accessing the server 20 via a mobile communication network. The communication device 31 is a device provided for sharing the driving emotion database 61 stored in a certain vehicle among multiple vehicles and for communicating with the external server 20, and can be omitted when the driving emotion database 61 is not shared.

[0057] (1-7. Vehicle driving control device)

[0058] The vehicle driving control device 35 performs vehicle driving control. The vehicle driving control device 35 includes one or more control devices that perform vehicle driving control. For example, the vehicle driving control device 35 includes control devices for controlling driving such as an engine and / or a power transmission mechanism including one or more drive motors and a transmission, a steering system, and a braking system. In the present embodiment, the vehicle driving control device 35 is configured to be able to perform vehicle autonomous driving control. When the driving mode is set to the autonomous driving mode, the vehicle driving control device 35 automatically controls at least a part of the vehicle driving control without relying on the driver's operation and drives the vehicle to the destination via a set driving route.

[0059] In addition, during the autonomous driving mode, the vehicle driving control device 35 receives an instruction from the electronic control device 50 and performs vehicle driving control. Specifically, the vehicle driving control device 35 uses the control parameters sent from the electronic control device 50 to perform vehicle autonomous driving control.

[0060] (1-8. Electronic control device)

[0061] The electronic control device 50 is configured to include an arithmetic processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and storage elements such as a RAM (Random Access Memory) and / or a ROM (Read Only Memory). The arithmetic processing device executes various arithmetic processes by executing a program stored in the storage element. The electronic control device 50 may include, together with or instead of the storage element, a storage medium such as an HDD (Hard Disk Drive), a CD (Compact Disc), a DVD (Digital Versatile Disc), an SSD (Solid State Drive), a USB (Universal Serial Bus) flash memory, or a storage device. It should be noted that part or all of the electronic control device 50 may be constituted by a device such as firmware that can be updated, or may be a program module or the like that is executed by an instruction from a CPU or the like.

[0062] The electronic control device 50 is directly or via a communication line such as a CAN (Controller Area Network) and / or a LIN (Local Inter-Net) connected to the occupant detection unit 41, the vehicle information detection unit 43, the input unit 45, the surrounding environment detection unit 47, the biological information detection unit 49, and the vehicle driving control device 35.

[0063] In the present embodiment, the electronic control device 50 includes: a driving mode setting unit 51, a surrounding environment determination unit 53, an occupant emotion estimation unit 55, an occupant emotion learning unit 57, an emotion deterioration determination unit 59, a driving emotion database 61, an occupant emotion model 63, and a control parameter setting unit 65. Among them, the driving mode setting unit 51, the surrounding environment determination unit 53, the occupant emotion estimation unit 55, the occupant emotion learning unit 57, the emotion deterioration determination unit 59, and the control parameter setting unit 65 may be functions realized by the arithmetic processing device executing a program. In addition, the driving emotion database 61 and the occupant emotion model 63 are constituted by data stored in the storage unit.

[0064] (1-8-1. Driving Mode Setting Unit)

[0065] The driving mode setting unit 51 switches the driving mode of the vehicle to a manual driving mode or an autonomous driving mode based on the operation input signal sent from the input unit 45. The driving mode setting unit 51 can be configured to be able to set the level of the autonomous driving mode. In the autonomous driving assistance device 10 of the present embodiment, the electronic control device 50 learns the emotions of the occupants during the period when the vehicle is traveling in the manual driving mode to construct an occupant emotion model 63. On the other hand, during the period when the vehicle is traveling in the autonomous driving mode, the electronic control device 50 uses the occupant emotion model 63 to estimate the emotions of the occupants, calculates the ideal driving state of the vehicle in which the emotions of the occupants are closer to the target emotions than the current emotions, and sets the control parameters for the autonomous driving control based on this ideal driving state.

[0066] (1-8-2. Surrounding environment determination unit)

[0067] The surrounding environment determination unit 53 determines the surrounding environment of the vehicle based on the information of the surrounding environment sent from the surrounding environment detection unit 47. Specifically, the surrounding environment determination unit 53 determines the position of the vehicle on the map and / or the positions and / or sizes of other vehicles including oncoming vehicles, bicycles, pedestrians, buildings, other obstacles, etc., the distances between these obstacles and the own vehicle, the relative speeds between the obstacles and the own vehicle, etc. The surrounding environment determination unit 53 determines the surrounding environment at a predetermined time interval according to the processing speed of the electronic control device 50.

[0068] (1-8-3. Occupant emotion estimation unit)

[0069] The occupant emotion estimation unit 55 estimates the emotions of the occupants based on the biological information sent from the biological information detection unit 49. For example, the occupant emotion estimation unit 55 can be configured to convert each biological information such as the heart rate and / or brain waves sent from the biological information detection unit 49 into an index of each pre-set emotion such as "fear" and / or "pleasure". Specifically, the occupant emotion estimation unit 55 can be configured to associate each biological information with a two-dimensional coordinate of each pre-set emotion. Each emotion can be defined as two levels of positive or negative, or multiple levels can be defined for each of the positive side and the negative side. In the present embodiment, each emotion is defined as 4 levels, and a total of 9 levels are defined including a neutral emotion in the middle.

[0070] It should be noted that in this specification, "positive" means a state of "good emotion" such as being on the safe side or the reassuring side for each emotion, and "negative" means a state of "bad emotion".

[0071] (1-8-4. Occupant emotion learning unit)

[0072] The occupant emotion learning unit 57 learns the occupant emotion model 63 while the driving mode is set to the manual driving mode. The occupant emotion learning unit 57 constructs the occupant emotion model 63 based on the information of the occupant's emotion estimated by the occupant emotion estimation unit 55, the information of the driving state of the vehicle detected by the vehicle information detection unit 43, and the information of the surrounding environment of the vehicle determined by the surrounding environment determination unit 53. Specifically, the occupant emotion learning unit 57 stores a data set composed of the information of the occupant's emotion, the information of the driving state of the vehicle temporally associated with the information of the occupant's emotion, and the information of the surrounding environment in the driving emotion database 61, and uses this data set to construct the occupant emotion model 63.

[0073] Figure 2 An example of the occupant emotion model 63 is shown. The occupant emotion model 63 is a learning model that takes the information of the driving state of the vehicle and the information of the surrounding environment as inputs and estimates the occupant emotion as an output. Since the data set used for the construction of the occupant emotion model 63 is a time-series data set, by inputting the time-series data of the information of the driving state of the vehicle and the information of the surrounding environment into the occupant emotion model 63 as inputs, the emotion of the occupant can be sequentially estimated.

[0074] As the information of the driving state of the vehicle set as input data, it preferably includes the time-series data of the acceleration in the front-rear, left-right, and up-down directions of the vehicle, the angular velocity of the yaw angle, pitch angle, and roll angle, vehicle speed, steering angle, accelerator operation amount, and brake operation amount. Further, as the information of the driving state of the vehicle, it may also include information such as engine speed, direction indicator output, number of occupants, and attributes of each occupant. In addition, as the information of the surrounding environment set as input data, it preferably includes the driving lane of the own vehicle, attributes related to traffic participants such as other vehicles and / or pedestrians, relative distance, relative speed, and traveling direction, and traffic rules and traffic control data such as the number of lanes of the road during driving, signal light information, and speed limit. Further, as the information of the surrounding environment, it may also include information such as weather, road surface condition, and attributes of buildings.

[0075] In addition, the occupant emotion model 63 can receive inputs corresponding to the number of input data while maintaining the time order of the above time-series data. Or, the occupant emotion model 63 can specify, for example, a time window to extract feature quantities from the above time-series data, such as extracting the maximum value, minimum value, and average value within the time window, and receiving them as input data. In this case, the number of input data is proportional to the number of feature quantities extracted from the time-series data. In addition, the occupant emotion model 63 outputs an estimated value (one value) of the occupant emotion.

[0076] It should be noted that the method for constructing the occupant emotion model 63 is not particularly limited. For example, well-known methods such as neural networks like support vector machines, nearest neighbor methods, deep learning, or Bayesian networks can be appropriately adopted.

[0077] (1-8-5. Emotion Deterioration Determination Unit)

[0078] During the period when the driving mode is set to the autonomous driving mode, the emotion deterioration determination unit 59 determines whether the emotion of the occupant has deteriorated. In the present embodiment, the emotion deterioration determination unit 59 determines whether the level of the emotion estimated by the occupant emotion estimation unit 55 is on the negative side compared to the target level of the emotion set via the input unit 45.

[0079] (1-8-6. Control Parameter Setting Unit)

[0080] During the period when the driving mode is set to the autonomous driving mode, the control parameter setting unit 65 sets control parameters for the vehicle driving control device 35 to perform autonomous driving control. The control parameter setting unit 65 calculates the ideal driving state of the vehicle in which the emotion of the occupant approaches the target level based on the occupant emotion model 63, and sets the control parameters for autonomous driving control based on the ideal driving state. In the present embodiment, the control parameter setting unit 65 inputs multiple input values related to the driving state of the vehicle, such as vehicle speed and / or acceleration / deceleration, steering angle, etc., into the occupant emotion model 63 respectively, and sets the input value in which the emotion of the occupant is closer to the target level than the current emotion among the input values input to the occupant emotion model 63 as the ideal driving state of the vehicle to set the control parameters.

[0081] The control parameter is a variable used to calculate the control amount of each control object when the vehicle driving control device 35 performs the autonomous driving control of the vehicle. For example, the control parameter includes values such as vehicle speed and / or acceleration / deceleration, steering rate, etc.

[0082] The target level of the emotion is set, for example, via the input unit 45. An occupant such as a driver can perform an operation input on the input unit 45 in advance and set the target level of the emotion. Alternatively, the control parameter setting unit 65 can always set the emotion level that is one level or more on the positive side than the emotion level of the occupant estimated by the occupant emotion estimation unit 55 as the target level. In addition, the control parameter setting unit 65 can always set the highest emotion level on the positive side as the target level. In addition, when the emotion level of the occupant estimated by the occupant emotion estimation unit 55 is on the negative side, the control parameter setting unit 65 can set the neutral emotion level as the target level.

[0083] Furthermore, when the emotion of the occupant estimated by the occupant emotion estimation unit 55 becomes negative while the vehicle is traveling under a predetermined condition, the control parameter setting unit 65 may ask the occupant about the target level of emotion under a similar condition and make a setting based on the response. For example, when the vehicle turns right, if the vehicle turns right despite approaching an oncoming vehicle, and the emotion of the occupant estimated by the occupant emotion estimation unit 55 based on the biological information of the occupant changes to the negative side compared to the emotion state before the right turn, the control parameter setting unit 65 asks about the target level of emotion under the same condition. The content of the question is set in advance and played through an in-vehicle speaker or the like when changing to the negative side.

[0084] Specifically, the control parameter setting unit 65 asks the occupant "Is it okay to drive without fear under the current condition?" or asks "Is it okay to drive while paying attention to timing even if a little uneasy under the current condition?". When the occupant gives an affirmative response of "It is okay to drive without fear under the current condition", the control parameter setting unit 65 sets the target level of emotion to neutral. In addition, when the occupant gives an affirmative response of "It is okay to drive while paying attention to timing even if a little uneasy under the current condition", the control parameter setting unit 65 sets the emotion level one level or multiple levels higher than the emotion level estimated at this time as the target level.

[0085] In addition, the control parameter setting unit 65 prepares a plurality of candidates for input values to be input to the occupant emotion model 63 and sequentially inputs the plurality of input values to the occupant emotion model 63. The control parameter setting unit 65 selects, from the plurality of emotion levels output from the occupant emotion model 63, the input value corresponding to the emotion level closest to the target level based on the plurality of input values. Then, the control parameter setting unit 65 sets the control parameter based on the selected input value with the input value as the ideal driving state. Thus, the automatic driving control of the vehicle is executed in such a way that the emotion of the occupant approaches the target level. It should be noted that when there are a plurality of the above input values closest to the target level, the input value closest to the current control parameter among these input values can be selected.

[0086] Figure 3 An example of an algorithm for setting the control parameter of the vehicle using the occupant emotion model 63 is shown. The input values when calculating the control parameter are composed of input values related to the operation state and behavior of the vehicle during the automatic driving control process. Since the input values of the surrounding environment are information that cannot be controlled (fixed values), the information (fixed values) of the surrounding environment detected by the surrounding environment detection unit 47 is used.

[0087] The number of candidates for the prepared input values is appropriately set based on at least one of the processing speed of the arithmetic processing unit constituting the electronic control device 50 and the update frequency of the control parameters. For example, as described below, the number of candidates for the multiple input values to be prepared is set in advance. First, based on the amount of data processing required by the occupant emotion model 63 and the processing capacity of the arithmetic processing unit that performs arithmetic processing using the occupant emotion model 63, the number of arithmetic operations for inputting input values to the occupant emotion model 63 per unit time to output the emotion of the occupant is calculated. Based on the calculated number of arithmetic operations and the update speed of the control parameters, the number of arithmetic operations that can be performed each time the control parameters are updated is calculated. This number of arithmetic operations is set as the number of candidates for the input values. In order to execute the autonomous driving control meticulously, it is preferable to set the number of arithmetic operations as the maximum number that can be set considering the processing speed of the arithmetic processing unit and the update frequency of the control parameters.

[0088] The control parameter setting unit 65 sequentially inputs a plurality of input values to the occupant emotion model 63 to obtain each emotion level, and selects the input value closest to the target level. Then, the control parameter setting unit 65 sets this input value as the ideal driving state, and sets the control parameters based on the selected input value. Specifically, the control parameter setting unit 65 extracts, from the data sets stored in the driving emotion database 61, the data set of the driving state of the vehicle when it is consistent with or similar to the information of the current surrounding environment detected by the surrounding environment detection unit 47 and is the emotion level closest to the target level of the set emotion. The control parameter setting unit 65 sets the extracted data set of the driving state of the vehicle as the reference operation target value.

[0089] The control parameter setting unit 65 generates a plurality of input value candidates equivalent to the above-mentioned number of arithmetic operations between the current value and the above-mentioned reference operation target value for information on the driving state of the vehicle such as the steering angle and / or acceleration. For example, for the vehicle speed data, when the current vehicle speed is 40 km / h, the reference operation target value is 30 km / h, and the number of arithmetic operations is 3, the candidates for the input value of the vehicle speed become 30 km / h, 35 km / h, and 40 km / h. The control parameter setting unit 65 performs the same processing on all other data and generates a plurality of input value candidates. It should be noted that a plurality of input value candidates are prepared because when the information on the surrounding environment of the data set set as the reference operation target value is not completely consistent with the current surrounding environment, the reference operation target value may not necessarily be the driving state of the vehicle that can achieve the target level of emotion.

[0090] The control parameter setting unit 65 sequentially inputs the generated multiple input values to the occupant emotion model 63, and selects the data set of the driving state of the vehicle in which the emotion of the output occupant is closest to the target level. Then, the control parameter setting unit 65 sets control parameters in the automatic driving control such as vehicle speed and / or acceleration / deceleration, steering rate, etc. based on the selected data set. The number of input value candidates is set based on at least one of the processing speed of the arithmetic processing device and the update frequency of the control parameters as described above. Therefore, the control parameter setting unit 65 can sequentially update the control parameters in each processing cycle of the arithmetic processing device. The control parameter setting unit 65 executes the setting of the control parameters using the occupant emotion model 63 in each processing cycle of the arithmetic processing device, and transmits the information of the set control parameters to the vehicle driving control device 35. Thereby, it is possible to sequentially obtain the control parameters for the automatic driving control that makes the emotion of the occupant reach the target level, and execute the automatic driving control that is pleasant for the occupant.

[0091] <2. Operation example>

[0092] So far, the configuration example of the automatic driving assistance device 10 of the present embodiment has been described. Next, based on Figures 4 to 6 the flowchart shown, the operation example of the automatic driving assistance device 10 of the present embodiment will be described. In the following description, the description of the content already described in the above configuration example may sometimes be omitted.

[0093] First, if the electronic control unit 50 of the automatic driving assistance device 10 detects the start of the system (step S11), the occupant detection unit 41 identifies the occupant of the vehicle (step S13). In addition, the driving mode setting unit 51 sets the driving mode of the vehicle to the manual driving mode or the automatic driving mode according to the driving mode input via the input unit 45 (step S15).

[0094] Next, the driving mode setting unit 51 determines whether the driving mode is an autonomous driving mode (step S17). When the driving mode is not the autonomous driving mode (S17 / No), that is, when the driving mode is set to the manual driving mode, the occupant emotion estimation unit 55 estimates the emotion of the occupant based on the biological information transmitted from the biological information detection unit 49 (step S27). In the present embodiment, the level of the emotion of the occupant, which is defined as multiple levels, is estimated. The emotion of the occupant estimated at this time is stored as information on the emotion at the start of driving. Next, the vehicle travel control device 35 starts the travel control of the vehicle according to the driving operation of the driver and thus starts the travel of the vehicle (step S29). After starting the travel of the vehicle in the manual driving mode, the electronic control device 50 executes the learning process of the occupant emotion model 63 by the occupant emotion learning unit 57 (step S31). During the period until the system stops (the period until step S25 is determined to be affirmative), in the state where the vehicle is set to the manual driving mode, the electronic control device 50 repeatedly executes the learning process of the occupant emotion model 63.

[0095] Figure 5 FIG. is a flowchart showing an example of the occupant emotion model learning process performed by the autonomous driving support device 10. First, the surrounding environment determination unit 53 determines the information on the current surrounding environment of the vehicle based on the information on the surrounding environment transmitted from the surrounding environment detection unit 47 (step S41). Next, the vehicle information detection unit 43 detects the driving state of the current vehicle (step S43). Next, the occupant emotion estimation unit 55 estimates the emotion of the occupant based on the biological information transmitted from the biological information detection unit 49 (step S45).

[0096] Next, the occupant emotion learning unit 57 acquires the information on the emotion of the occupant, the information on the surrounding environment, and the information on the driving state of the vehicle, and stores the information on the emotion of the occupant, and the information on the surrounding environment and the driving state of the vehicle that is temporally associated with the information on the emotion of the occupant in the travel emotion database 61 (step S47). Next, the occupant emotion learning unit 57 uses the information on the emotion of the occupant, and the information on the surrounding environment and the driving state of the vehicle that is temporally associated with the information on the emotion of the occupant, and uses a known method such as deep learning to construct or update the occupant emotion model 63 (step S49). The occupant emotion learning unit 57 repeatedly executes the processes of steps S41 to S49 in each processing cycle of the arithmetic processing device, and constructs the occupant emotion model 63.

[0097] On the other hand, in step S17 described above, when the driving mode is the autonomous driving mode (S17 / Yes), the occupant emotion estimation unit 55 estimates the emotion of the occupant based on the biological information transmitted from the biological information detection unit 49 (step S19). The emotion of the occupant estimated at this time is stored as information on the emotion at the start of driving. Next, the vehicle driving control device 35 starts the driving control of the vehicle in the autonomous driving mode to start the driving of the vehicle (step S21). After starting the driving of the vehicle in the autonomous driving mode, the electronic control device 50 executes the driving control of the vehicle based on the emotion of the occupant (step S23). During the period until the system stops (the period until step S25 is determined to be affirmative), in a state where the vehicle is set to the autonomous driving mode, the electronic control device 50 continues to perform the driving control of the vehicle based on the emotion of the occupant.

[0098] Figure 6 FIG. shows an example of a flowchart of the driving control process based on the driving emotion performed by the autonomous driving support device 10. First, the surrounding environment determination unit 53 determines the information on the current surrounding environment of the vehicle based on the information on the surrounding environment transmitted from the surrounding environment detection unit 47 (step S51). Next, the occupant emotion estimation unit 55 estimates the emotion of the occupant based on the biological information transmitted from the biological information detection unit 49 (step S53). Next, the emotion deterioration determination unit 59 determines whether the emotion of the occupant estimated has deteriorated (step S55). In the present embodiment, the emotion deterioration determination unit 59 discriminates whether the grade of the emotion of the occupant estimated has changed to the negative side compared with the previous time.

[0099] When the emotion of the occupant has not deteriorated (S55 / No), the autonomous driving support device 10 returns to step S51 and repeats the processes of step S51 to step S55. On the other hand, when the emotion of the occupant has deteriorated (S55 / Yes), the control parameter setting unit 65 refers to the set target emotion value (step S57). In the present embodiment, the information on the target grade of the set emotion is referred to.

[0100] Next, the control parameter setting unit 65 refers to the driving emotion database 61 to generate candidates for a plurality of input values input to the occupant emotion model 63 (step S59). Specifically, the control parameter setting unit 65 refers to the driving emotion database 61 and extracts a data set of the driving state of the vehicle in which the emotion of the occupant is closest to the target grade from the data sets of the surrounding environments that are the same as or similar to the current surrounding environment. Next, the control parameter setting unit 65 sets the extracted data set of the driving state of the vehicle as the reference operation target value, and generates candidates for a plurality of input values corresponding to the preset number of operations between the value of the current driving state of the vehicle and the reference operation target value.

[0101] Next, the control parameter setting unit 65 inputs one of the candidates of the input value to the occupant emotion model 63 and obtains the information on the emotion of the occupant output (step S61). Next, the control parameter setting unit 65 calculates the difference between the value of the emotion of the occupant output from the occupant emotion model 63 and the target emotion value (step S63). In the present embodiment, the control parameter setting unit 65 calculates the difference between the emotion level output from the occupant emotion model 63 and the target level of the emotion. Next, the control parameter setting unit 65 determines whether all the candidates of the input value have been input to the occupant emotion model 63 (step S65). If not all the candidates of the input value have been input (S65 / No), the control parameter setting unit 65 returns to step S61 and inputs the next candidate of the input value to the occupant emotion model 63. The control parameter setting unit 65 repeatedly executes the processes of steps S61 to S65 until all the candidates of the input value have been input.

[0102] If all the candidates of the input value have been input (S65 / Yes), the control parameter setting unit 65 selects the input value with the smallest difference between the emotion level output from the occupant emotion model 63 and the target level of the emotion (step S67). The input value selected here is the input value for which the emotion level of the occupant is closest to the target level. Next, the control parameter setting unit 65 converts the selected input value into a control parameter for the autonomous driving control (step S69). Next, the control parameter setting unit 65 sends the calculated control parameter to the vehicle driving control device 35 and reflects it in the autonomous driving control (step S71).

[0103] When performing the driving control based on the occupant emotion during the autonomous driving mode (step S23), or when performing the learning process of the occupant emotion model 63 during the manual driving mode (step S31), in either case, the autonomous driving support device 10 determines whether the system has stopped (step S25). If the system has not stopped (S25 / No), the autonomous driving support device 10 repeats the above processes of steps S17 to S31. On the other hand, if the system has stopped (S25 / Yes), the autonomous driving support device 10 ends the control process.

[0104] As described above, during the period when the driving mode of the automatic driving assistance device 10 of the present embodiment is set to the manual driving mode, the information on the emotion of the occupant, and the information on the driving state of the vehicle and the surrounding environment that are temporally correlated with the information on the emotion of the occupant are stored in the driving emotion database 61, and these information are used to construct the occupant emotion model 63. Further, during the period when the driving mode of the automatic driving assistance device 10 is set to the automatic driving mode, referring to the driving emotion database 61, a plurality of candidates for the input values of the driving state of the vehicle input to the occupant emotion model 63 are generated, and the control parameters of the automatic driving control are set based on the input value whose emotion level output from the occupant emotion model 63 is closest to the target level.

[0105] The driving emotion database 61 is a data set of information that is temporally correlated. The control parameter setting unit 65 sets the control parameters based on the input value whose occupant emotion level is close to the target level from among the plurality of candidates for the input values of the driving state of the vehicle in each predetermined processing cycle. Therefore, it is possible to specifically and sequentially set the control parameters that can make the emotion of the occupant close to the target emotion.

[0106] In addition, the control parameter setting unit 65 sets the control parameters using the data set of the surrounding environment that is the same as or similar to the current surrounding environment of the vehicle from the driving emotion database 61. Therefore, even when there is no data set with the same surrounding environment in the driving emotion database 61, it is possible to achieve the driving control that guides to a state close to the target emotion.

[0107] <3. Variation Example>

[0108] So far, the automatic driving assistance device 10 of the present embodiment has been described, but the automatic driving assistance device 10 of the present embodiment can be variously deformed. Hereinafter, a variation example in which the control parameter setting unit 65 further sets the control parameters in cooperation with each occupant will be described.

[0109] The first variation example is an example in which, for the plurality of candidates for the input values input to the occupant emotion model 63, the occupant determines in advance the value of one or more data items that are prioritized in each data item of the input values. For example, when a certain occupant attaches importance to the vehicle speed in the automatic driving control, the occupant pre-sets the vehicle speed as the priority item. When the control parameter setting unit 65 generates the candidates for the plurality of input values, the vehicle speed data is fixed to the vehicle speed that constitutes the reference operation target value extracted from the driving emotion database 61, and the candidates for the input values are generated for the other data items. Thereby, it is possible to set the control parameters that reflect the preference of the occupant regarding the driving state of the vehicle and make the emotion of the occupant close to the target emotion.

[0110] The second modified example is an example in which the control parameter setting unit 65 extracts, for each occupant, data items of the driving state of the vehicle that have a great influence on the mood from the driving mood database 61, and fixes the data items for the occupant to generate candidates for input values. For example, the occupant detection unit 41 identifies and recognizes each occupant, and creates a driving mood database 61 that stores data sets for each occupant. Then, the occupant mood learning unit 57 analyzes the sensitivity to the occupant mood model 63 and extracts data items that have a great influence on the mood. The analysis of the sensitivity can be, for example, a method of analyzing data items whose output mood level changes significantly when one item is removed or the value of one item is changed among the input data items. The control parameter setting unit 65 fixes the determined data items as input values that constitute the reference operation target value extracted from the driving mood database 61, and generates candidates for input values for other data items. Thus, it is possible to preferentially set input values for data items that have a great influence on the mood of each occupant, and to set control parameters that bring the mood of the occupant closer to the target mood.

[0111] Furthermore, when reflecting the control parameters calculated by using the method of the first modified example above in the automatic driving control, if the mood level of the occupant estimated based on the biological information detected by the biological information detection unit 49 is more negative than the mood level calculated by using the occupant mood model 63, it is possible to recommend to the occupant to set, as priority data items, items that have a great influence on the mood obtained by the method of the second modified example.

[0112] According to these modified examples, it is possible to reflect at least one of the preferences of the occupant and the objective tendency based on the stored driving mood database 61 in the automatic driving control using the occupant mood model 63, and to perform automatic driving control that reflects the intentions and / or characteristics of each occupant.

[0113] As described above, the preferred embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to this example. Those having ordinary knowledge in the technical field to which the present invention pertains will know that various modification examples or correction examples can be conceived within the scope of the technical idea described in the claims, and will understand that these modification examples or correction examples also belong to the technical scope of the present invention.

[0114] For example, in the above embodiment, the occupant mood learning unit 57 stores data sets in the driving mood database 61 and updates the occupant mood model 63 when the driving mode is the manual driving mode, but the present invention is not limited to this example. The occupant mood learning unit 57 may also store data sets in the driving mood database 61 and update the occupant mood model 63 even when the driving mode is the automatic driving mode.

Claims

1. An autonomous driving assistance device, characterized in that, Comprising: An occupant emotion learning unit that constructs an occupant emotion model for inferring an occupant's emotion from the driving state of the vehicle based on information on the driving state of the vehicle and the occupant's emotion during a period when the driving mode of the vehicle is set to a manual driving mode; And A control parameter setting unit that, during a period when the driving mode is set to an autonomous driving mode, calculates an ideal driving state of the vehicle in which the occupant's emotion approaches a target emotion based on the occupant emotion model, and sets control parameters for autonomous driving control based on the ideal driving state, The control parameter setting unit inputs a plurality of input values related to the driving state of the vehicle into the occupant emotion model, and sets the control parameters with the input value in which the occupant's emotion is closer to the target emotion than the current emotion among the input values input into the occupant emotion model as the ideal driving state of the vehicle.

2. The autonomous driving assistance device according to claim 1, wherein The occupant emotion model is an occupant emotion model that infers the occupant's emotion based on information on the surrounding environment of the vehicle and information on the driving state of the vehicle, The control parameter setting unit obtains an ideal driving state of the vehicle in the surrounding environment corresponding to the current surrounding environment of the vehicle and sets the control parameters.

3. The autonomous driving assistance device according to claim 2, wherein The occupant emotion learning unit stores a data set associating information on the inferred occupant's emotion, information on the surrounding environment of the vehicle, and information on the driving state of the vehicle, The control parameter setting unit extracts a driving state of the vehicle that is close to the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle from the stored data set, and generates the plurality of input values based on the information on the extracted driving state of the vehicle.

4. The autonomous driving assistance device according to claim 3, wherein The control parameter setting unit sets the plurality of input values between the value of a predetermined data item of the extracted driving state of the vehicle and the value of the predetermined data item of the current driving state of the vehicle.

5. The autonomous driving assistance device according to claim 3 or 4, wherein The predetermined data item of the driving state of the vehicle includes a plurality of data items, At least one of the plurality of data items can be set as a priority item by the user, When there is a priority item set by the user, the control parameter setting unit fixes the value of the priority item among the data items of the plurality of input values input into the occupant emotion model as the value extracted as the driving state of the vehicle that is close to the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle, and generates the plurality of input values for other data items.

6. The autonomous driving assistance device according to claim 5, wherein When the emotion of the occupant during the execution of the autonomous driving control with the control parameter obtained by fixing the value of the priority item deteriorates compared to the emotion of the occupant calculated using the occupant emotion model, the control parameter setting unit prompts the user to fix at least one data item that has a great impact on the emotion of the occupant as a value extracted as the driving state of the vehicle approaching the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle.

7. The autonomous driving assistance device according to claim 3, wherein: A predetermined data item of the driving state of the vehicle includes a plurality of data items. Based on the stored data set, the occupant emotion learning unit extracts at least one data item that has a great impact on the emotion of the occupant. The control parameter setting unit fixes at least one data item that has a great impact on the emotion of the occupant as a value extracted as the driving state of the vehicle approaching the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle, thereby obtaining the plurality of input values, and uses the plurality of input values to set the control parameter.

8. The autonomous driving assistance device according to any one of claims 1 to 4 and 7, wherein: The control parameter setting unit sets the number of the plurality of input values input to the occupant emotion model based on at least one of the processing speed of the arithmetic processing device performing the operation and the update frequency of the control parameter.

9. The autonomous driving assistance device according to claim 5, wherein: The control parameter setting unit sets the number of the plurality of input values input to the occupant emotion model based on at least one of the processing speed of the arithmetic processing device performing the operation and the update frequency of the control parameter.

10. The autonomous driving assistance device according to claim 6, wherein: The control parameter setting unit sets the number of the plurality of input values input to the occupant emotion model based on at least one of the processing speed of the arithmetic processing device performing the operation and the update frequency of the control parameter.

11. The autonomous driving assistance device according to claim 1, wherein: The occupant emotion learning unit constructs the occupant emotion model using the information on the surrounding environment of the vehicle and the information on the driving state obtained during the travel of the vehicle in the manual driving mode, and the information on the emotion of the occupant associated with the information on the surrounding environment and the information on the driving state.

Images