Autonomous Driving Assistance Device

Abstract

To provide an automatic operation assistance device that is able to specifically and sequentially calculate automatic operation control parameters for a vehicle in order to bring emotion of an occupant such as a driver or a passenger close to an ideal state.SOLUTION: An automatic drive assistance device comprises: an occupant feeling learning unit that constructs an occupant feeling model for estimating an emotion of an occupant from an operating state of a vehicle, based on information on the operating state of the vehicle and information on the emotion of the occupant; and a control parameter setting unit that calculates an ideal operating state of the vehicle, in which the emotion of the occupant approaches a target emotion, based on the occupant feeling model and sets a control parameter for automatic operation control, based on the ideal operating state. The control parameter setting unit inputs a plurality of input values relating to an operating state of the vehicle into the occupant feeling model and sets the control parameter so that, among the input values input to the occupant feeling model, an input value at which the emotion of the occupant is closer to the target emotion than the current emotion is the ideal operating state of the vehicle.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an automatic driving assistance device. [Background technology]

[0002] In an autonomous vehicle, a technology has been proposed to suppress negative emotions such as anxiety and discomfort by reflecting the driving characteristics of occupants such as the driver and passengers in the driving control of the vehicle. For example, Patent Document 1 discloses a vehicle driving assistance system that includes a personal server that learns a personal driver model specific to the driver based on the driving data of the driver, and an in-vehicle controller that is provided in the driver's vehicle and executes a predetermined vehicle control process, the personal server includes a recommendation engine that instructs the in-vehicle controller to execute a recommended process, and the recommendation engine analyzes the current emotional state of the driver based on the voice data of the driver included in the driving data, and determines a recommended process according to the analyzed emotional state based on the personal driver model.

[0003] Furthermore, Patent Document 2 discloses an electronic control device that estimates the emotions of a driver or passenger of a vehicle using biometric information acquired from a biometric sensor that measures the biometric information of the driver or passenger, and controls the driving of the vehicle based on the estimated emotions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-169704 A [Patent Document 2] JP 2017-136922 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, although Patent Document 1 describes that the recommendation engine derives an appropriate recommendation process and selects and outputs an appropriate recommendation signal from among a vehicle interior space recommendation signal, a driving route recommendation signal, and an information presentation recommendation signal according to the situation, it does not describe a procedure for specifically calculating the control parameters. Patent Document 2 also does not describe a procedure for specifically calculating parameters for controlling the driving of the vehicle based on the estimated emotion. In order to execute the automatic driving control of the vehicle, the control parameters must be calculated sequentially, and a method for specifically and sequentially calculating the control parameters is required.

[0006] The present invention has been made in consideration of the above problems, and an object of the present invention is to provide an automatic driving assistance device that can specifically and sequentially calculate automatic driving control parameters for a vehicle to bring the emotions of occupants such as the driver and passengers closer to an ideal state. [Means for solving the problem]

[0007] In order to solve the above problems, according to one aspect of the present invention, an automatic driving assistance device is provided, which includes an occupant emotion learning unit that constructs an occupant emotion model that estimates the occupant's emotions from the vehicle's driving state based on information on the vehicle's driving state and the occupant's emotions, and a control parameter setting unit that calculates an ideal driving state of the vehicle in which the occupant's emotions approach a target emotion based on the occupant emotion model, and sets control parameters for automatic driving control based on the ideal driving state, wherein the control parameter setting unit inputs a plurality of input values ​​related to the vehicle's driving state to the occupant emotion model, and sets the control parameters as the ideal driving state of the vehicle, based on the input values ​​input to the occupant emotion model that bring the occupant's emotions closer to the target emotion than the current emotion.

[0008] In addition, the occupant emotion model is an occupant emotion model that estimates the emotions of the occupant based on information on the vehicle's surrounding environment and information on the vehicle's driving state, and the control parameter setting unit may set the control parameters by determining an ideal driving state of the vehicle in an surrounding environment corresponding to the current vehicle's surrounding environment.

[0009] In addition, the occupant emotion learning unit may accumulate a data set linking information on the estimated occupant emotions, information on the vehicle's surrounding environment, and information on the vehicle's driving state, and the control parameter setting unit may extract from the accumulated data set a vehicle driving state that approaches a target emotion in an surrounding environment corresponding to the current vehicle's surrounding environment, and create multiple input values ​​based on the extracted information on the vehicle's driving state.

[0010] The control parameter setting unit may set a plurality of input values ​​between the value of a predetermined data item in the extracted driving state of the vehicle and the value of the predetermined data item in the current driving state of the vehicle.

[0011] In addition, the specified data item in the vehicle driving state includes multiple data items, and at least one of the multiple data items can be set by the user as a priority item, and when there is a priority item set by the user, the control parameter setting unit may fix the value of the priority item among the multiple input value data items to be input to the occupant emotion model to a value extracted as the vehicle driving state that approaches the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle, and may create multiple input values ​​for the other data items.

[0012] In addition, if the emotions of the occupants when autonomous driving control is executed using the control parameters obtained by fixing the values ​​of the priority items deteriorates compared to the emotions of the occupants calculated using the occupant emotion model, the control parameter setting unit may present to the user that at least one data item that has a large impact on the emotions of the occupants is to be fixed to a value extracted as a vehicle driving state that approaches the target emotion in the surrounding environment corresponding to the current surrounding environment of the vehicle.

[0013] In addition, the specified data item in the vehicle's driving state may include multiple data items, and the occupant emotion learning unit may extract at least one data item that has a large impact on the occupant's emotions based on the accumulated data set, and the control parameter setting unit may fix the at least one data item that has a large impact on the occupant's emotions to a value extracted as a vehicle driving state that approaches a target emotion in an ambient environment corresponding to the current ambient environment of the vehicle, to obtain multiple input values ​​and set the control parameters.

[0014] The control parameter setting section may set the number of the plurality of input values ​​to be input to the occupant emotion model based on at least one of the processing speed of a calculation processing device that performs the calculations and the update frequency of the control parameters. Effect of the Invention

[0015] As described above, according to the present invention, it is possible to specifically calculate vehicle automatic driving control parameters for bringing the emotions of occupants such as the driver and passengers closer to an ideal state, and it is possible to realize automatic driving control that can bring the emotions of occupants closer to an ideal state. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing a configuration example of an automatic driving assistance device according to an embodiment of the present invention. [Diagram 2] FIG. 4 is an explanatory diagram showing an example of an occupant emotion model. [Diagram 3] FIG. 4 is an explanatory diagram showing an algorithm for setting vehicle control parameters using an occupant emotion model. [Figure 4] 4 is a flowchart showing an example of the operation of the autonomous driving assistance device according to the embodiment. [Diagram 5] 5 is a flowchart showing an occupant emotion model learning process performed by the autonomous driving assistance device according to the embodiment. [Figure 6] 5 is a flowchart showing a driving control process based on passenger emotions performed by the autonomous driving assistance device according to the embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and duplicated explanations will be omitted.

[0018] <1. Configuration example of autonomous driving assistance device> First, a configuration example of an automatic driving assistance device according to an embodiment of the present invention will be described. Fig. 1 is a block diagram showing a configuration example of an automatic driving assistance device 10 according to this embodiment.

[0019] The automatic driving support device 10 is mounted on a vehicle and configured to detect information on vehicle occupants, the driving state of the vehicle, and the surrounding environment of the vehicle, and to execute control to support driving of the vehicle using the various detected information. The automatic driving support device 10 includes an occupant detection unit 41, a vehicle information detection unit 43, an input unit 45, a surrounding environment detection unit 47, a biological information detection unit 49, an electronic control unit 50, a communication unit 31, and a vehicle driving control unit 35.

[0020] (1-1. Occupant detection section) The occupant detection unit 41 is provided in the vehicle and detects occupants of the vehicle, such as a driver and a passenger. The electronic control unit 50 is configured to be able to acquire information detected by the occupant detection unit 41. The occupant detection unit 41 may at least detect the presence of an occupant in the vehicle, and may identify each occupant. In this embodiment, the occupant detection unit 41 includes a camera that captures an image of the interior of the vehicle, and an image processing device that identifies each occupant based on image data acquired by the camera. The image processing device calculates facial features of people by image processing the image data, and identifies each person. The occupant detection unit 41 may identify the position where the detected occupant sits. The electronic control unit 50 uses the acquired occupant information to learn the emotions of each occupant.

[0021] (1-2. Vehicle information detection unit) The vehicle information detection unit 43 detects information on the driving state of the vehicle. The driving state of the vehicle includes the operation state and behavior of the vehicle. The electronic control unit 50 is configured to be able to acquire information detected by the vehicle information detection unit 43. The vehicle information detection unit 43 detects information on the behavior of the vehicle, such as vehicle speed, acceleration, and yaw rate. The vehicle information detection unit 43 may include at least one of an engine speed sensor, a vehicle speed sensor, an acceleration sensor, and an angular velocity sensor, for example. The vehicle information detection unit 43 also detects information on the operation state of the vehicle, such as an accelerator operation amount, a brake operation amount, and a steering angle. The vehicle information detection unit 43 may include at least one of an accelerator position sensor, a brake stroke sensor, and a steering angle sensor, for example. The electronic control unit 50 uses the acquired vehicle information to learn the emotions of each occupant.

[0022] (1-3. Input section) The input unit 45 accepts input operations from the driver, passengers, and other users. In this embodiment, the input unit 45 accepts an input operation to switch the driving mode between the manual driving mode and the automatic driving mode. In addition, when the driving mode is set to the automatic driving mode, the input unit 45 accepts an input operation to set the target emotion of the occupant when setting the control parameters using the occupant emotion model 63. As described later, in this embodiment, positive and negative emotions are each defined as four levels, and the emotions of the occupant are defined as a total of nine levels including a neutral emotion in between. Therefore, the occupant such as the driver sets an emotion of any one of the nine levels as the target emotion.

[0023] The input unit 45 is not particularly limited and may be an appropriate input device such as a touch panel, a dial switch, a button switch, etc. Alternatively, the input unit 45 may be a device that accepts input by voice or gesture.

[0024] (1-4. Surrounding environment detection section) The surrounding environment detection unit 47 detects information on the surrounding environment of the vehicle. The electronic control unit 50 is configured to be able to acquire information detected by the surrounding environment detection unit 47. The surrounding environment detection unit 47 detects people, other vehicles, bicycles, buildings, and other obstacles around the vehicle as information on the surrounding environment of the vehicle. The surrounding environment detection unit 47 also detects the weather, road surface conditions, and sunlight conditions of the vehicle's running position or running area. The surrounding environment detection unit 47 includes at least one of detectors such as a camera that captures images of the surroundings of the vehicle, a radar that detects objects around the vehicle, and a LiDAR that detects distances and directions to objects around the vehicle. The surrounding environment detection unit 47 may also include a communication device that acquires information from a device outside the vehicle, such as vehicle-to-vehicle communication or road-to-vehicle communication. Furthermore, the surrounding environment detection unit 47 may also include a detector that detects information related to road surface friction. The electronic control unit 50 uses the acquired information on the surrounding environment to determine the surrounding environment.

[0025] (1-5. Biometric information detection section) The biometric information detection unit 49 is composed of one or more detection devices that detect information for estimating the emotions and sensitivities of the occupant. The electronic control unit 50 is configured to be able to acquire information detected by the occupant detection unit 41. The camera and image processing device that configure the occupant detection unit 41 may function as the biometric information detection unit 49. For example, the image processing device can detect biometric information such as the heart rate, pulse, and body temperature of the occupant based on a change in color of a facial image acquired by the camera. In addition, the biological information detection unit 49 may include at least one of, for example, a radio wave Doppler sensor for detecting the heart rate of the occupant, a non-wearable pulse sensor for detecting the pulse rate of the occupant, an electrode embedded in the steering wheel for measuring the driver's heart rate or electrocardiogram, a pressure gauge embedded in the driver's seat for measuring the seat pressure distribution while the occupant is seated in the seat, a device for detecting a change in the position of the seat belt to measure the occupant's heart rate or breathing, a TOF (Time of Flight) sensor for detecting information on the position (biological position) of the occupant, or a thermograph for measuring the surface temperature of the skin of the occupant. The occupant detection unit 41 may also include a wearable detector such as a wearable device that is worn by the occupant to detect the biological information of the occupant.

[0026] (1-6. Communication Devices) The communication device 31 is an interface for the electronic control device 50 to transmit and receive information to and from the external server 20. For example, the communication device 31 may be a communication interface that can access the server 20 via a mobile communication network. The communication device 31 is provided to communicate with the external server 20 in order to share the driving emotion database 61 accumulated in a certain vehicle among a plurality of vehicles, and may be omitted when the driving emotion database 61 is not shared.

[0027] (1-7. Vehicle driving control device) The vehicle driving control device 35 executes the driving control of the vehicle. The vehicle driving control device 35 includes one or more control devices that execute the driving control of the vehicle. For example, the vehicle driving control device 35 includes a control device that controls the driving of an engine, a power transmission mechanism including one or more driving motors and a transmission, a steering system, a brake system, and the like. In this embodiment, the vehicle driving control device 35 is configured to be able to execute automatic driving control of the vehicle. When the driving mode is set to the automatic driving mode, the vehicle driving control device 35 automatically controls at least a part of the driving control of the vehicle without the driver's operation, and drives the vehicle to the destination via a set driving route.

[0028] In addition, during the autonomous driving mode, the vehicle driving control device 35 receives commands from the electronic control device 50 and executes vehicle driving control. Specifically, the vehicle driving control device 35 executes autonomous driving control of the vehicle using control parameters transmitted from the electronic control device 50.

[0029] (1-8. Electronic control device) The electronic control device 50 is configured to include, for example, an arithmetic processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage element such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The arithmetic processing device executes various arithmetic processing by executing programs stored in the storage element. The electronic control device 50 may include a storage medium such as an HDD (Hard Disk Drive), a CD (Compact Disc), a DVD (Digital Versatile Disc), an SSD (Solid State Drive), a USB (Universal Serial Bus) flash, or a storage device in addition to or instead of the storage element. Note that a part or all of the electronic control device 50 may be configured with updatable firmware, or may be a program module executed by a command from a CPU, etc.

[0030] The electronic control unit 50 is connected to the occupant detection unit 41, the vehicle information detection unit 43, the input unit 45, the surrounding environment detection unit 47, the biometric information detection unit 49, and the vehicle driving control device 35 directly or via a communication line such as CAN (Controller Area Network) or LIN (Local Inter-Net).

[0031] In this embodiment, the electronic control device 50 includes a driving mode setting unit 51, a surrounding environment identification unit 53, an occupant emotion estimation unit 55, an occupant emotion learning unit 57, an emotion deterioration determination unit 59, a driving emotion database 61, an occupant emotion model 63, and a control parameter setting unit 65. Among these, the driving mode setting unit 51, the surrounding environment identification unit 53, the occupant emotion estimation unit 55, the occupant emotion learning unit 57, the emotion deterioration determination unit 59, and the control parameter setting unit 65 may be functions realized by the execution of a program by a calculation processing device. In addition, the driving emotion database 61 and the occupant emotion model 63 are composed of data stored in a storage unit.

[0032] (1-8-1. Operation mode setting section) The driving mode setting unit 51 switches the driving mode of the vehicle to a manual driving mode or an automatic driving mode based on an operation input signal transmitted from the input unit 45. The driving mode setting unit 51 may be configured to be able to set the level of the automatic driving mode. In the automatic driving assistance device 10 according to this embodiment, the electronic control unit 50 learns the emotions of the occupants while the vehicle is traveling in the manual driving mode to construct an occupant emotion model 63. Meanwhile, the electronic control unit 50 estimates the emotions of the occupants using the occupant emotion model 63 while the vehicle is traveling in the automatic driving mode, calculates an ideal driving state of the vehicle in which the emotions of the occupants are closer to a target emotion than the current emotion, and sets a control parameter for automatic driving control based on the ideal driving state.

[0033] (1-8-2. Surrounding environment identification part) The surrounding environment identification unit 53 identifies the surrounding environment of the vehicle based on the information on the surrounding environment transmitted from the surrounding environment detection unit 47. Specifically, the surrounding environment identification unit 53 identifies the position of the vehicle on a map, the positions and sizes of other vehicles including oncoming vehicles, bicycles, pedestrians, buildings, and other obstacles, the distance between these obstacles and the vehicle, the relative speed between the obstacles and the vehicle, etc. The surrounding environment identification unit 53 identifies the surrounding environment at a predetermined time interval according to the processing speed of the electronic control unit 50.

[0034] (1-8-3. Passenger emotion estimation unit) The occupant emotion estimation unit 55 estimates the emotion of the occupant based on the bioinformation transmitted from the bioinformation detection unit 49. For example, the occupant emotion estimation unit 55 may be configured to convert each piece of bioinformation, such as a heart rate or an electroencephalogram, transmitted from the bioinformation detection unit 49 into an index of each of pre-set emotions, such as "fear" or "comfortable". Specifically, the occupant emotion estimation unit 55 may be configured to associate each piece of bioinformation with a two-dimensional coordinate of each of pre-set emotions. Each emotion may be defined in two stages, positive or negative, or may be defined in multiple stages for each of the positive and negative sides. In this embodiment, each emotion is defined in four stages, and a total of nine stages are defined, including a neutral emotion in between.

[0035] In this specification, "positive" means a state of "good emotions" such as safety or security for each emotion, and "negative" means a state of "bad emotions."

[0036] (1-8-4. Passenger Emotion Learning Unit) The occupant emotion learning unit 57 learns the occupant emotion model 63 while the driving mode is set to the manual driving mode. The occupant emotion learning unit 57 constructs the occupant emotion model 63 based on the information on the occupant's emotion estimated by the occupant emotion estimation unit 55, the information on the driving state of the vehicle detected by the vehicle information detection unit 43, and the information on the surrounding environment of the vehicle identified by the surrounding environment identification unit 53. Specifically, the occupant emotion learning unit 57 accumulates a data set consisting of the information on the occupant's emotion, the information on the driving state of the vehicle and the information on the surrounding environment linked to the information on the occupant's emotion in terms of time in the driving emotion database 61, and constructs the occupant emotion model 63 using the data set.

[0037] 2 shows an example of the occupant emotion model 63. The occupant emotion model 63 is a learning model that receives information on the driving state of the vehicle and information on the surrounding environment as input and estimates occupant emotions as output. Since the data set used to construct the occupant emotion model 63 is a time-series data set, the occupant emotions can be sequentially estimated by inputting time-series data on the driving state of the vehicle and information on the surrounding environment to the occupant emotion model 63 as input.

[0038] The information on the driving state of the vehicle to be input data preferably includes time series data on acceleration in each of the forward / rearward, left / right, and up / down directions of the vehicle, angular velocities of the yaw angle, pitch angle, and roll angle, vehicle speed, steering angle, accelerator operation amount, and brake operation amount. Furthermore, the information on the driving state of the vehicle may include information such as engine speed, turn signal output, number of occupants, and attributes of each occupant. Furthermore, the information on the surrounding environment to be input data preferably includes data on the lane in which the vehicle is traveling, attributes of traffic participants such as other vehicles and pedestrians, relative distance, relative speed, and traveling direction, as well as data on traffic rules and traffic regulations such as the number of lanes on the road being traveled, traffic signal information, and speed limits. Furthermore, the information on the surrounding environment may include information such as weather, road surface conditions, and attributes of buildings.

[0039] The occupant emotion model 63 may receive inputs corresponding in number to the input data while maintaining the time order of the time series data. Alternatively, the occupant emotion model 63 may extract features from the time series data by defining a time window, for example, extracting the maximum value, minimum value, average value, etc. within the time window, and receive them as input data. In this case, the number of input data is proportional to the number of features extracted from the time series data. The occupant emotion model 63 outputs an estimated value (one value) of the occupant emotion.

[0040] The method for constructing the occupant emotion model 63 is not particularly limited, and any known method such as a support vector machine, a nearest neighbor method, a neural network such as deep learning, or a Bayesian network can be appropriately adopted.

[0041] (1-8-5. Emotional deterioration judgment section) The emotion deterioration determination unit 59 determines whether the emotion of the occupant has deteriorated while the driving mode is set to the automatic driving mode. In this embodiment, the emotion deterioration determination unit 59 determines whether the emotion level estimated by the occupant emotion estimation unit 55 is on the negative side of the target emotion level set via the input unit 45.

[0042] (1-8-6. Control parameter setting section) The control parameter setting unit 65 sets control parameters used for automatic driving control by the vehicle driving control device 35 while the driving mode is set to the automatic driving mode. The control parameter setting unit 65 calculates an ideal driving state of the vehicle in which the emotion of the occupant approaches a target level based on the occupant emotion model 63, and sets control parameters for automatic driving control based on the ideal driving state. In this embodiment, the control parameter setting unit 65 inputs a plurality of input values ​​related to the driving state of the vehicle, such as the vehicle speed, acceleration / deceleration, and steering angle, to the occupant emotion model 63, and sets the control parameters as the ideal driving state of the vehicle, using the input values ​​input to the occupant emotion model 63 that bring the emotion of the occupant closer to the target level than the current emotion.

[0043] The control parameters are variables used to calculate the control amount of each control target when the vehicle driving control device 35 executes automatic driving control of the vehicle. For example, the control parameters include values ​​of the vehicle speed, acceleration / deceleration, steering ratio, etc.

[0044] The target emotion level is set, for example, via the input unit 45. An occupant such as a driver may set the target emotion level in advance by operating the input unit 45. Alternatively, the control parameter setting unit 65 may always set the target emotion level to an emotion level that is one or more steps more positive than the emotion level of the occupant estimated by the occupant emotion estimation unit 55. The control parameter setting unit 65 may always set the highest emotion level on the positive side to the target level. The control parameter setting unit 65 may also set the target emotion level to a neutral emotion level when the emotion level of the occupant estimated by the occupant emotion estimation unit 55 is on the negative side.

[0045] Furthermore, when the emotion of the occupant estimated by the occupant emotion estimation unit 55 becomes negative when the vehicle is traveling under a predetermined situation, the control parameter setting unit 65 may ask the occupant about the target level of emotion under a similar situation and set it based on the reaction. For example, when the vehicle turns right despite being close to an oncoming vehicle, if the emotion of the occupant estimated from the biological information of the occupant by the occupant emotion estimation unit 55 changes to the negative side compared to the emotional state before the right turn, the control parameter setting unit 65 asks about the target level of emotion under the same situation. The content of the question is set in advance, and when the emotion changes to the negative side, it is played through an in-vehicle speaker or the like.

[0046] Specifically, the control parameter setting unit 65 asks the occupant, "Would you prefer driving that is not scary in the current situation?" or "Would you prefer driving that places importance on timing even if you feel a little anxious in the current situation?" If the occupant responds affirmatively to "Would you prefer driving that is not scary in the current situation?", the control parameter setting unit 65 sets the target emotion level to neutral. Also, if the occupant responds affirmatively to "Would you prefer driving that places importance on timing even if you feel a little anxious in the current situation", the control parameter setting unit 65 sets the target emotion level to an emotion level that is one or more steps higher than the emotion level estimated at that time.

[0047] Further, the control parameter setting unit 65 prepares a plurality of candidates for input values ​​to be input to the occupant emotion model 63, and sequentially inputs the plurality of input values ​​to the occupant emotion model 63. The control parameter setting unit 65 selects an input value corresponding to an emotion level that is closest to a target level from among a plurality of emotion levels output from the occupant emotion model 63 based on the plurality of input values. Then, the control parameter setting unit 65 sets a control parameter based on the selected input value, with the input value being regarded as an ideal driving state. In this way, automatic driving control of the vehicle is executed so that the emotion of the occupant approaches the target level. Note that, when the plurality of input values ​​that are closest to the target level exist, the input value that is closest to the current control parameter may be selected from among the input values.

[0048] 3 shows an example of an algorithm for setting vehicle control parameters using the occupant emotion model 63. The input values ​​for determining the control parameters consist of input values ​​related to the vehicle's operation state and behavior during autonomous driving control. Since the input values ​​for the surrounding environment are information (fixed values) that cannot be controlled, the information (fixed values) of the surrounding environment detected by the surrounding environment detection unit 47 is used.

[0049] The number of candidates for the input values ​​to be prepared is appropriately set based on at least one of the processing speed of the arithmetic processing device constituting the electronic control device 50 and the update frequency of the control parameters. For example, the number of candidates for the input values ​​to be prepared is set in advance as follows. First, the number of calculations per unit time for inputting the input values ​​to the occupant emotion model 63 and outputting the emotions of the occupant is calculated based on the data processing amount required for the occupant emotion model 63 and the processing capacity of the arithmetic processing device that performs arithmetic processing using the occupant emotion model 63. Based on the calculated number of calculations and the update speed of the control parameters, the number of calculations that can be processed each time the control parameters are updated is calculated. This number of calculations is set as the number of candidates for the input values. In order to precisely execute the automatic driving control, it is preferable to set the maximum number that can be set in consideration of the processing speed of the arithmetic processing device and the update frequency of the control parameters.

[0050] The control parameter setting unit 65 sequentially inputs a plurality of input values ​​to the occupant emotion model 63 to obtain each emotion level, and selects the input value that is closest to the target level. Then, the control parameter setting unit 65 sets the control parameters based on the selected input value, regarding the input value as an ideal driving state. Specifically, the control parameter setting unit 65 extracts, from the data sets stored in the driving emotion database 61, a data set of the vehicle driving state when the emotion level matches or is similar to the information of the current surrounding environment detected by the surrounding environment detection unit 47 and is closest to the target level of the emotion that has been set. The control parameter setting unit 65 sets the extracted data set of the vehicle driving state as the reference operation target value.

[0051] The control parameter setting unit 65 creates multiple input value candidates corresponding to the above number of operations between the current value and the above reference operation target value, for information on the driving state of the vehicle, such as steering angle and acceleration. For example, for vehicle speed data, if the current vehicle speed is 40 km / h, the reference operation target value is 30 km / h, and the number of operations is 3, the input value candidates for the vehicle speed are 30 km / h, 35 km / h, and 40 km / h. The control parameter setting unit 65 executes the same process for all other data to create multiple input value candidates. Note that the reason for preparing multiple input value candidates is that if the information on the surrounding environment of the data set set as the reference operation target value does not completely match the current surrounding environment, the reference operation target value does not necessarily represent a driving state of the vehicle that can realize the target level of emotion.

[0052] The control parameter setting unit 65 sequentially inputs the created multiple input values ​​to the occupant emotion model 63, and selects a data set of the vehicle driving state in which the output emotion of the occupant is closest to the target level. Then, the control parameter setting unit 65 sets the control parameters for the automatic driving control, such as the vehicle speed, acceleration / deceleration, and steering rate, based on the selected data set. As described above, the number of input value candidates is set based on at least one of the processing speed of the arithmetic processing device and the update frequency of the control parameters, so that the control parameter setting unit 65 can sequentially update the control parameters for each processing cycle of the arithmetic processing device. The control parameter setting unit 65 executes the setting of the control parameters using the occupant emotion model 63 for each processing cycle of the arithmetic processing device, and transmits information on the set control parameters to the vehicle driving control device 35. As a result, the control parameters for the automatic driving control that bring the emotion of the occupant to the target level are sequentially obtained, and automatic driving control that is comfortable for the occupant can be executed.

[0053] <2. Example of operation> So far, a configuration example of the automatic driving support device 10 according to this embodiment has been described. Next, an operation example of the automatic driving support device 10 according to this embodiment will be described based on the flowcharts shown in Fig. 4 to Fig. 6. In the following description, the description of the contents already described in the above configuration example may be omitted.

[0054] First, when the electronic control device 50 of the automatic driving assistance device 10 detects the start of the system (step S11), the occupant detection unit 41 recognizes the occupant of the vehicle (step S13). In addition, the driving mode setting unit 51 sets the driving mode of the vehicle to the manual driving mode or the automatic driving mode according to the driving mode input via the input unit 45 (step S15).

[0055] Next, the driving mode setting unit 51 determines whether the driving mode is an automatic driving mode or not (step S17). If the driving mode is not the automatic driving mode (S17 / No), that is, if the driving mode is set to the manual driving mode, the occupant emotion estimation unit 55 estimates the emotion of the occupant based on the biometric information transmitted from the biometric information detection unit 49 (step S27). In this embodiment, the level of the emotion of the occupant defined in a plurality of stages is estimated. The emotion of the occupant estimated at this time is stored as the emotion information at the start of driving. Next, the vehicle driving control device 35 starts the driving control of the vehicle according to the driving operation of the driver to start the vehicle driving (step S29). After starting the vehicle driving in the manual driving mode, the electronic control device 50 executes the learning process of the occupant emotion model 63 by the occupant emotion learning unit 57 (step S31). Until the system is stopped (until a positive determination is made in step S25), the electronic control unit 50 repeatedly executes the learning process of the occupant emotion model 63 while the vehicle is set in the manual driving mode.

[0056] 5 is a flowchart showing an example of an occupant emotion model learning process by the automatic driving assistance device 10. First, the surrounding environment identification unit 53 identifies information on the current surrounding environment of the vehicle based on the information on the surrounding environment transmitted from the surrounding environment detection unit 47 (step S41). Next, the vehicle information detection unit 43 detects the current driving state of the vehicle (step S43). Next, the occupant emotion estimation unit 55 estimates the emotion of the occupant based on the bio-information transmitted from the bio-information detection unit 49 (step S45).

[0057] Next, the occupant emotion learning unit 57 acquires information on the emotion of the occupant, information on the surrounding environment, and information on the driving state of the vehicle, and accumulates the information on the emotion of the occupant, information on the surrounding environment linked to the information on the emotion of the occupant in terms of time, and information on the driving state of the vehicle in the driving emotion database 61 (step S47). Next, the occupant emotion learning unit 57 uses the information on the emotion of the occupant, information on the surrounding environment linked to the information on the emotion of the occupant in terms of time, and information on the driving state of the vehicle to build or update the occupant emotion model 63 using a known method such as deep learning (step S49). The occupant emotion learning unit 57 repeatedly executes the processes from step S41 to step S49 for each processing cycle of the arithmetic processing device, and builds the occupant emotion model 63.

[0058] On the other hand, in the above step S17, if the driving mode is the automatic driving mode (S17 / Yes), the occupant emotion estimation unit 55 estimates the emotion of the occupant based on the biometric information transmitted from the biometric information detection unit 49 (step S19). The emotion of the occupant estimated at this time is stored as emotion information at the start of driving. Next, the vehicle driving control device 35 starts the driving control of the vehicle in the automatic driving mode to start the vehicle driving (step S21). After starting the vehicle driving in the automatic driving mode, the electronic control device 50 executes the driving control of the vehicle based on the emotion of the occupant (step S23). Until the system is stopped (until the affirmative determination is made in step S25), the electronic control device 50 continues the driving control of the vehicle based on the emotion of the occupant while the vehicle is set to the automatic driving mode.

[0059] FIG. 6 shows an example of a flowchart of the driving control process based on the driving emotion by the automatic driving assistance device 10. First, the surrounding environment specification unit 53 specifies information on the current surrounding environment of the vehicle based on the information on the surrounding environment transmitted from the surrounding environment detection unit 47 (step S51). Next, the occupant emotion estimation unit 55 estimates the emotion of the occupant based on the bio-information transmitted from the bio-information detection unit 49 (step S53). Next, the emotion deterioration determination unit 59 determines whether the estimated emotion of the occupant has deteriorated (step S55). In this embodiment, the emotion deterioration determination unit 59 determines whether the level of the estimated emotion of the occupant has changed to the negative side compared to the previous time.

[0060] If the occupant's emotion has not worsened (S55 / No), the automatic driving assistance device 10 returns to step S51 and repeats the processes of steps S51 to S55. On the other hand, if the occupant's emotion has worsened (S55 / Yes), the control parameter setting unit 65 refers to the set target emotion value (step S57). In this embodiment, the information on the set target emotion level is referred to.

[0061] Next, the control parameter setting unit 65 refers to the driving emotion database 61 and creates multiple candidates of input values ​​to be input to the occupant emotion model 63 (step S59). Specifically, the control parameter setting unit 65 refers to the driving emotion database 61 and extracts a data set of the vehicle driving state in which the occupant's emotion is closest to the target level from the data set of the surrounding environment that matches or is similar to the current surrounding environment. Next, the control parameter setting unit 65 sets the extracted data set of the vehicle driving state as a reference operation target value, and creates multiple candidates of input values ​​corresponding to a preset number of operations between the value of the current vehicle driving state and the reference operation target value.

[0062] Next, the control parameter setting unit 65 inputs one of the candidates of the input value to the occupant emotion model 63 and acquires the information of the emotion of the occupant to be output (step S61). Next, the control parameter setting unit 65 calculates the difference between the value of the emotion of the occupant output from the occupant emotion model 63 and the target emotion value (step S63). In this embodiment, the control parameter setting unit 65 calculates the difference between the emotion level output from the occupant emotion model 63 and the target emotion level. Next, the control parameter setting unit 65 determines whether or not the input of all the candidates of the input value to the occupant emotion model 63 is completed (step S65). If the input of all the candidates of the input value is not completed (S65 / No), the control parameter setting unit 65 returns to step S61 and inputs the next candidate of the input value to the occupant emotion model 63. The control parameter setting unit 65 repeatedly executes the processing of steps S61 to S65 until the input of all the candidates of the input value is completed.

[0063] When the input of all the candidate input values ​​is completed (S65 / Yes), the control parameter setting unit 65 selects the input value that minimizes the difference between the emotion level output from the occupant emotion model 63 and the target emotion level (step S67). The input value selected here is the input value that brings the emotion level of the occupant closest to the target level. Next, the control parameter setting unit 65 converts the selected input value into a control parameter for automatic driving control (step S69). Next, the control parameter setting unit 65 transmits the calculated control parameter to the vehicle driving control device 35 and reflects it in the automatic driving control (step S71).

[0064] When driving control based on occupant emotion is executed during the automatic driving mode (step S23), or when learning processing of the occupant emotion model 63 is executed during the manual driving mode (step S31), the automatic driving assistance device 10 determines whether the system has stopped (step S25). When the system has not stopped (S25 / No), the automatic driving assistance device 10 repeats the above-mentioned processing from step S17 to step S31. On the other hand, when the system has stopped (S25 / Yes), the automatic driving assistance device 10 ends the control processing.

[0065] As described above, while the driving mode is set to the manual driving mode, the automatic driving assistance device 10 according to this embodiment accumulates information on the emotions of the occupants, information on the driving state of the vehicle linked in time to the information on the emotions of the occupants, and information on the surrounding environment in the driving emotion database 61, and uses these pieces of information to construct the occupant emotion model 63. Furthermore, while the driving mode is set to the automatic driving mode, the automatic driving assistance device 10 refers to the driving emotion database 61 to create multiple candidates for the input value of the driving state of the vehicle to be input to the occupant emotion model 63, and sets the control parameters for automatic driving control based on the input value that most closely approximates the emotion level output from the occupant emotion model 63 to the target level.

[0066] The driving emotion database 61 is a data set of information linked over time, and the control parameter setting unit 65 sets the control parameters based on the input values ​​that bring the emotion level of the occupant closer to the target level from among multiple candidates of the input values ​​of the driving state of the vehicle for each predetermined processing cycle. Therefore, it is possible to specifically and sequentially set the control parameters that can bring the emotion of the occupant closer to the target emotion.

[0067] In addition, the control parameter setting unit 65 sets the control parameters using a data set of the surrounding environment that matches or is similar to the surrounding environment of the current vehicle from the driving emotion database 61. Therefore, even if there is no data set that matches the surrounding environment in the driving emotion database 61, it is possible to realize driving control that leads to a state close to the target emotion.

[0068] <3. Modifications> Although the automatic driving assistance device 10 according to this embodiment has been described so far, various modifications are possible to the automatic driving assistance device 10 according to this embodiment. Below, a modified example in which the control parameter setting unit 65 sets the control parameters further in accordance with each individual occupant will be described.

[0069] The first modified example is an example in which an occupant predetermines the value of one or more data items to be prioritized among the data items of the multiple input value candidates to be input to the occupant emotion model 63. For example, if a certain occupant places importance on the vehicle speed in the automatic driving control, the occupant predetermines the vehicle speed as the priority item. When creating multiple input value candidates, the control parameter setting unit 65 fixes the vehicle speed data to the vehicle speed constituting the reference operation target value extracted from the driving emotion database 61, and creates input value candidates for the other data items. This makes it possible to set control parameters that bring the occupant's emotions closer to the target emotions while reflecting the occupant's preferences regarding the driving state of the vehicle.

[0070] The second modified example is an example in which the control parameter setting unit 65 extracts data items of the driving state of the vehicle that have a large effect on the emotions for each occupant based on the driving emotion database 61, and creates input value candidates for the occupant by fixing the data items. For example, the occupant detection unit 41 identifies and recognizes each occupant, and creates the driving emotion database 61 in which a data set is accumulated for each occupant. Then, the occupant emotion learning unit 57 analyzes the sensitivity to the occupant emotion model 63 and extracts data items that have a large effect on the emotions. The analysis of the sensitivity may be, for example, a method of analyzing data items whose output emotion level changes significantly when one item is removed or the value of one item is changed among the input data items. The control parameter setting unit 65 fixes the specified data items to the input values ​​constituting the reference operation target values ​​extracted from the driving emotion database 61, and creates input value candidates for the other data items. This makes it possible to set control parameters that bring the occupant's emotions closer to the target emotions while preferentially setting input values ​​for data items that have a large effect on the emotions of each occupant.

[0071] Furthermore, when the control parameters calculated using the above-mentioned first variant method are reflected in the automatic driving control, if the occupant's emotional level estimated based on the biometric information detected by the biometric information detection unit 49 is more negative than the emotional level calculated using the occupant emotion model 63, the occupant may be suggested to set the items that have a large impact on the emotions obtained using the second variant method as priority data items.

[0072] According to these modified examples, the automatic driving control using the occupant emotion model 63 can reflect at least one of the occupant preferences or objective tendencies based on the accumulated driving emotion database 61, and automatic driving control can be executed that reflects the intentions and characteristics of each occupant.

[0073] Although the preferred embodiment of the present invention has been described in detail above with reference to the accompanying drawings, the present invention is not limited to such an example. It is clear that a person having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present invention.

[0074] For example, in the above embodiment, the occupant emotion learning unit 57 accumulates a data set in the driving emotion database 61 and updates the occupant emotion model 63 when the driving mode is the manual driving mode, but the present invention is not limited to such an example. The occupant emotion learning unit 57 may accumulate a data set in the driving emotion database 61 and update the occupant emotion model 63 even when the driving mode is the automatic driving mode. [Explanation of symbols]

[0075] 10...automatic driving assistance device, 35...vehicle driving control device, 50...electronic control device, 51...driving mode setting unit, 53...surrounding environment identification unit, 55...occupant emotion estimation unit, 57...occupant emotion learning unit, 59...emotion deterioration determination unit, 61...driving emotion database, 63...occupant emotion model, 65...control parameter setting unit

Claims

1. an occupant emotion learning unit that constructs an occupant emotion model based on information on the vehicle's surrounding environment, information on a driving state of the vehicle, and information on an occupant's emotion, the occupant emotion model estimating the emotion of the occupant from information on the vehicle's surrounding environment and information on the driving state of the vehicle; a control parameter setting unit that calculates an ideal driving state of the vehicle in which the emotion of the occupant approaches a target emotion based on the occupant emotion model, and sets a control parameter for automatic driving control based on the ideal driving state; The occupant emotion learning unit accumulates a data set linking information on the estimated emotion of the occupant, information on the surrounding environment of the vehicle, and information on the driving state of the vehicle; the control parameter setting unit extracts, from the accumulated data set, a driving state of the vehicle that approaches a target emotion in an ambient environment corresponding to the current ambient environment of the vehicle, creates a plurality of input values ​​based on the extracted information on the driving state of the vehicle, and inputs the plurality of input values ​​to the occupant emotion model, and sets the control parameter as an ideal driving state of the vehicle, an input value among the input values ​​input to the occupant emotion model that brings the emotion of the occupant in the ambient environment corresponding to the current ambient environment of the vehicle closer to the target emotion than the current emotion; The data items of the vehicle's operating state include a plurality of data items, At least one of the plurality of data items can be set by a user as a priority item; The control parameter setting unit, when there is a priority item set by the user, fixes the value of the priority item among the data items of the multiple input values ​​to be input to the occupant emotion model to a value extracted as a driving state of the vehicle that approaches a target emotion in an ambient environment corresponding to the current ambient environment of the vehicle, and creates the multiple input values ​​for the data items other than the priority item.

2. The automatic driving assistance device of claim 1, wherein, when the emotions of the occupant when the automatic driving control is executed using the control parameters obtained by fixing the values ​​of the priority items become worse than the emotions of the occupant calculated using the occupant emotion model, the control parameter setting unit presents to the user that at least one of the data items that has a large impact on the emotions of the occupant is to be fixed to a value extracted as a driving state of the vehicle that approaches the target emotion in an ambient environment corresponding to the current ambient environment of the vehicle.

3. 3. The autonomous driving assistance device according to claim 1, wherein the control parameter setting unit sets the plurality of input values ​​between values ​​of a predetermined data item in the extracted driving state of the vehicle and values ​​of the predetermined data item in a current driving state of the vehicle.

4. an occupant emotion learning unit that constructs an occupant emotion model based on information on the vehicle's surrounding environment, information on a driving state of the vehicle, and information on an occupant's emotion, the occupant emotion model estimating the emotion of the occupant from information on the vehicle's surrounding environment and information on the driving state of the vehicle; a control parameter setting unit that calculates an ideal driving state of the vehicle in which the emotion of the occupant approaches a target emotion based on the occupant emotion model, and sets a control parameter for automatic driving control based on the ideal driving state; The occupant emotion learning unit accumulates a data set linking information on the estimated emotion of the occupant, information on the surrounding environment of the vehicle, and information on the driving state of the vehicle; the control parameter setting unit extracts, from the accumulated data set, a driving state of the vehicle that approaches a target emotion in an ambient environment corresponding to the current ambient environment of the vehicle, creates the plurality of input values ​​based on information on the extracted driving state of the vehicle, and inputs the plurality of input values ​​to the occupant emotion model, and sets the control parameter as an ideal driving state of the vehicle, an input value among the input values ​​input to the occupant emotion model that brings the emotion of the occupant in the ambient environment corresponding to the current ambient environment of the vehicle closer to the target emotion than the current emotion; The data items of the vehicle's operating state include a plurality of data items, The occupant emotion learning unit extracts at least one of the data items having a large effect on the emotion of the occupant based on the accumulated data set, The control parameter setting unit fixes the at least one data item that has a large impact on the emotions of the occupant to a value extracted as a driving state of the vehicle that approaches the target emotion in an ambient environment corresponding to the current ambient environment of the vehicle, and creates the multiple input values ​​for the data items other than the at least one data item.

5. an occupant emotion learning unit that constructs an occupant emotion model that estimates the emotion of an occupant from a driving state of the vehicle based on information on the driving state of the vehicle and the emotion of the occupant; a control parameter setting unit that calculates an ideal driving state of the vehicle in which the emotion of the occupant approaches a target emotion based on the occupant emotion model, and sets a control parameter for automatic driving control based on the ideal driving state; The control parameter setting unit sets the number of multiple input values ​​related to the driving state of the vehicle to be input to the occupant emotion model based on at least one of the processing speed of a calculation processing device that performs calculations and the update frequency of the control parameters, inputs the set number of the multiple input values ​​to the occupant emotion model, and sets the control parameter as an ideal driving state of the vehicle using an input value among the input values ​​input to the occupant emotion model that brings the emotion of the occupant closer to the target emotion than the current emotion.

Images