Driving assistance devices

Abstract

To improve a feeling of an occupant from an unpleasant feeling to a pleasant feeling by giving priority to the feeling of the occupant at a seating position where the unpleasant feeling is likely to be generated.SOLUTION: A driving assistance device is provided in a moving body having a plurality of seating positions and includes a processor. The processor is configured: to set a priority order of seating positions in a case of performing improvement control of the moving body with respect to a feeling of an occupant riding in the moving body; to input at least one of a captured image and biological information of the occupant to a learning model as an input parameter; to output a feeling estimation result of the occupant as an output parameter; to select, in accordance with the priority order of the seating positions, an occupant who is seated at a seating position for which the priority order is set high among occupants who need to be dealt with by improvement control; to determine improvement control for the moving body, on the basis of the selected feeling estimation result of the occupant; and to control the moving body, on the basis of the determined improvement control.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a driving assistance device. [Background technology]

[0002] Patent Document 1 proposes a technology for displaying passenger emotions for each seating position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-216241 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 does not consider which emotion should be improved when there are multiple occupants. Therefore, it is unclear which occupant's emotion should be used as the basis for vehicle control. For this reason, there is a need for a technology that can determine which occupant's emotion should be prioritized.

[0005] The present disclosure has been made in consideration of the above, and its purpose is to provide a driving assistance device that can prioritize the feelings of occupants in seating positions where unpleasant feelings are likely to occur, and improve the feelings of occupants from unpleasant to pleasant. [Means for solving the problem]

[0006] The driving assistance device according to the present disclosure is a driving assistance device that is provided in a moving body having a plurality of seating positions and that includes a processor, wherein when the processor performs improvement control of the moving body in response to the emotions of occupants aboard the moving body, it sets priorities for the plurality of seating positions, acquires at least one of an image and biometric information of the occupant and inputs it as an input parameter to a learning model, outputs an emotion estimation result of the occupant as an output parameter, selects an occupant who is seated in a seating position with a high priority from among occupants who need to be addressed by improvement control in accordance with the priority for the seating positions, determines the improvement control for the moving body based on the emotion estimation result of the selected occupant, and controls the moving body based on the determined improvement control. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to improve the occupant's feelings from discomfort to comfort by prioritizing the feelings of the occupant in a seating position where discomfort is likely to occur. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a driving assistance system according to one embodiment. [Figure 2] FIG. 2 is a diagram illustrating a flow for determining the priority order of vehicle occupants in the driving assistance system according to one embodiment. [Figure 3] FIG. 3 is a flowchart illustrating an information processing method executed in a vehicle according to an embodiment. [Figure 4] FIG. 4 is a diagram showing a first example of a method for setting priority levels in an information processing method according to an embodiment. [Figure 5] FIG. 5 is a diagram showing another example of the first embodiment of the method for setting priority levels in the information processing method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings of the following embodiment, the same or corresponding parts are designated by the same reference numerals. Furthermore, the present disclosure is not limited to the embodiments described below.

[0010] In one embodiment of the present disclosure, a learning model is used to estimate the emotions of an occupant in a vehicle that is a moving object traveling on a road. The learning model is also referred to as a trained model, a learned model, or simply a model. In this embodiment, when vehicle control is performed to improve the occupant's emotions, a priority of the vehicle control for improvement is set according to the seating position.

[0011] A driving assistance device mounted on a vehicle as a moving body includes a control unit. The control unit has an emotion estimation unit configured with a learning model. The emotion estimation unit functions as a processing unit that estimates and determines at least the driver's emotion of pleasure or discomfort based on the facial expressions of occupants including the driver and vital information such as heart rate and sweating. The emotion estimation unit can estimate, for example, three levels of emotion. The three levels of emotion include, for example, positive (pleasant), negative (unpleasant), and normal (normal). The emotion estimation unit can also estimate, for example, four or more levels of emotion. When four or more levels of emotion are used, at least positive (pleasant) and negative (unpleasant) emotions are included. The level at which the emotion hierarchy is set can be changed as appropriate.

[0012] The estimation result of the emotion estimation unit in the driving assistance device may include a numerical index of comfort or discomfort, a probability of a specific emotion (e.g., sleepiness, discomfort due to car sickness), etc. The emotion estimation unit can estimate emotions based on image data captured by a camera, for example.

[0013] The control unit has a seating priority setting unit configured by a learning model or a rule base. The seating priority setting unit functions as a setting unit that sets priorities for seating positions of occupants in the vehicle based on emotion estimation result information (emotion estimation result information) generated by the emotion estimation unit. The control unit has a vehicle control unit including, for example, an ECU. The vehicle control unit outputs control signals for controlling the vehicle to each unit in accordance with vehicle control parameters. The control unit also has a control decision unit that determines control content to be performed by the vehicle control unit. The control decision unit is a decision unit that determines control based on emotion estimation result information of the occupant sitting in the position prioritized by the seating priority setting unit. When the decision information determined by the control decision unit is input, the vehicle control unit controls the vehicle by transmitting signals based on the determined control to each unit of the vehicle. In other words, the vehicle control content to be performed by the vehicle control unit is not fixed, but is changed depending on the emotion of the occupant whose priority is set. Note that the emotion estimation unit may perform emotion estimation based on both image data obtained by capturing an expression of the occupant and biometric data of the occupant. Emotion estimation may be performed based on both the image data (facial expression) and biometric data of the occupant.

[0014] Next, a driving assistance system for assisting driving of a moving object equipped with a driving assistance device according to an embodiment of the present disclosure based on the above principles will be described. Fig. 1 shows a driving assistance system according to an embodiment. Fig. 1 is a block diagram showing a driving assistance system 1 according to an embodiment.

[0015] As shown in FIG. 1, the driving assistance system 1 includes a vehicle 4 that is capable of communicating with the outside world via a network 2 and transmitting and receiving information. The vehicle 4 is capable of outputting information related to emotions to the network 2 using a driving assistance device 40 installed in the vehicle 4. The network 2 is configured from an internet network, a mobile phone network, or the like. The network 2 is, for example, a public communication network such as the internet, and may also include other communication networks such as a wide area network (WAN), a telephone communication network such as a mobile phone, or a wireless communication network such as WiFi (registered trademark).

[0016] (vehicle) The vehicle 4, which serves as a mobile object capable of communicating with the network 2, may be a vehicle driven by a driver. It is also possible to use a semi-autonomous or autonomous vehicle 4 capable of autonomous driving according to driving instructions provided by a predetermined program or the like. The vehicle 4 can be driven toward a desired destination by a user operating the steering wheel or the like. Mobile objects other than the vehicle 4 may also be used. Mobile objects other than the vehicle 4 include light vehicles such as motorcycles and other vehicles that travel on roads. In other words, in this embodiment, examples of mobile objects that can be used include electric motorcycles, such as motorcycles equipped with a motor and a battery, bicycles, or kick scooters, tricycles, buses, and trucks.

[0017] The vehicle 4 includes a driving assistance device 40 as an information processing device that is an on-board device, a sensor group 44, a communication unit 45, a positioning unit 46, and a drive unit 47. The driving assistance device 40 includes a control unit 41, a storage unit 42, and an alarm unit 43. The driving assistance device 40 has the configuration of a general computer that is capable of communication via the network 2. The driving assistance device 40 is an on-board device that is mounted on the vehicle 4. The driving assistance device 40 is capable of in-vehicle communication with the sensor group 44, the communication unit 45, the positioning unit 46, and the drive unit 47. The sensor group 44 includes a cabin camera 441 and a wearable device 442.

[0018] Specifically, the control unit 41 having hardware includes a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field-Programmable Gate Array), and a main memory unit such as a RAM (Random Access Memory) or a ROM (Read Only Memory).

[0019] The control unit 41 comprehensively controls the operations of various components mounted on the vehicle 4. The control unit 41 further loads a program stored in the storage unit 42 into a work area of ​​the main storage unit and executes it, and through the execution of the program, can realize the functions of an emotion estimation unit 411, a seating priority setting unit 412, a control decision unit 413, a vehicle control unit 414, and a model generation unit 415.

[0020] The storage unit 42 is configured with a storage medium selected from an EPROM (Erasable Programmable ROM), a hard disk drive (HDD, Hard Disk Drive), and removable media. Examples of removable media include a USB (Universal Serial Bus) memory and a disk recording medium. Examples of disk recording media include a CD (Compact Disc), a DVD (Digital Versatile Disc), and a BD (Blu-ray (registered trademark) Disc). The storage unit 42 can store various programs, various tables, various databases, and the like. The various programs are, for example, an operating system (OS).

[0021] The control unit 41 loads the program stored in the storage unit 42 into a work area of ​​the main storage unit and executes it. The control unit 41 can realize various functions of the control unit 41 through the execution of the program. Specifically, the control unit 41 can realize the functions of an emotion estimation unit 411, a seating priority setting unit 412, a control decision unit 413, a vehicle control unit 414, and a model generation unit 415 through the execution of the program.

[0022] The memory unit 42 stores an emotion estimation model 421 for implementing the emotion estimation unit 411. The emotion estimation unit 411 of the control unit 41 estimates the emotion of the occupant using various information, such as image data and vital sign data, acquired from the sensor group 44 of the vehicle 4. The emotion estimation unit 411 estimates whether the occupant is experiencing a negative emotion using the emotion estimation model 421. The emotion estimation unit 411 can estimate the emotion of the occupant, etc., based on the emotion estimation model 421, which is a learning model or a pre-trained model trained in advance by machine learning. When the emotion estimation model 421 is used, input parameters are, for example, image data of the occupant and vital sign data of the occupant. The output parameter is, for example, a probability value of the occupant's comfort or discomfort. In other words, the emotion estimation unit 411 is configured to be able to determine the emotion of the occupant using a learning model and artificial intelligence (AI) based on the captured image data of the occupant obtained by capturing an image of the interior of the vehicle 4 and vital sign data including the occupant's biological information measured by the sensor group 44. Accordingly, the emotion estimation unit 411 can estimate the emotions of the occupant while prioritizing them from the captured image data and vital data of the occupant in accordance with the priority order.

[0023] The memory unit 42 stores a seating setting model 422 for realizing the seating priority setting unit 412. The seating priority setting unit 412 uses the seating setting model 422 to set the priority of seating positions based on the seating positions of the user in the cabin of the vehicle 4. The seating priority setting unit 412 can set the priority of seating positions based on the seating setting model 422. The seating setting model 422 is a learned model or a trained model that has been trained in advance by machine learning. When the seating setting model is used, the input parameter is, for example, an estimation result of the emotion of the occupant (emotion estimation result). The output parameter is, for example, ranking information of emotion changes for each seat position in the cabin. The seating priority setting unit 412 may also be configured to set the priority based on a rule base.

[0024] The control decision unit 413 decides the control for the drive unit 47. The vehicle control unit 414 comprehensively controls the operations of various components mounted on the vehicle 4. Specifically, the vehicle control unit 414 is composed of an ECU (Electronic Control Unit) that controls each component of the vehicle 4, such as the engine, electric motor, and steering device, and is configured to be able to control the vehicle speed, steering angle, etc. of the vehicle 4. The vehicle control unit 414 controls the vehicle speed by adjusting the throttle opening of the engine and the braking force of the brakes. The vehicle control unit 414 steers the vehicle 4 by adjusting the steering angle of the wheels that rotate during driving. With this configuration, the vehicle control unit 414 controls the drive unit 47 of the vehicle 4 based on the control decided by the control decision unit 413.

[0025] The model generation unit 415 generates the emotion estimation model 421 and the seating setting model 422, which are learning models, by machine learning. The learned model or learning model can be generated by machine learning, such as deep learning using a neural network, using an input / output data set of predetermined input parameters and output parameters as training data. For example, in supervised learning to generate the seating setting model 422, the user's emotion estimation result is used as a learning input parameter, and the priority of the seating position is used as a learning output parameter. The model generation unit 415 can generate the seating setting model 422 by using these input / output data sets as training data.

[0026] The sensor group 44 includes, for example, a cabin camera 441 capable of capturing images of various conditions inside the vehicle cabin. The sensor group 44 also includes a wearable device 442 worn by an occupant in the vehicle 4. The wearable device 442 detects the state of the occupant by detecting vital information such as the occupant's body temperature, pulse, brain waves, blood pressure, and sweating level. The detected vital information is output to the driving assistance device 40. The sensor group 44 may further include sensors related to the running of the vehicle 4, such as a vehicle speed sensor, an acceleration sensor, and a fuel sensor. Sensor information detected by the cabin camera 441, wearable device 442, and various sensors constituting the sensor group 44 is output to the driving assistance device 40 via a vehicle information network (CAN: Control Area Network) consisting of transmission paths connected to the various sensors.

[0027] The positioning unit 46, which serves as a position information acquisition unit, receives radio waves from GPS satellites using a GPS (Global Positioning System) sensor to detect the position of the vehicle 4. The detected position and travel route are searchably stored in the storage unit 42 as position information and travel route information in the travel information.

[0028] The sensor group 44 and the positioning unit 46 output various information to the driving assistance device 40 sequentially, as needed, or at predetermined timing. Among the various types of information, vehicle information as moving object information includes vehicle identification information and sensor information. Among the various types of information, the sensor information includes imaging information obtained by capturing images of occupants in the vehicle cabin of the vehicle 4. Among the various types of information, driving information as movement information includes information related to driving, such as the driving route and location information of the vehicle 4. User information includes user identification information and personal information. Examples of user identification information include information for mutually identifying users, such as the driver of the vehicle 4 and occupants aboard the vehicle 4. Note that users whose emotions are monitored are primarily occupants other than the driver, but the driver may also be included in the users whose emotions are monitored. Examples of user state information include information related to the user. Examples of personal information include user-specific information such as name, age, address, date of birth, and age, as well as behavioral pattern information such as driving history. Note that the information listed above is not necessarily limited to the exemplified information.

[0029] The drive unit 47 is composed of a plurality of drive devices required for the travel of the vehicle 4. Specifically, the vehicle 4 is equipped with an engine as a drive source, and the engine is configured to be capable of generating electricity using an electric motor or the like when driven by the combustion of fuel. The generated electricity is charged into a rechargeable battery. Furthermore, the vehicle 4 is equipped with a drive transmission mechanism that transmits the driving force of the engine, drive wheels for travel, and the like.

[0030] The notification unit 43 is configured to be able to notify a user in the vehicle of predetermined information. The communication unit 45 transmits various information to an external device, such as a server or another vehicle 4, via the network 2. The map database stored in the storage unit 42, the notification unit 43, the communication unit 45, and the positioning unit 46 constitute a car navigation system.

[0031] (Vehicle control method based on seating position priority) Next, an information processing method executed by the driving assistance system 1 will be described. The information processing method according to this embodiment is a vehicle control method based on seating position. FIG. 2 is a diagram for explaining a flow for estimating the emotions of an occupant of the vehicle 4 and controlling the vehicle 4 in the driving assistance system 1 according to this embodiment. FIG. 3 is a flowchart for explaining the information processing method according to this embodiment. Note that FIGS. 2 and 3 will be used together in the following description. The flowchart shown in FIG. 3 is repeatedly executed while an occupant is present in the vehicle 4.

[0032] 2 and 3, first, in step ST1, the cabin camera 441 of the sensor group 44 of the vehicle 4 captures images of occupants and the like in the cabin of the vehicle 4. The sensor group 44 outputs the captured image data (image data) to the control unit 41. Similarly, the wearable device 442 of the sensor group 44 measures biological information of the occupants and the like. The wearable device 442 outputs the measured biological information to the control unit 41 as vital data. The control unit 41 stores the acquired image data and vital data in the storage unit 42.

[0033] Next, the process proceeds to step ST2, where the emotion estimation unit 411 of the control unit 41 reads out the image data, vital data, and emotion estimation model 421 from the storage unit 42. The emotion estimation unit 411 inputs the read-out image data and vital data to the emotion estimation model 421. The emotion estimation unit 411 outputs information (emotion estimation result information) on an emotion that has changed from pleasant to unpleasant among the emotion estimation results obtained by the emotion estimation model 421. That is, the emotion estimation unit 411 outputs emotion estimation result information on a change in emotion estimated based on the image data and vital data to the seating priority setting unit 412.

[0034] Next, the process proceeds to step ST3, where the seating priority setting unit 412 of the control unit 41 sets the priority of seating positions in the vehicle cabin based on the information on the emotion estimation result acquired from the emotion estimation unit 411. That is, the seating priority setting unit 412 determines the seating positions of occupants in the vehicle 4 based on the image data and the emotion estimation result information, and sets priorities for the determined seating positions.

[0035] An example of the priority order will now be described with reference to Figures 4 and 5. Figures 4 and 5 are diagrams each showing a first example of setting the priority order in an information processing method according to an embodiment.

[0036] (First Example) In the first embodiment, the priority is set based on the so-called upper and lower seats. In this case, the seating priority setting unit 412 can set priorities for multiple seating positions based on a rule base. That is, as shown in FIG. 4 , in the first embodiment, the seating priority setting unit 412 sets the priority of the seating position of user C in the middle of a three-row seating system in which the second row seats three people (priority (1)) among the seating positions of users A, B, C, D, E, and F, and then sets the priority of the seating positions of users F (priority (2)) and E (priority (3)) in the last row seats. Note that the reverse may also be possible. Furthermore, the seating priority setting unit 412 sets the priority of user A as priority (6), user B as priority (5), and user D as priority (4) for the seating positions of the other users A, B, and D, respectively. When the seating priority setting unit 412 functions based on a rule base, the priorities can be set in advance.

[0037] Furthermore, for example, if the vehicle 4 is a taxi or the like, as shown in FIG. 5, the seating priority setting unit 412 may set the priority of user A in the passenger seat to be high (priority (1)) among the seating positions of users A, B, C, D, E, and F, and then set the priority of users F (priority (2)) and E (priority (3)) in the rearmost seats to be high. The reverse may also be possible. Furthermore, the seating priority setting unit 412 may set the priority of user B to be priority (5), user C to be priority (6), and user D to be priority (4) among the seating positions of the other users B, C, and D, respectively.

[0038] It is also possible for the model generation unit 415 to generate a seating setting model 422 that sets priorities for multiple seating position values ​​based on the emotion estimation result for each occupant's riding occasion. In this case, information on changes in emotion based on the emotion estimation result information for each occupant is used as a learning input parameter. Also, the priority for executing measures for each occupant is used as a learning output parameter. The model generation unit 415 can generate the seating setting model 422 by machine learning such as deep learning using a learning input / output dataset of the learning input parameters and the learning output parameters. In other words, by setting priorities for executing measures for occupants, the priority for seating positions can be set by associating occupants with seating positions.

[0039] (Second Example) In the second embodiment, the seating priority setting unit 412 sets a priority according to the type of seat at which the user sits. That is, if a normal seat and a booster seat coexist in the cabin of the vehicle 4, the seating position of the booster seat is set to a higher priority. Furthermore, if a child seat is installed in the cabin, the seating position of the child seat is set to the highest priority, that is, priority (1). Note that if multiple child seats are installed in the cabin, the seating priority setting unit 412 sets the priority of the seating positions of the multiple child seats higher than the priority of the other seating positions, and further sets priorities for the multiple child seats. The priority described in the first embodiment can be used to set the priority for the seating positions of multiple child seats. Furthermore, the seating priority setting unit 412 can set a priority based on the birth year and month of the infant using the child seat.

[0040] (Other Examples) It is also possible to combine the first and second embodiments. That is, the seating priority setting unit 412 can determine the priority of seating positions by multiplying the priority of seating positions according to the first embodiment by the priority based on the seat type according to the second embodiment. Also, the priority of seating positions and the priority based on the seat type may be multiplied with different weights.

[0041] 2 and 3. After the seating priority setting unit 412 sets the priority order in step ST3, the seating priority setting unit 412 outputs the preferentially selected emotion estimation result information (information after prioritizing the emotion estimation result information) to the control determination unit 413. After that, the control process proceeds to step ST4.

[0042] In step ST4, the control decision unit 413 determines, for each occupant in accordance with the priority order, based on the emotion estimation result information for the occupant in the seating position set to a high priority, whether or not adjustment (improvement control) based on the emotion of the occupant is necessary for the driving of the vehicle 4. If the control decision unit 413 determines that no action is necessary for any occupant and that improvement control for the vehicle 4 is also unnecessary (step ST4: No), the control processing according to this embodiment ends.

[0043] On the other hand, in step ST4, the control decision unit 413 judges the emotion estimation result information in descending order of priority, and if it determines that action is required for at least one occupant and that improvement control is also required (step ST4: Yes), it proceeds to step ST5.

[0044] In step ST5, the control decision unit 413 decides the type of improvement control based on the emotion estimation result information on the occupant who has a relatively high priority and needs to be addressed in order of priority. Specifically, if the emotion of the occupant includes drowsiness, the control that reduces the damping force of the damper in the drive unit 47 of the vehicle 4 is selected as the improvement control. This can improve the ride comfort of the vehicle 4 to a soft and fluffy state.

[0045] Specifically, for example, a passenger's emotion that requires relatively high priority and requires attention may include motion sickness. In this case, in a moving vehicle 4, control is selected as improvement control to increase the damping force of the damper in the drive unit 47 of the vehicle 4 to reduce the vibration transmissibility. That is, in addition to vibrations from the engine of the drive unit 47, the vehicle 4 also generates vertical suspension vibrations, including sprung vertical resonance (approximately 1 to 2 Hz) and unsprung vertical resonance (approximately 14 Hz). The resonant frequency band of the suspension vertical vibrations, particularly the unsprung vertical resonance frequency band, is included in the vibration frequency band (4 Hz to 14 Hz) to which humans have a relatively high vibration sensitivity. Because the unsprung vertical resonance frequency band affects the user's ride comfort, improvement control is performed by controlling the damping characteristics of the absorbers provided in the suspension so that the damping effect is enhanced in the resonant frequency band of the suspension vertical vibrations. This suppresses vibrations in the resonant frequency band, thereby improving ride comfort.

[0046] Thereafter, the process proceeds to step ST6, where the control decision unit 413 outputs the decided type of improvement control as decision information to the vehicle control unit 414. The vehicle control unit 414 controls the drive unit 47 based on the input decision information, thereby executing improvement control for the traveling of the vehicle 4. This completes the control processing according to this embodiment.

[0047] According to the embodiment of the present disclosure described above, the feelings of an occupant in a seating position where discomfort is likely to occur can be prioritized, and the occupant's feelings can be improved from discomfort to comfort.

[0048] Although the embodiments of the present disclosure have been specifically described above, the present disclosure is not limited to the above-described embodiments, and various modifications based on the technical ideas of the present disclosure and embodiments that combine each other may be adopted. For example, the input / output data sets and training data listed in the above-described embodiments are merely examples, and different input / output data sets and training data may be used as needed.

[0049] For example, in the above-described embodiment, the method used to construct a learning model by the model generation unit 415 is not particularly limited, and various machine learning methods such as deep learning using a neural network, support vector machines, decision trees, naive Bayes, and k-nearest neighbor methods can be used. Semi-supervised learning may be used instead of supervised learning. Furthermore, reinforcement learning or deep reinforcement learning may be used as machine learning.

[0050] Furthermore, the emotion estimation model 421 and the seat setting model 422 may be generated outside the vehicle 4. When generated outside the vehicle 4, for example, a server installed outside the vehicle 4 may generate the emotion estimation model 421 and the seat setting model 422. In this case, the vehicle 4 acquires the emotion estimation model 421 and the seat setting model 422 through wireless communication with the server. The emotion estimation model 421 and the seat setting model 422 acquired from the server are stored in the storage unit 42. Furthermore, by storing the emotion estimation model 421 and the seat setting model 422 in the storage unit 42, the emotion estimation model 421 and the seat setting model 422 can be updated to the latest information.

[0051] In another embodiment, the functions of the control unit 41, the storage unit 42, the emotion deduction unit 411, the seating priority setting unit 412, the control decision unit 413, the vehicle control unit 414, and the model generation unit 415 may be divided and executed by multiple devices that can communicate with each other via the network 2. For example, at least some of the functions of the control unit 41 may be executed by a first device having a first processor. The functions of the emotion deduction unit 411 may be executed by a second device having a second processor. The functions of the seating priority setting unit 412 may be executed by a third device having a third processor. The functions of the model generation unit 415 may be executed by a fourth device having a fourth processor. Here, the first to fourth devices may be configured to be able to transmit and receive information to and from each other via the network 2 or the like. In this case, at least one of the first to fourth devices, for example, at least one of the first device and the second device, may be installed in the vehicle 4.

[0052] (Recording medium) In the above-described embodiment, the driving assistance device 40 and a program capable of executing a processing method performed by the driving assistance device 40 can be recorded on a recording medium readable by a computer or other machine or device (hereinafter, referred to as a computer, etc.). By having a computer, etc., read and execute the program from the recording medium, the computer, etc. functions as the control unit 41 of the driving assistance device 40 or the control unit 41 of the vehicle 4. Here, a computer-readable recording medium refers to a non-transitory recording medium that stores information such as data and programs electrically, magnetically, optically, mechanically, or chemically and can be read by a computer, etc. Examples of such recording media that are removable from a computer, etc. include flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs (Digital Versatile Disks), BDs, DATs, magnetic tapes, and memory cards such as flash memories. Furthermore, examples of recording media that are fixed to a computer, etc. include hard disks and ROMs. Furthermore, SSDs can be used as both recording media that are removable from a computer, etc. and recording media that are fixed to a computer, etc.

[0053] (Other embodiments) Furthermore, in the vehicle 4 and the driving assistance device 40 according to an embodiment, the "unit" can be read as a "circuit" or the like. For example, the communication unit can be read as a communication circuit. Furthermore, the program executed by the driving assistance device 40 and the vehicle 4 according to an embodiment may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0054] Further advantages and modifications will readily occur to those skilled in the art. The disclosure in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0055] 1. Driving assistance systems 2 Network 4 vehicles 40 Driving assistance devices 41 Control Unit 42 Storage section 43 Information Department 44 sensors 45 Communications Department 46 Positioning unit 47 Drive unit 411 Emotion estimation part 412 Seating priority setting unit 413 Control Decision Unit 414 Vehicle control unit 415 Model Generation Unit 421 Emotion Estimation Model 422 Seated Setting Model 441 Cabin camera 442 Wearable Devices A, B, C, D, E, F users

Claims

1. A driving assistance device provided in a moving body having a plurality of seating positions and including a processor, The processor: when performing an improvement control of the vehicle in response to the emotions of a passenger riding in the vehicle, prioritizing the plurality of seating positions; acquiring at least one of a captured image and biological information of the occupant and inputting the acquired image and biological information as input parameters to a learning model, and outputting an emotion estimation result of the occupant as an output parameter; selecting an occupant who is seated in a seating position that has a high priority among the occupants who need to be addressed by the improvement control according to the priority order for the seating position; determining the improved control for the vehicle based on the emotion estimation result of the selected occupant; controlling the moving object based on the determined improved control; When the plurality of seating positions are a central position in a three-seater seat, a rearmost position in a three-row seat, or an auxiliary seat, the priority of the seating position is set high. Driving assistance device.

2. A driving assistance device provided in a mobile body having a plurality of seating positions and including a processor, The processor: when performing an improvement control of the vehicle in response to the emotions of a passenger riding in the vehicle, prioritizing the plurality of seating positions; acquiring at least one of a captured image and biological information of the occupant and inputting the acquired image and biological information as input parameters to a learning model, and outputting an emotion estimation result of the occupant as an output parameter; selecting an occupant who is seated in a seating position that has a high priority among the occupants who need to be addressed by the improvement control according to the priority order for the seating position; determining the improved control for the vehicle based on the emotion estimation result of the selected occupant; controlling the moving object based on the determined improved control; The emotion estimation result obtained; Determine the priority of the seating position by multiplying the priority of the seating position by the priority based on the seat type. Driving assistance device.

3. The processor: When the occupant's emotion estimation result includes a feeling of drowsiness, As the improvement control, a control for reducing the damping force of the damper of the moving body is selected. The driving assistance device according to claim 1 or 2.

4. The processor: If the occupant's emotion estimation result includes a feeling of sickness, As the improvement control, a control for increasing the damping force of the damper of the moving body to reduce the vibration transmissibility is selected. The driving assistance device according to claim 1 or 2.

Images