Information processing device, information processing method, and information processing program

Abstract

The present invention provides an information processing device capable of accurately calculating the load placed on the driver of a vehicle. [Solution] The information processing device 20 comprises a load information acquisition unit 221 and an evaluation value calculation unit 222. The load information acquisition unit 221 acquires driver load information relating to the load on the vehicle driver and passenger load information relating to the load on the vehicle passengers. Based on the driver load information and passenger load information, the evaluation value calculation unit 222 calculates evaluation values ​​that assess the load on the driver and passengers due to the vehicle's operation.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] In order for a vehicle driver to drive safely and comfortably, it is necessary to consider not imposing an excessive load on the driver. In view of this, there is a system that determines the degree of driver fatigue from information on continuous driving time and driving area, and gives advice to take a rest at an appropriate timing based on the determined result. By using this system, the driver can drive comfortably and safely.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a plurality of people including passengers other than the driver ride in a vehicle, the load on the driver includes not only physical factors such as continuous driving time or driving area, but also the load imposed by the interaction with the passengers. However, the above-described technology does not consider the load related to passengers, and there is a problem that the load on the driver may not be accurately evaluated in some cases.

[0005] An object of the present invention is to provide an information processing apparatus, an information processing method, and an information processing program capable of accurately calculating the load on a vehicle driver.

Means for Solving the Problems

[0006] An information processing device according to one or more embodiments for achieving the above objective includes: a load information acquisition unit that acquires driver load information relating to the load on the driver of a vehicle and passenger load information relating to the load on passengers of the vehicle; and an evaluation value calculation unit that calculates evaluation values ​​that evaluate the load on the driver and passengers due to the driving of the vehicle, based on the driver load information and the passenger load information.

[0007] Furthermore, in an information processing method according to one or more embodiments, the information processing device acquires driver load information relating to the load on the driver of the vehicle and passenger load information relating to the load on the passengers of the vehicle, and calculates evaluation values ​​that evaluate the load on the driver and passengers, respectively, due to the driving of the vehicle, based on the driver load information and the passenger load information.

[0008] Furthermore, an information processing program according to one or more embodiments provides the information processing device with a function to acquire driver load information relating to the load on the driver of a vehicle and passenger load information relating to the load on the passengers of the vehicle, and a function to calculate evaluation values ​​that evaluate the load on the driver and passengers due to the driving of the vehicle, based on the driver load information and the passenger load information. [Effects of the Invention]

[0009] According to an information processing device, information processing method, and information processing program relating to one or more embodiments, the load on the vehicle driver can be calculated with high accuracy. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of a driver assistance system using an information processing device according to one embodiment of the present invention. [Figure 2A] This table shows an example of the detailed contents of the physical burden on the driver, which is acquired by an information processing device according to one embodiment of the present invention. [Figure 2B] This table shows an example of the detailed content of the mental burden on the driver, which is acquired by an information processing device according to one embodiment of the present invention. [Figure 3] This table shows an example of the detailed contents of the load on passengers acquired by an information processing device according to one embodiment of the present invention. [Figure 4] This flowchart shows the operation of an information processing device according to one embodiment of the present invention. [Figure 5A] This is an example of display information, including evaluation values ​​for the driver and passenger, displayed by an information processing device according to one embodiment of the present invention. [Figure 5B] This is another example of display information, including evaluation values ​​for the driver and passenger, displayed by an information processing device according to one embodiment of the present invention. [Figure 6] This diagram shows the mental stress levels of the driver and passenger, as displayed by an information processing device according to one embodiment of the present invention, in a radar chart for each item. [Modes for carrying out the invention]

[0011] A driving assistance system constructed using an information processing device according to an embodiment of the present invention will be described below with reference to the drawings. In this embodiment, the case in which three people, driver D, first passenger E1, and second passenger E2, are driving in vehicle X will be described.

[0012] <Configuration of a driver assistance system according to one embodiment> Figure 1 is a block diagram showing the configuration of the driver assistance system 1 according to this embodiment. The driver assistance system 1 includes a driver terminal 11, a first passenger terminal 12, a second passenger terminal 13, a drive recorder 14, a vehicle navigation device 15, a display audio device 16, a driving control device 17, and an information processing device 20. In this embodiment, the case in which the driver assistance system 1 is equipped with two passenger terminals is described, but it is not limited to this, and it may be one or three or more.

[0013] The driver terminal 11 is a mobile terminal owned by the driver D. The first passenger terminal 12 is a mobile terminal owned by the first passenger E1. The second passenger terminal 13 is a mobile terminal owned by the second passenger E2. These driver terminal 11, first passenger terminal 12, and second passenger terminal 13 are composed of smart devices such as smartphones and tablet terminals.

[0014] The drive recorder 14, vehicle navigation device 15, display audio device 16, and driving control device 17 are installed inside the vehicle X. Also, the vehicle X may be provided with a vibration device 18 that vibrates the seat.

[0015] The drive recorder 14 has a front camera device 141 that captures the front of the vehicle X, a rear camera device 142 that captures the rear, and a sound collection device 143.

[0016] The front camera device 141 and the rear camera device 142 capture the outside and inside of the vehicle X. The sound collection device 143 collects the sound inside the vehicle X. The drive recorder 14 transmits the video captured outside and inside the vehicle X and the sound inside the vehicle X collected to the information processing device 20.

[0017] The vehicle navigation device 15 is a device that outputs driving guidance information such as the driving route, position information, and information on surrounding facilities of the vehicle X, and has a first display device 151 and a first speaker device 152.

[0018] The display audio device 16 is a car audio device that operates in cooperation with any one of the driver terminal 11, first passenger terminal 12, and second passenger terminal 13 composed of, for example, a smart device, and has a second display device 161 and a second speaker device 162. The display audio device 16 can also function as a navigation device by using an application installed on the driver terminal 11, first passenger terminal 12, or second passenger terminal 13.

[0019] The vehicle navigation device 15 and the display audio device 16 transmit information on the route along which the vehicle X travels to the information processing device 20. Also, the vehicle navigation device 15 and the display audio device 16 transmit, as operation information, the operation content, the number of times, and the operation time of operations performed by the driver D or the passengers E1, E2 on the vehicle navigation device 15 and the display audio device 16 to the information processing device 20.

[0020] In the present embodiment, operation information in the case where the display content in the display audio device 16 is controlled by operations from the driver terminal 11, the first passenger terminal 12, and the second passenger terminal 13 is also referred to as operation information of the display audio device 16.

[0021] Also, the search for the travel route, the acquisition of the position information of the vehicle X, the map display, etc. may be performed by the vehicle navigation device 15 or may be performed by the display audio device 16.

[0022] The driving control device 17 is a device that controls the vehicle X. The driving control device 17 transmits information related to the driving operation of the vehicle X, such as the steering operation status, the number of brake operations, etc. to the information processing device 20. Also, the driving control device 17 acquires information on vehicle equipment such as the doors, windows, air conditioner, etc. of the vehicle and transmits it to the information processing device 20.

[0023] The vehicle X may be provided with various sensors (not shown). The various sensors are, for example, a vibration sensor that detects the vibration of the vehicle X, a contact sensor provided on the steering wheel, a seating sensor provided on each seat, etc. The driving control device 17 transmits the information acquired by the various sensors to the information processing device 20. Also, information related to the driving operation of the vehicle X may be acquired by the vehicle navigation device 15 from the vehicle X via CAN (Controller Area Network) and transmitted to the information processing device 20.

[0024] The vibration device 18 vibrates the seats inside the vehicle X.

[0025] The information processing device 20 is communicatively connected to the driver terminal 11, the first passenger terminal 12, the second passenger terminal 13, the drive recorder 14, the vehicle navigation system 15, the display audio system 16, the driving control device 17, and the vibration device 18. The information processing device 20 may be connected to the passenger terminals of all passengers riding in vehicle X, or it may be connected to the passenger terminals of only some of the passengers.

[0026] The information processing device 20 may be implemented as an application in the driver terminal 11, the first passenger terminal 12, or the second passenger terminal 13, or it may be installed in the drive recorder 14, the vehicle navigation system 15, the display audio system 16, or the driving control device 17. Alternatively, it may be implemented as an application in the driver terminal 11, the first passenger terminal 12, or the second passenger terminal 13, and the processed information may be displayed on the display device in the vehicle navigation system 15 or the display audio system 16. If the information processing device 20 is installed in the display audio system 16, the driving route and location information of the vehicle X may be obtained from a smart device such as the driver terminal 11, the vehicle navigation system 15, or the driving control device 17. Alternatively, the information processing device 20 may be configured as a server located remotely from the vehicle X, and be able to communicate with the driver terminal 11, the first passenger terminal 12, the second passenger terminal 13, and each device within the vehicle X. Alternatively, the information processing device 20 may be provided as a single independent device. Furthermore, an output device such as a display device or a speaker device may be provided within the information processing device 20.

[0027] The information processing device 20 includes a storage unit 21 and a CPU 22. The storage unit 21 is composed of, for example, RAM (Random Access Memory) or ROM (Read Only Memory). The storage unit 21 includes a driver information storage unit 211 and a passenger information storage unit 212.

[0028] The driver information storage unit 211 stores driver registration information for driver D that has been registered in advance. Driver registration information includes, for example, information that has been pre-entered to indicate driver D's driving preferences, such as driving history, information indicating driving frequency, information indicating the average driving time per trip, information on frequently driven areas, routes, or road types, road types on which driver D is good or bad at driving, road conditions, weather, time of day, etc. The driver information storage unit 211 may also record information such as the type of vehicle (vehicle type) that driver D usually drives and the functions installed in the vehicle as part of the driver registration information.

[0029] The driver information storage unit 211 may also store information about driver D's values ​​as information indicating driver D's driving preferences. Driver D's values ​​include, for example, values ​​regarding driving itself, such as disliking driving; values ​​regarding driving operations, such as being bad at parking and lane changes; values ​​regarding external factors, such as being quick to anger at things like cutting in or people suddenly appearing; values ​​regarding expenses, such as fuel efficiency and driving on toll roads; and values ​​related to personality, such as anxiety about causing an accident. In addition, information regarding driver D's values ​​may also be stored as "NG behaviors," such as times when driver D does not want to be spoken to while driving and words or actions that driver D does not want to hear while driving.

[0030] The driver registration information may be pre-weighted for each item based on driver D's driving preferences. For example, by assigning a higher weight to items in the driver registration information that driver D deems to be of high importance, these items will be preferentially used as calculation elements when calculating driver D's evaluation value, as will be described later.

[0031] The passenger information storage unit 212 stores passenger registration information for passengers E1 and E2 that have been registered in advance. Passenger registration information includes, for example, information on whether or not they can take turns driving based on whether or not they have a driver's license, information on their driving history, information on the frequency of driving, information on the average driving time per trip, information on the areas, routes, or types of roads they frequently drive, information on the types of roads they are good or bad at driving, road conditions, weather, time of day, etc., and information that indicates the driving preferences of passengers E1 and E2. The passenger information storage unit 212 may also record information such as the type (vehicle type) and installed functions of the vehicle that passengers E1 and E2 usually drive as passenger registration information.

[0032] Furthermore, the passenger information storage unit 212 may also store information indicating the driving preferences of passengers E1 and E2, such as information on when they do not want to be spoken to while driving, information on words and actions they do not want to hear while driving, and information indicating their wishes for the passengers.

[0033] Similar to the driver registration information, passenger registration information may also be pre-weighted for each item based on the driving preferences of the first passengers E1 and E2. Furthermore, the weighting values ​​for the driver registration information and passenger registration information described above may be changed depending on whether or not the vehicle X is equipped with functions that provide driving assistance, such as automatic braking.

[0034] This driver registration information and passenger registration information are input into the information processing device 20 by driver D and passengers E1 and E2 before the start of driving, and stored in the storage unit 21. This information may be input as numerical values, or it may be input by selecting from one of several provided levels.

[0035] Furthermore, this driver registration information and passenger registration information may also be information estimated from past driving information acquired by the vehicle navigation system 15 and the display audio system 16, as well as from video footage and audio recordings of the driver D's facial expressions and speech based on these recordings, acquired by the drive recorder 14.

[0036] CPU22 is, for example, the CPU (Central Processing Unit) of a general-purpose microcomputer, and by installing and executing a predetermined driving support program, it configures one or more information processing units as shown below.

[0037] The CPU 22 includes a load information acquisition unit 221, an evaluation value calculation unit 222, a presentation information generation unit 223, and an output control unit 224.

[0038] The load information acquisition unit 221 acquires (a) driver load information relating to the load on driver D, and (b) passenger load information relating to the load on the first passenger E1 and the second passenger E2, based on information received from the devices and storage unit 21 that are connected to the communication device.

[0039] Specifically, the load information acquisition unit 221 receives video footage from the drive recorder 14 of the area outside the vehicle X, video footage from the inside of the vehicle X, and audio recorded inside the vehicle X. The load information acquisition unit 221 also acquires information such as traffic congestion, weather, and road surface conditions from the video footage of the area outside the vehicle X. The load information acquisition unit 221 also acquires the driver D's facial expressions, gaze, and behavior based on the video and audio from inside the vehicle X. The load information acquisition unit 221 also acquires the facial expressions, behavior, and actions of passengers E1 and E2 based on the video and audio from inside the vehicle X. The load information acquisition unit 221 also identifies the seating positions of passengers E1 and E2 based on the video from inside the vehicle X and acquires the intensity of sunlight received by passengers E1 and E2. The load information acquisition unit 221 may also receive and store video footage of preparations before starting to drive, video footage of the inside of the vehicle X during past driving, and recorded audio.

[0040] The load information acquisition unit 221 also receives information about the route that vehicle X is traveling from the vehicle navigation system 15 or the display audio system 16. The load information acquisition unit 221 also receives operation information from the vehicle navigation system 15 or the display audio system 16, including the operation content, the number of times it was performed, and the duration of the operation. The load information acquisition unit 221 may also receive and store operation information from before the start of driving.

[0041] The load information acquisition unit 221 also receives information from the driving control device 17 regarding the operation of the vehicle X, information about the vehicle equipment, information about vehicle vibration, information about seat vibration, etc.

[0042] (a) The burden on driver D consists of (a-1) physical burden and (a-2) mental burden. Figure 2A is a table showing an example of the detailed contents of (a-1) physical burden. (a-1) Physical burden includes "burden of preparation," "burden of operating the vehicle and equipment," and "burden due to external factors."

[0043] "Preparation burden" refers to the burden incurred by, for example, checking the completeness of the equipment inside the vehicle, or searching for the driving route in advance using the driver terminal 11, vehicle navigation system 15, or display audio system 16.

[0044] The load information acquisition unit 221 acquires information such as the content and time of pre-operation preparations as "pre-operation load" based on pre-operation operation information received from the vehicle navigation system 15 and the display audio system 16, or video footage from inside the vehicle X received from the drive recorder 14 before the start of operation.

[0045] "Load from vehicle and equipment operation" refers to the load incurred from operating the vehicle and its installed equipment, including general driving operations, as well as operations such as locking doors, opening and closing windows, and using the air conditioner.

[0046] The load information acquisition unit 221 acquires information on the operation of the vehicle X received from the driving control device 17, as well as information on the vehicle equipment and information from various sensors installed in the vehicle X, as "operational load of the vehicle," such as driving time, the number of times operations such as steering wheel and brakes are performed, the total time the steering wheel is held, the strength of the grip on the steering wheel, and the number of times the vehicle and equipment installed in the vehicle are operated.

[0047] Furthermore, the load information acquisition unit 221 acquires the time that driver D is concentrating on driving, the time that driver D feels drowsy or fatigued, and the degree of concentration, drowsiness, and fatigue as numerical values ​​or multi-level indicators of "car operation load" based on the driver D's facial expressions and gaze in the video footage from vehicle X received from the drive recorder 14. In addition, in vehicles equipped with a driving assistance function, if the auto brake is activated more than a predetermined number of times or frequency, the load information acquisition unit 221 may acquire the number of times or frequency as "car operation load" on the assumption that driver D's concentration level has decreased.

[0048] "External factors" refer to the load caused by external environmental factors that affect operation, such as weather (rain, snow, etc.), traffic congestion, road surface conditions, and the presence or absence of roads requiring special attention while driving (tunnels, slopes, etc.).

[0049] The load information acquisition unit 221 acquires information such as rainfall / snowfall amount, wind speed, visibility clarity, road type and driving time as "external load factors" based on video footage from outside the vehicle X received from the drive recorder 14, the driving route received from the vehicle navigation system 15 or display audio system 16, and vibration sensor information received from the driving control device 17. The load information acquisition unit 221 may also acquire information such as traffic congestion, weather, and road surface conditions as "external load factors" from the driver terminal 11, the first passenger terminal 12, or the second passenger terminal 13.

[0050] Figure 2B is a table showing an example of the detailed contents of (a-2) mental burden. (a-2) Mental burden includes "burden related to the driver's values" and "burden related to interpersonal relationships".

[0051] "Driver's value-based burden" refers to the burden that driver D perceives as a burden due to operations and situations. The burden felt regarding driving varies from driver to driver. The load information acquisition unit 221 acquires from the driver information storage unit 211, as "driver's value-based burden," the operations and situations that driver D perceives as a burden and the degree to which they perceive it, such as values ​​regarding driving itself, values ​​regarding driving operations, values ​​regarding external factors, values ​​regarding expenses, and values ​​regarding personality, either numerically or at multiple levels.

[0052] The load information acquisition unit 221 acquires information on the number of occurrences and duration of operations and situations that the driver D perceives as burdensome, as "load based on the driver's values," based on video footage from outside the vehicle X received from the drive recorder 14, the driving route received from the vehicle navigation system 15 or display audio system 16, and information on the driving operations of the vehicle X received from the driving control device 17.

[0053] "Interpersonal burden" refers to the burden based on the words or actions of passengers. "Interpersonal burden" refers to the burden of driving operations / equipment operation other than normal driving at the request of passengers E1 and E2, such as being asked by passengers E1 and E2 to operate the vehicle navigation system 15 or the display audio system 16, or being asked to make sudden route change instructions such as "I want to stop at a service area."

[0054] Furthermore, "interpersonal burden" refers to the burden caused by the words and actions of passengers that disrupt the driver D's concentration or interfere with driving, such as certain interjections ("Oh no!", "Ah!", etc.), repeated confirmations ("Can you hear me? Can you hear me?", etc.), words or complaints that show frustration during traffic jams, or words that rush the driver ("Are we there yet?", "Are we there soon?", etc.).

[0055] Furthermore, "interpersonal burden" refers to the burden caused by the words and actions of passengers who are indifferent to the driver's driving, such as fiddling with their smartphones, dozing off, or fidgeting. It also refers to the burden caused by feeling unable to join in on conversations in the car, or feeling the need for someone to talk to.

[0056] The load information acquisition unit 221 acquires, based on the video and audio from the drive recorder 14 within the vehicle X, the number of times the relevant passengers E1 and E2 speak or act, the time between the passengers' words or actions and the driver D's actions, and the time or rate at which driver D does not participate in conversations with passengers E1 and E2, as "interpersonal burden." The load information acquisition unit 221 may also acquire, as "interpersonal burden," the number of times NG behaviors, which are stored as information related to driver D's values.

[0057] Furthermore, the load information acquisition unit 221 may acquire the mental burden of driver D based on the degree to which driver D felt burdened, based on the facial expressions of driver D in the video footage inside vehicle X received from the drive recorder 14.

[0058] (b) The burden on the first passenger E1 and the second passenger E2 also consists of (b-1) physical burden and (b-2) mental burden. Figure 3 is a table showing examples of the detailed contents of (b-1) physical burden and (b-2) mental burden.

[0059] (b-1) Physical loads include "loads from sitting" and "loads from operating equipment." "Loads from sitting" include loads from vibrations of the vehicle X, varying seating comfort due to the seat position resulting from feeling gravitational acceleration, changes in physical condition such as drowsiness or motion sickness, and external environmental factors such as sunlight. "The burden of operating the equipment" refers to the burden on passengers E1 and E2 to operate the vehicle navigation system 15 or the display audio system 16 on behalf of driver D.

[0060] The load information acquisition unit 221 acquires the following information as "seat-related load" based on the video and audio received from the drive recorder 14 inside the vehicle X: the duration of facial expressions of passengers E1 and E2 indicating drowsiness or motion sickness, the duration and frequency of conversations, and the level of sunlight shining on passengers E1 and E2.

[0061] Furthermore, the load information acquisition unit 221 acquires the amount of vibration or gravitational acceleration of each seat as "load due to seating" based on information from sensors installed in the vehicle X received from the driving control device 17. The load information acquisition unit 221 may also acquire information indicating the seating comfort numerically or in multiple levels based on vibration or gravitational acceleration as "load due to seating". In addition, the physical load of passengers E1 and E2 may be acquired by sensors installed in passenger terminals 12 and 13.

[0062] (b-2) Mental stress includes "stress due to seating position," "stress from supporting the driver," and "stress from creating a pleasant atmosphere inside the vehicle."

[0063] "Seat-related stress" refers to the mental stress specific to a particular seating position, such as the fear experienced while driving in the passenger seat, or the need to pay attention to and be considerate of a child when sitting next to them. The stress at each seating position varies depending on the seating arrangement of the occupants in the vehicle.

[0064] The load based on seat position is predetermined as a numerical value or multiple levels based on each passenger's seat position and seat position relationship. The seat positions are input by driver D or passengers E1 and E2 and stored in the driver information storage unit 211.

[0065] The load information acquisition unit 221 acquires the "load due to seat position" as a numerical value or level based on the seat position and seat position relationship received from the driver information storage unit 211. The load information acquisition unit 221 may also acquire the seat position based on the video footage inside the vehicle X received from the drive recorder 14.

[0066] "The burden of supporting the driver" refers to the burden of supporting driver D, such as asking driver D for requests for breaks or meals, assisting with meals such as opening plastic bottles, operating the vehicle navigation system 15 and display audio system 16 at driver D's request, receiving consultations from driver D regarding driving and routes, and showing empathy to driver D.

[0067] The load information acquisition unit 221 acquires the number of times the relevant passengers E1 and E2 perform certain actions and behaviors as "load due to supporting the driver," based on the video footage and audio recordings of the vehicle X received from the drive recorder 14.

[0068] "The burden of creating a pleasant atmosphere inside the vehicle" refers to the burden of actions and words taken for the benefit of other passengers, such as making comments to encourage conversation among passengers, researching destinations, and providing information.

[0069] The load information acquisition unit 221 acquires the number of times the relevant passenger's words and actions occur, based on the video footage and audio recordings of the vehicle X received from the drive recorder 14, as "load caused by creating a certain atmosphere inside the vehicle."

[0070] As described above, the load information acquisition unit 221 acquires quantified driver load information and passenger load information by associating each piece of information received from the communicatively connected devices and the storage unit 21 with the type of load. Note that the contents of each load are not limited to the examples shown in Figures 2A, 2B, and 3.

[0071] The evaluation value calculation unit 222 calculates evaluation values ​​for the loads placed on driver D, first passenger E1, and second passenger E2 due to the operation of vehicle X, based on the driver load information and passenger load information acquired by the load information acquisition unit 221. For example, the evaluation value calculation unit 222 quantifies each load and calculates the evaluation value. The calculation process for the evaluation value of driver D and the evaluation values ​​of passengers E1 and E2 will be explained below.

[0072] [Driver D's evaluation score] The evaluation value calculation unit 222 adds an evaluation value based on driving time to the driver D's evaluation value. The evaluation value calculation unit 222 may further add an evaluation value based on the number of times vehicle X and the equipment installed on vehicle X are operated to the driver D's evaluation value. The evaluation value calculation unit 222 may further add an evaluation value based on the driver D's level of concentration on driving and the driver D's level of drowsiness and fatigue to the driver D's evaluation value. Specifically, the evaluation value calculation unit 222 may calculate a higher evaluation value for the time during which driver D is concentrating on driving, drowsiness, and fatigue, based on the level of concentration, drowsiness, and fatigue. The evaluation value calculation unit 222 may further add an evaluation value based on the burden of pre-driving preparations to the driver D's evaluation value.

[0073] Furthermore, the evaluation value calculation unit 222 may add an evaluation value based on the load caused by external factors to the driver D's evaluation value. For example, the evaluation value calculation unit 222 may add an evaluation value based on the rainfall level and the time spent driving during rainfall to the driver D's evaluation value. Specifically, the evaluation value calculation unit 222 may calculate a higher evaluation value for the time spent driving during rainfall within the total driving time, based on the rainfall level.

[0074] The evaluation value calculation unit 222 may also add an evaluation value based on the burden according to driver D's values ​​to driver D's evaluation value. For example, if driver D feels that changing lanes is difficult, the evaluation value calculation unit 222 adds an evaluation value based on the number of lane changes to driver D's evaluation value.

[0075] The evaluation value calculation unit 222 may also add an evaluation value based on the burden caused by interpersonal relationships to the driver D's evaluation value. For example, the evaluation value calculation unit 222 may add an evaluation value based on the number of times the passenger's words and actions caused a burden to the driver D's evaluation value, or it may add an evaluation value based on the degree to which the driver D felt burdened.

[0076] Specifically, if the evaluation value calculation unit 222 detects that the first passenger E1 or the second passenger E2 has taken an action that upsets driver D, such as clicking their tongue, during a traffic jam, it determines that driver D's mental burden, or "interpersonal burden," has increased, and adds to driver D's evaluation value. Also, if the evaluation value calculation unit 222 detects that the first passenger E1 or the second passenger E2 has fallen asleep during the drive, it determines that driver D's mental burden, or "interpersonal burden," has increased, and adds to driver D's evaluation value. In these cases, the evaluation value calculation unit 222 may also subtract the evaluation value of the passenger that contributed to increasing driver D's "interpersonal burden."

[0077] Furthermore, the evaluation value calculation unit 222 may add evaluation values ​​based on both interpersonal burden and the burden based on driver D's values ​​to driver D's evaluation value. For example, based on the driver burden information and passenger burden information described above, when the occupants in vehicle X are having a lively conversation, the evaluation value calculation unit 222 detects from the image information captured by the front camera device 141 of the drive recorder 14 that driver D is making a lane change, which is something he is not comfortable with, meaning that he should be concentrating on driving, and identifies the occupants who are talking. When the evaluation value calculation unit 222 recognizes from the image information that the first passenger E1 and the second passenger E2 are talking, and that driver D is looking ahead and not speaking, it determines that driver D's level of concentration on driving is high, and that the physical burden on driver D, "burden of operating the car," is high, and adds to driver D's evaluation value.

[0078] Furthermore, the evaluation value calculation unit 222 may, based on the weighted information of driver D's driving preferences stored in the driver information storage unit 211, increase the evaluation priority for loads that driver D judges to be of high importance, i.e., calculate a higher evaluation value, and decrease the evaluation priority for loads that driver D judges to be of low importance, i.e., calculate a lower evaluation value. For example, with respect to driver D, the evaluation value calculation unit 222 may calculate a higher evaluation value based on driving time if driver D's values ​​are "dislike of driving," and a lower evaluation value if driver D's values ​​are "like driving."

[0079] For example, the evaluation value calculation unit 222 calculates a higher evaluation value for driver D when it comes to driving time under road types, road conditions, weather, time of day, etc., which are set as conditions in which driver D is not good at driving, because it considers that driver D's workload is high.

[0080] Furthermore, if the driver D has set a driving preference to have passengers sleep, the evaluation value calculation unit 222 calculates a lower evaluation value, assuming that the driver D's workload is low in relation to passengers E1 and E2 falling asleep.

[0081] Furthermore, the evaluation value calculation unit 222 may calculate a lower evaluation value for driving time if vehicle X is equipped with a driving assistance function than if the vehicle is not equipped with a driving assistance function. Also, the evaluation value calculation unit 222 may calculate a higher evaluation value for driving time if the vehicle that driver D usually drives is equipped with a driving assistance function, and vehicle X is not equipped with a driving assistance function.

[0082] [Evaluation scores for passengers E1 and E2] The evaluation value calculation unit 222 adds an evaluation value to the evaluation values ​​of passengers E1 and E2 based on the level of comfort of each seat. The evaluation value calculation unit 222 may also add an evaluation value based on the duration of facial expressions indicating drowsiness or motion sickness. Furthermore, if passengers E1 and E2 operate any of the devices, the evaluation value calculation unit 222 may add an evaluation value based on the number of operations and the duration of those operations. The evaluation value calculation unit 222 may also add an evaluation value to passengers E1 and E2 who support driver D or create a pleasant atmosphere in the vehicle based on the number of their words and actions. Furthermore, the evaluation value calculation unit 222 may subtract an evaluation value from passengers E1 and E2 who perform burdensome words and actions.

[0083] Furthermore, if passenger E1 or E2 takes over driving, the evaluation value calculation unit 222 adds an evaluation value based on driving time to the evaluation value of the passenger E1 or E2 who took over driving. The evaluation value calculation unit 222 may also add evaluation values ​​based on the burden due to external factors, evaluation values ​​based on the burden due to the values ​​of passenger E1 or E2, and evaluation values ​​based on the burden due to interpersonal relationships to the evaluation value of the passenger E1 or E2 who took over driving, similar to the evaluation value of driver D. The evaluation value calculation unit 222 may also, based on weighted information based on the driving inclination of passenger E1 or E2, increase the evaluation priority for burdens that are judged to be of high importance, i.e., calculate a higher evaluation value, and decrease the evaluation priority for burdens that are judged to be of low importance, i.e., calculate a lower evaluation value.

[0084] The presentation information generation unit 223 generates information to be presented to at least one of driver D, first passenger E1, and second passenger E2, based on the driver load information, passenger load information acquired by the load information acquisition unit 221, or the evaluation value calculated by the evaluation value calculation unit 222. The output control unit 224 causes the information calculated by the evaluation value calculation unit 222 and the information generated by the presentation information generation unit 223 to be output from any device that is connected via communication.

[0085] <Operation of a driver assistance system according to one embodiment> Figure 4 is a flowchart showing the driver assistance processes executed by the information processing device 20 when three people, driver D, first passenger E1, and second passenger E2, are in vehicle X and driving using the driver assistance system 1.

[0086] Before the vehicle X starts driving, the application of the information processing device 20 is activated (S1 "YES"), and information on the occupants, including driver D, first passenger E1, and second passenger E2, is input. At this time, information such as the age, physical condition, and body type of each occupant may also be input, and the information processing device 20 may generate suggested seating positions for each occupant, suggested driver assistants, and suggested BGM operators, taking into account the input information, and output this information to one of the devices.

[0087] When the vehicle X starts to operate, the load information acquisition unit 221 acquires driver load information relating to the load on driver D, and passenger load information relating to the load on first passenger E1 and second passenger E2 from the devices and storage unit 21 that are connected to the vehicle via communication (S2).

[0088] The evaluation value calculation unit 222 calculates evaluation values ​​(S3) that assess the load on driver D, first passenger E1, and second passenger E2 due to the driving of vehicle X, based on the driver load information and passenger load information acquired by the load information acquisition unit 221. The higher the load on the occupants being calculated, the higher the value calculated.

[0089] The evaluation value calculation unit 222 determines whether it is time to output information to at least one of the driver D, the first passenger E1, or the second passenger E2 (S4). This timing is set based on at least one of the following: the driving route of vehicle X, the state of the occupants based on the calculated evaluation value, or the state of vehicle X.

[0090] When the evaluation value calculation unit 222 determines that it is time to output information (YES in S4), the presentation information generation unit 223 generates presentation information (S5). The output control unit 224 selects one of the devices connected to communicate and outputs the information generated by the presentation information generation unit 223 (S6).

[0091] In step S4 described above, if the evaluation value calculation unit 222 determines that the timing for outputting information has not yet arrived ("NO" in S4), the process returns to step S2.

[0092] Specific examples (1) to (5) of steps S4 to S6 of the driving support processing performed by the information processing device 20 are given below.

[0093] [Specific example (1): Output of information presented at predetermined time intervals] In this example, the information processing device 20 outputs information at predetermined time intervals during the drive. The information generation unit 223 generates information indicating the degree of each person's load from the evaluation values ​​of driver D, first passenger E1, and second passenger E2 calculated by the evaluation value calculation unit 222. The output control unit 224 causes the information to be output from at least one of the driver terminal 11, first passenger terminal 12, second passenger terminal 13, vehicle navigation device 15, or display audio device 16 at predetermined time intervals. This allows driver D, first passenger E1, and second passenger E2 to recognize their own and others' load levels, leading to considerate words and actions towards others.

[0094] [Specific example (2): Output of information displayed at the parking timing] In this example, the information processing device 20 outputs information to be presented when the vehicle takes a break, such as stopping at a service area. When the information processing device 20 detects that the engine of vehicle X is OFF during driving, based on control information from the driving control device 17, the information presentation generation unit 223 generates information to be presented that indicates the driver's load, using the evaluation value of driver D calculated by the evaluation value calculation unit 222. The output control unit 224 displays the information to be presented generated by the information presentation generation unit 223 on the first display device 151 of the vehicle navigation device 15 or the second display device 161 of the display audio device 16.

[0095] The output control unit 224 may output the information to be presented when it detects that the engine of vehicle X is OFF and that the vehicle is parked in a place where it can be parked for a predetermined period of time or longer, such as a convenience store or service area. Alternatively, the output control unit 224 may output the information to be presented when it detects that the engine of vehicle X is OFF and that the driver D or passengers E1 and E2 are inside vehicle X.

[0096] The output control unit 224 may output information when the engine of vehicle X is OFF and the evaluation value indicating the driver D's load is above a predetermined value, or when a specific load among the driver D's load information is above a predetermined value. This "specific load" could be, for example, the number of times driver D yawns within a predetermined period, or the number of times the auto brake is activated.

[0097] Figure 5A shows an example of the information displayed by the output control unit 224. In addition to the driver D's evaluation value, the output control unit 224 may also display information acquired by the load information acquisition unit 221 as the information to be displayed. The information displayed in Figure 5A shows a donut chart of driver D's evaluation value (score) of "70%", current image information P1 of driver D taken by the drive recorder 14, and information indicating the degree of drowsiness, which shows the number of times driver D has yawned in the most recent predetermined time, "10 times".

[0098] Furthermore, based on the calculated evaluation value, the presentation information generation unit 223 generates presentation information that suggests a change of driver (e.g., "Would you like to drive?") along with information indicating the driver D's workload.

[0099] The output control unit 224 selects the first passenger E1, who has a low load and is available to take over driving, from among passengers E1 and E2 based on the load information of passengers E1 and E2 calculated by the evaluation value calculation unit 222 and the information on whether or not a driver change is possible stored in the passenger information storage unit 212. The output control unit 224 then displays the presentation information generated by the presentation information generation unit 223 on the display unit of the first passenger terminal 12 held by the selected first passenger E1. In other words, the output device and the timing of the output are appropriately changed based on the presentation information to be output. This makes it possible to suggest a driver change at an appropriate timing according to the driver D's fatigue level.

[0100] Furthermore, the information generation unit 223 may generate and display information prompting a passenger who is skilled at driving on a type of road that driver D is not good at (such as a mountain road) to take over driving, based on the driving preferences of driver D, passengers E1 and E2 stored in the driver information storage unit 211 and passenger information storage unit 212. This allows for an appropriate suggestion of driver changes depending on the degree of difficulty driver D is having driving.

[0101] [Specific example (3): Output of information to be displayed at the time when prompting a change of seating arrangement] In this example, the information processing device 20 outputs presentation information when the load on a particular passenger increases during a drive. The presentation information generation unit 223 detects that the difference between the evaluation value of the first passenger E1 sitting in the passenger seat and the evaluation value of the second passenger E2 sitting in the rear seat, calculated by the evaluation value calculation unit 222, exceeds a predetermined value, and generates presentation information showing the evaluation values ​​of the driver D, the first passenger E1, and the second passenger E2. The output control unit 224 displays the presentation information generated by the presentation information generation unit 223 on the first display device 151 of the vehicle navigation device 15 or the second display device 161 of the display audio device 16.

[0102] Figure 5B shows an example of the display information shown by the output control unit 224. The display information shown in Figure 5B shows a donut chart of the driver D's evaluation value (score) of "70%", the first passenger E1's evaluation value (score) of "50%", and the second passenger E2's evaluation value (score) of "15%".

[0103] Furthermore, as shown in Figure 6, the mental stress levels of driver D and passengers (only the first passenger E1 is shown in Figure 6) may be represented by radar charts for each item.

[0104] Furthermore, based on the calculated evaluation values, the presentation information generation unit 223 determines that the difference between the evaluation value of the first passenger E1 ("50%") and the evaluation value of the second passenger E2 ("15%") is 30% or more, and generates text information "Shall we change seats?" suggesting a seat change. The output control unit 224 displays the presentation information generated by the presentation information generation unit 223 on the display unit of the first passenger terminal 12 held by the first passenger E1, for example. This makes it possible to suggest a seat change at an appropriate time when the load of multiple passengers differs depending on their seating position. The output control unit 224 may also display the presentation information generated by the presentation information generation unit 223 on the display unit of the passenger terminal of the passenger who is suggesting a seat change.

[0105] [Specific example (4): Output of information presented at the timing of notification of evaluation value deduction and prompting of evaluation value addition] During a drive, if the evaluation value calculation unit 222 detects that a passenger (for example, first passenger E1) has performed an NG behavior previously stored in the driver information storage unit 211, it subtracts the evaluation value of the first passenger E1. The evaluation value calculation unit 222 may also consider words or actions that hinder the driver D's concentration or interfere with driving, or words or actions that show indifference to driving, as acquired by the load information acquisition unit 221 as "load due to interpersonal relationships," as NG behaviors.

[0106] Furthermore, if the load information acquisition unit 221 detects an NG behavior, the presentation information generation unit 223 generates presentation information to warn against the NG behavior. This presentation information may be text information such as "Don't talk to me!" or "That's an NG word." The output control unit 224 displays the presentation information generated by the presentation information generation unit 223, along with information indicating the evaluation values ​​of driver D and first passenger E1, on the display unit of the first passenger terminal 12 held by the first passenger E1.

[0107] The output control unit 224 may also display this information and alert the first passenger E1 by vibrating the seat where the first passenger E1 is sitting using a vibration device, or by sounding a buzzer or other sound from the first speaker device 152 of the vehicle navigation system 15 or the second speaker device 162 of the display audio system 16.

[0108] Furthermore, the information generation unit 223 may generate information for adding an evaluation value on the display unit of the first passenger terminal 12 held by the first passenger E1. This information is for the first passenger E1 to support the driver D. Specifically, the information may include text information displayed when vehicle X arrives near the destination, such as "We are near our destination, let's say 'good job'", text information displayed when conversations between occupants in vehicle X have decreased and the engine of vehicle X is ON, such as "Why don't you put on some music?", and text information displayed when driver D has not had a drink for a long time and appears thirsty, such as "You might be thirsty".

[0109] The output control unit 224 displays the presentation information generated by the presentation information generation unit 223 on the display unit of the first passenger terminal 12 held by the first passenger E1, for example. When the first passenger E1 recognizes the displayed presentation information and takes an action to support the driver D, the evaluation value calculation unit 222 detects this and adds to the evaluation value of the first passenger E1.

[0110] This helps to minimize the workload on driver D, and if the workload does increase, it can guide passengers E1 and E2 to take actions that reduce the workload.

[0111] [Specific example (5): Output of information presented at the time of drive termination] At the end of the drive, the evaluation values ​​calculated by the evaluation value calculation unit 222 for driver D, first passenger E1, and second passenger E2, along with the amount of the cost share for the drive calculated based on these evaluation values, are output from at least one of the driver terminal 11, first passenger terminal 12, second passenger terminal 13, vehicle navigation system 15, or display audio system 16. The cost share can be calculated based on the driving expenses, such as gasoline costs and highway tolls, according to each passenger's contribution to the drive.

[0112] The process of generating the presented information described above may be configured to be executed using AI (Artificial Intelligence).

[0113] [Effects of the Embodiment] According to the embodiment described above, the information processing device includes a load information acquisition unit that acquires driver load information relating to the load on the vehicle driver and passenger load information relating to the load on the vehicle passengers, and an evaluation value calculation unit that calculates evaluation values ​​that evaluate the load on the driver and passengers due to the vehicle's operation based on the driver load information and passenger load information.

[0114] This allows for the accurate calculation of evaluation values ​​that assess the driver's and passenger's workload based on the specific circumstances of the driver and passenger. Furthermore, these calculated evaluation values ​​can be used to provide information that helps drivers to drive comfortably and safely.

[0115] Furthermore, driver load information may refer to information regarding the driver's physical and mental load, and passenger load information may refer to information regarding the passenger's physical and mental load. This allows for the calculation of evaluation values ​​for both the driver and passenger with even greater accuracy.

[0116] Furthermore, the driver load information may be weighted based on at least one of the driver's driving preferences or the functions installed in the vehicle, and the evaluation value calculation unit may use the weighted information to calculate the evaluation value. This makes it possible to calculate a more accurate evaluation value by taking into account the driver's driving preferences.

[0117] Furthermore, by programming the functional configuration of the information processing device 20 described above and incorporating it into a computer, it is also possible to construct an information processing program that allows the computer to function as the information processing device 20. [Explanation of Symbols]

[0118] 1. Driver assistance system 11. Driver terminal 12. First passenger terminal 13. Second passenger terminal 14. Dashcam 15. Vehicle navigation system 16 Display Audio Device 17. Operation control device 18. Vibration devices 20 Information Processing Devices 21 Memory section 141 Front camera device 142 Rear camera device 143 Sound collection device 151 1st display device 152 First speaker device 161 2nd display device 162 Second speaker unit 211 Driver Information Storage Unit 212 Passenger Information Storage Unit 221 Load information acquisition section 222 Evaluation Value Calculation Unit 223 Presentation information generation unit 224 Output Control Unit

Claims

1. A load information acquisition unit that acquires driver load information relating to the load on the vehicle's driver, and passenger load information relating to the load on the vehicle's passengers, An information processing device comprising: an evaluation value calculation unit that calculates evaluation values ​​that evaluate the load on the driver and the passenger due to the operation of the vehicle, based on the driver load information and the passenger load information.

2. The aforementioned driver load information is information relating to the driver's physical and mental load, The information processing apparatus according to claim 1, wherein the passenger load information is information relating to the physical and mental load of the passenger.

3. The aforementioned driver load information is weighted based on at least one of the driver's driving preferences or the functions installed in the vehicle. The information processing apparatus according to claim 1, wherein the evaluation value calculation unit calculates the evaluation value using the weighting information.

4. Information processing device, The system acquires driver load information relating to the load on the vehicle's driver, and passenger load information relating to the load on the vehicle's passengers. An information processing method that calculates evaluation values ​​based on the driver load information and passenger load information to evaluate the load on the driver and passenger due to the operation of the vehicle.

5. In an information processing device, A function to acquire driver load information relating to the load on the vehicle driver, and passenger load information relating to the load on the passengers of the vehicle, An information processing program for implementing a function that calculates evaluation values ​​that evaluate the loads imposed on the driver and passengers by the operation of the vehicle, based on the driver load information and passenger load information.

Images