Information processing device, information processing method, and program

Abstract

To execute driving support in consideration of a decrease in driving skill due to the presence of a fellow passenger, and to cause a vehicle to feel embarrassed by executing the driving support in the middle of driving.SOLUTION: To determine whether or not a fellow passenger other than a driver is on a vehicle. To evaluate the discomfort degree of a driver. When a fellow passenger is riding in the vehicle and the discomfort degree is equal to or greater than the first threshold value, driving support of the vehicle is executed.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program. [Background Art]

[0002] There is a technology that provides driving support according to a driver's emotional state while the driver is operating a vehicle. Patent Document 1 discloses that an emotion map divided into a pleasure region where the driver feels the emotion of pleasure, a boredom region where the driver feels the emotion of boredom, and an anxiety region where the driver feels anxiety is set with environmental difficulty and driving skill as parameters, and describes that when the driving skill is equal to or higher than a predetermined value and the driver's state is in the anxiety region, driving support is provided to shift the state to the pleasure region, while when the driving skill is lower than the predetermined value, autonomous driving is performed regardless of the environmental difficulty. [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2015-110417 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] However, when there is a passenger on board, the driver's driving skill may fluctuate due to tension, impatience, or the like. In addition, when there is a passenger, the driver does not want to make driving mistakes, but may feel embarrassed to turn on driving assistance in the middle of driving, and the technology described in Patent Document 1 has a problem in that it cannot accommodate such driver's emotions.

[0005] Accordingly, an object of the present invention is to perform driving support in consideration of a decrease in driving skill caused by the presence of a passenger, and to attribute the embarrassment caused by activating driving assistance in the middle of driving to the vehicle. [Means for Solving the Problem]

[0006] According to the present invention, A determination means for determining whether or not there are passengers other than the driver in the vehicle, Estimation means for estimating the driver's discomfort index, The degree of discomfort of the aforementioned driver Based on the aforementioned unpleasant emotion index, the elapsed time, and The evaluation methods used for evaluation, An execution means for performing driving assistance for the vehicle when the passenger is in the vehicle and the level of discomfort is equal to or greater than a first threshold, Equipped with 、 The level of discomfort becomes equal to or greater than the first threshold when the state in which the discomfort index is equal to or greater than the second threshold continues for a threshold time or longer. An information processing device is provided. [Effects of the Invention]

[0007] According to the present invention, driving assistance is provided while taking into account the decline in driving skills due to the presence of a passenger, and it is also possible to blame the embarrassment caused by providing driving assistance midway through driving on the vehicle. [Brief explanation of the drawing]

[0008] [Figure 1] Block diagram of the vehicle and its control system. [Figure 2] Diagram showing the driver assistance features performed in each mode. [Figure 3] A block diagram showing an example of the functional configuration of an information processing device. [Figure 4] A flowchart illustrating an example of information processing by an information processing device. [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more of the features described in the embodiments may be combined in any way. Furthermore, identical or similar configurations will be given the same reference numeral, and redundant descriptions will be omitted.

[0010] An embodiment of the present invention will now be described. Figure 1 is a block diagram of a vehicle V and its control device CNT according to this embodiment. In Figure 1, the general outline of the vehicle V is shown in a plan view and a side view. The vehicle V in this embodiment is, for example, a sedan-type four-wheeled passenger car, and may be, for example, a parallel hybrid vehicle. Note that the vehicle V is not limited to a four-wheeled passenger car, and may be a saddle-type vehicle (motorcycle, motorized tricycle), or a large vehicle such as a truck or bus.

[0011] [Configuration of the vehicle's control system] The control device CNT includes a controller 1, which is an electronic circuit that performs control of the vehicle V, including driving assistance for the vehicle V. The controller 1 comprises a plurality of ECUs (Electronic Control Units). The ECUs are provided, for example, for each function of the control device CNT. Each ECU includes a processor, such as a CPU (Central Processing Unit), a storage device such as semiconductor memory, and an interface with an external device. The storage device stores programs executed by the processor or data used by the processor for processing. The interface includes an input / output interface and a communication interface. Each ECU may comprise a plurality of processors, a plurality of storage devices, and a plurality of interfaces.

[0012] Controller 1 controls the driving (acceleration) of vehicle V by controlling power unit (power plant) 2. Power unit 2 is a drive unit that outputs driving force to rotate the drive wheels of vehicle V, and may include an internal combustion engine, a motor, and an automatic transmission. The motor can be used as a driving source to accelerate vehicle V, and can also be used as a generator during deceleration (regenerative braking).

[0013] In this embodiment, the controller 1 controls the output of the internal combustion engine or motor, or switches the gear of the automatic transmission, in response to the driver's driving operations and vehicle speed detected by the operation detection sensor 2a provided on the accelerator pedal AP or the operation detection sensor 2b provided on the brake pedal BP. The automatic transmission is equipped with a rotation speed sensor 2c that detects the rotation speed of the output shaft of the automatic transmission as a sensor that detects the driving state of the vehicle V. The vehicle speed of the vehicle V can be calculated from the detection result of the rotation speed sensor 2c.

[0014] Controller 1 controls the braking (deceleration) of the vehicle V by controlling the hydraulic system 3. The driver's braking operation applied to the brake pedal BP is converted into hydraulic pressure in the brake master cylinder BM and transmitted to the hydraulic system 3. The hydraulic system 3 is an actuator capable of controlling the hydraulic pressure of the hydraulic fluid supplied to the brake systems 3a (e.g., disc brake systems) provided on each of the four wheels, based on the hydraulic pressure transmitted from the brake master cylinder BM.

[0015] Controller 1 can control the braking of the vehicle V by controlling the drive of solenoid valves and other components of the hydraulic system 3. Controller 1 can also configure an electric servo brake system by controlling the distribution of braking force from the brake device 3a and braking force from regenerative braking of the motor in the power unit 2. Controller 1 may also illuminate the brake lamp 3b during braking.

[0016] Controller 1 controls the steering of the vehicle V by controlling the electric power steering system 4. The electric power steering system 4 includes a mechanism that steers the front wheels in response to the driver's driving operation (steering operation) on the steering wheel ST. The electric power steering system 4 includes a drive unit 4a that includes a motor that provides driving force (sometimes referred to as steering assist torque) to assist the steering operation or to automatically steer the front wheels, a steering angle sensor 4b, a torque sensor 4c that detects the steering torque borne by the driver (referred to as steering load torque and distinguished from steering assist torque), and the like.

[0017] Controller 1 controls the electric parking brake device 3c provided on the rear wheels. The electric parking brake device 3c includes a mechanism that locks the rear wheels. The controller 1 is capable of controlling locking and unlocking of the rear wheels by the electric parking brake device 3c.

[0018] Controller 1 controls the information output device 5 that notifies information inside the vehicle. The information output device 5 includes, for example, a display device 5a that notifies information to a driver via an image and / or an audio output device 5b that notifies information to the driver via audio. The display device 5a may be provided, for example, on an instrument panel or a steering wheel ST. The display device 5a may be a head-up display. The information output device 5 may notify information to an occupant via vibration or light.

[0019] Controller 1 receives an instruction input from an occupant (e.g., a driver) via the input device 6. The input device 6 is disposed at a position operable by the driver, and includes, for example, a switch group 6a via which the driver gives an instruction to the vehicle V and / or a turn signal lever 6b that activates a direction indicator (turn signal).

[0020] Controller 1 recognizes and determines the current position and course (attitude) of vehicle V. In this embodiment, vehicle V is equipped with a gyro sensor 7a, a GNSS (Global Navigation Satellite System) sensor 7b, and a communication device 7c. The gyro sensor 7a detects the rotational motion (yaw rate) of vehicle V. The GNSS sensor 7b detects the current position of vehicle V. The communication device 7c also communicates wirelessly with a server that provides map information and traffic information to acquire this information. In this embodiment, controller 1 determines the course of vehicle V based on the detection results of the gyro sensor 7a and the GNSS sensor 7b, and sequentially acquires high-precision map information related to the course from the server via the communication device 7c and stores it in the database 7d (storage device). Vehicle V may also be equipped with sensors to detect the state of vehicle V, such as a speed sensor to detect the speed of vehicle V and an acceleration sensor to detect the acceleration of vehicle V.

[0021] Controller 1 performs driving assistance for vehicle V based on the detection results of various detection units installed in vehicle V. Vehicle V is equipped with external sensors, which are ambient detection units 8a to 8b, that detect the outside of vehicle V (surrounding conditions), and internal sensors, which are internal detection units 9a to 9b, that detect the conditions inside the vehicle (driver's condition). Controller 1 can grasp the surrounding conditions of vehicle V based on the detection results of ambient detection units 8a to 8b and perform driving assistance according to those conditions. In addition, controller 1 can determine whether the driver is performing the predetermined actions required of the driver when performing driving assistance, based on the detection results of internal detection units 9a to 9b.

[0022] The surrounding detection unit 8a is an imaging device that captures images of the area in front of the vehicle V (hereinafter sometimes referred to as the front camera 8a), and is mounted, for example, on the interior side of the front windshield at the front of the roof of the vehicle V. The controller 1 can extract the contours of objects and the lane markings (white lines, etc.) on the road by analyzing the images captured by the front camera 8a.

[0023] The surrounding detection unit 8b is a millimeter-wave radar (hereinafter sometimes referred to as radar 8b) that uses radio waves to detect targets around the vehicle V and detects (measures) the distance to the targets and the direction (azimuth) of the targets relative to the vehicle V. In the example shown in Figure 1, there are five radar 8b units: one in the center of the front of the vehicle V, one at each of the left and right corners of the front, and one at each of the left and right corners of the rear.

[0024] The surrounding detection unit installed on vehicle V is not limited to the above configuration; the number of cameras and radars may be changed, and a LiDAR (Light Detection and Ranging) for detecting targets around vehicle V may also be installed.

[0025] The in-vehicle detection unit 9a is an imaging device that captures images of the interior of the vehicle (hereinafter sometimes referred to as the in-vehicle camera 9a), and is installed, for example, on the interior side of the front part of the roof of the vehicle interior V. In this embodiment, the in-vehicle camera 9a is a driver monitor camera that captures images of the driver (for example, the driver's eyes and face). The controller 1 can determine the driver's gaze and the direction of their face by analyzing the image (driver's face image) captured by the in-vehicle camera 9a.

[0026] The in-vehicle detection unit 9b is a grip sensor (hereinafter sometimes referred to as grip sensor 9b) that detects the driver's grip on the steering wheel ST, and is provided, for example, on at least a part of the steering wheel ST. Alternatively, a torque sensor 4c that detects the driver's steering torque may be used as the in-vehicle detection unit.

[0027] Examples of driver assistance for vehicle V include acceleration / deceleration assistance, lane keeping assistance, lane change assistance, and parking assistance. Acceleration / deceleration assistance is an adaptive cruise control (ACC) system that controls the acceleration and deceleration of vehicle V within a predetermined speed while maintaining a safe distance from the vehicle ahead by controlling the power unit 2 and the hydraulic system 3. Lane keeping assistance is a lane keeping assist system (LKAS) that controls the electric power steering system 4 to keep vehicle V within its lane. Lane change assistance is an auto lane changing (ALC, ALCA) system that controls the electric power steering system 4 to change the vehicle V's lane to an adjacent lane. Advanced parking assist (APA) is a driver assistance system that controls the power unit 2, the hydraulic system 3, and the electric power steering system 4 to park vehicle V in a predetermined position. The parking assistance also includes providing instructions to the driver regarding the operation to be performed in manual driving mode based on the detection results of the surrounding detection units 8a to 8b (for example, displaying an image of the area behind the vehicle, or notifying the driver of the accuracy of the parking position with an audible alert). Furthermore, the driving assistance performed by the controller 1 may also include collision mitigation braking, ABS function, traction control, and / or vehicle V attitude control, which assist in avoiding collisions with road targets (for example, pedestrians, other vehicles, or obstacles) by controlling the hydraulic system 3.

[0028] The driver assistance for vehicle V (acceleration / deceleration assistance, lane keeping assistance, lane change assistance, and parking assistance) is performed in multiple driving modes, including manual driving mode, normal assistance mode, and extended assistance mode. Figure 2 shows the driver assistance performed in each of the manual driving mode, normal assistance mode, and extended assistance mode of this embodiment. In manual driving mode, acceleration / deceleration assistance, lane keeping assistance, and lane change assistance are not performed, and the driver manually operates vehicle V. If parking assistance is performed in manual driving mode, the driver manually operates vehicle V along with the parking assistance.

[0029] In manual driving mode, if the driver gives an instruction to set adaptive cruise control (ACC) via input device 6 (e.g., switch group 6a), the ACC is started and the system switches from manual driving mode to normal assistance mode. In normal assistance mode, in addition to ACC, lane keeping assist (LKAS) can be performed. Lane keeping assist is started if the driver gives an instruction to set lane keeping assist via input device 6 (e.g., switch group 6a) while ACC is set. Both ACC and lane keeping assist are terminated if the driver gives an instruction to cancel their settings via input device 6 (e.g., switch group 6a).

[0030] In addition, in normal assistance mode, the driver is required to perform certain actions, such as monitoring the surroundings and holding the steering wheel. If the in-vehicle detection unit 9b determines that the driver has not performed the required actions, a notification is sent via the information output device 5 to prompt the driver to perform the required actions.

[0031] If driving on a specific road begins while the normal support mode is running, high-precision map information is acquired by the communication device 7c. If the high-precision map information is successfully matched with the image captured by the front camera 8a, the mode automatically switches from normal support mode to extended support mode. Specific roads are roads for which high-precision map information is provided, such as expressways or motorways. In addition to normal information such as the route and location of the specific road, the high-precision map information includes detailed information about the shape of the specific road, such as the presence and curvature of curves, changes in the number of lanes, and gradients. When the mode switches from normal support mode to extended support mode, the information output device 5 notifies the driver that the mode has been switched to extended support mode, for example, by changing the illumination color of the display device 5a on the steering wheel ST.

[0032] In the extended support mode, acceleration and deceleration assistance (and lane keeping assistance) is provided in conjunction with high-precision map information. For example, based on high-precision map information, the controller 1 can perform more advanced acceleration and deceleration assistance than in the normal support mode, such as decelerating the vehicle V before a curve or before a point where the lane narrows, or adjusting the speed of the vehicle V according to the curvature of the curve. In the extended support mode, as in the normal support mode, the driver is required to perform certain actions, such as monitoring the surroundings and holding the steering wheel. If the in-vehicle detection unit 9b determines that the driver has not performed the required actions, a notification prompting the driver to perform the required actions is sent via the information output device 5.

[0033] Furthermore, in extended support mode, lane change assistance can be further implemented. The lane change assistance according to this embodiment includes system-led lane change assistance (ALC: Auto Lane Changing), which automatically changes lanes based on the judgment of the controller 1, and driver-led lane change assistance (ALCA: Active Lane Change Assist), which automatically changes lanes in response to the driver's instruction input. In both system-led lane change assistance (ALC) and driver-led lane change assistance (ALCA), the driver is required to perform predetermined actions such as monitoring the surroundings and holding the steering wheel when performing lane change assistance.

[0034] System-driven lane change assist (ALC) is initiated when the driver gives an instruction to set ALC via input device 6 (e.g., switch group 6a) in extended assist mode. While ALC is being set, controller 1 sequentially determines, based on high-precision map information (information such as lane additions, deletions, or branching), whether a lane change is necessary to reach a destination set in advance by the driver, and automatically performs a lane change if it determines that a lane change is necessary. While ALC is being set, one or more lane changes can be performed depending on the controller 1's determination. ALC ends when the destination is reached or when a specific road ends. ALC may also end when the driver gives an instruction to cancel the setting via input device 6 (e.g., switch group 6a).

[0035] Driver-initiated lane change assistance (ALCA) performs a single lane change in response to driver input. In extended assistance mode, ALCA is executed when the driver gives an instruction to perform ALCA via the input device 6 (e.g., turn signal lever 6b). With ALCA, the driver can input an instruction for the direction in which they wish to change lanes via the input device 6 (turn signal lever 6b), and the controller 1 automatically performs a lane change to the adjacent lane in the direction instructed by the driver. ALCA can also be executed while system-initiated lane change assistance (ALC) is set.

[0036] Furthermore, parking assistance can be further performed in normal assistance mode and extended assistance mode. The parking assistance according to this embodiment includes parking assistance (APA) in normal assistance mode and extended assistance mode, which automatically performs parking based on the judgment of the controller 1, and parking assistance in manual driving mode (manual parking assistance) which presents instructions to the driver when parking. When performing parking assistance, the driver is required to perform certain actions such as monitoring the surroundings and holding the steering wheel.

[0037] In normal and extended support modes, parking assistance (APA) is initiated when the driver gives an instruction to set up APA via the input device 6 (e.g., switch group 6a). Alternatively, APA may be initiated automatically without instruction input when the destination is reached. While APA is running, the controller 1 calculates the target parking position and the amount of driving control required to reach that position based on, for example, the detection results of the radar 8b, and automatically parks the vehicle to that position. Manual parking assistance also provides support for the execution of such driving control, such as by voice or image display. In this parking assistance, the target parking position is highlighted along with an image of the rear of the vehicle, or instructions for parking operations to reach that position are given via image display or voice. Manual parking assistance may also restrict control of the electric power steering system 4, etc., to prevent exceeding such driving control.

[0038] The information processing device 100 according to this embodiment will be described below with reference to Figure 3. Figure 3 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a determination unit 111, an estimation unit 112, an evaluation unit 113, and an execution unit 114. Each of these functional units will be described below. The information processing device 100 may be implemented as a control device CNT, or as a server or external device capable of communicating with the control device CNT, and there are no limitations on its form as long as it can similarly perform driving support.

[0039] The determination unit 111 determines whether or not there are passengers other than the driver in the vehicle V. For example, the determination unit 111 can determine that there are passengers in the vehicle V if the seat belts in seats other than the driver's seat are fastened, using the seat belt sensors provided in the vehicle V. Alternatively, the determination unit 111 may determine whether or not there are passengers by detecting faces in each seat based on images captured by a camera that captures images of the interior of the vehicle. Furthermore, the presence of passengers can be detected by any method, such as determining the presence or absence of passengers using weight sensors provided in each seat.

[0040] The estimation unit 112 can estimate the driver's discomfort index. The driver's discomfort index is a value that evaluates the degree to which the driver feels discomfort among their emotions. In this embodiment, the discomfort index is represented by a value from 1 to 5, but this value is not particularly limited. Examples of discomfort indexes in this embodiment will be described below.

[0041] The estimation unit 112 can estimate an unpleasant emotion index (hereinafter sometimes referred to as the image unpleasant emotion index) based on, for example, an image of the driver. Alternatively, the designation unit 112 may estimate an unpleasant emotion index (hereinafter sometimes referred to as the voice unpleasant emotion index) based on voice (hereinafter simply referred to as the user voice) collected by a sound collection device positioned to collect the driver's voice. In the following, several examples of the image unpleasant emotion index and the voice unpleasant emotion index will be described, but the estimation unit 112 may select any of these multiple unpleasant emotion indices as the overall unpleasant emotion index, or it may estimate the unpleasant emotion index based on these multiple unpleasant emotion indices (for example, by adding them all together). The estimation unit 112 may also calculate the amount of unpleasant emotion index added per unit of time for each item for which the unpleasant emotion index is estimated, and estimate the overall unpleasant emotion index by performing the respective addition process.

[0042] For example, the estimation unit 112 may estimate an unpleasant emotion index corresponding to an expression based on the driver's facial image captured by the in-vehicle camera 9a. In this case, the estimation unit 112 may output an unpleasant emotion index corresponding to the expression, and may add a variation amount corresponding to the expression to an initial value of the unpleasant emotion index (e.g., 0 or 2).

[0043] The estimation unit 112 can estimate the driver's discomfort index from a facial image based on a machine learning model that has been pre-trained to take a human facial image as input and output a corresponding discomfort index (or its fluctuation). In this embodiment, a machine learning model can be used that recognizes facial expressions that are pre-set as indicating discomfort or tension, such as sweating, having a pale complexion, frowning, or wide-eyed, and sets the discomfort index accordingly. For example, the estimation unit 112 may increase the discomfort index by 1 every 30 seconds while a tense facial expression is recognized, and decrease the discomfort index by 1 every 30 seconds while no facial expression that increases the discomfort index is recognized.

[0044] The estimation unit 112 may also estimate the discomfort index based on the driver's posture recognized from the image of the driver. Here, for example, if a predetermined posture is detected, such as leaning forward or having stiff shoulders, the system may set a discomfort index corresponding to that posture for the user (or increase the discomfort index set for the vehicle by 1). The estimation unit 112 can estimate the driver's discomfort index from an image based on a machine learning model that has been pre-trained to take a human image (especially an image of the upper body) as input and output a corresponding discomfort index. With such a configuration, it is possible to perform estimation processing such that the discomfort index is high when the vehicle is in a tense posture and low when the vehicle is in a relaxed posture.

[0045] Furthermore, the estimation unit 112 may estimate the unpleasant emotion index based on the user's voice, as described above. The sound collection device used here is not particularly limited as long as it is capable of collecting the driver's voice. The sound collection device may be, for example, an in-vehicle camera 9a, an in-vehicle device such as a steering wheel ST, a dedicated microphone worn by the driver, or a mobile terminal such as a smartphone connected to the vehicle V.

[0046] The estimation unit 112 may estimate the displeasure index based, for example, on the tone of the user's voice. The estimation unit 112 may set a displeasure index for the driver corresponding to a specific tone when the tone of the user's voice reaches that tone (or increase the displeasure index set for the driver by 1). The "specific tone" here can be arbitrarily set according to the expected user group, and for example, a tone above a certain level or below a certain level that is expected to be emitted by an abnormal person may be used. Alternatively, the estimation unit 112 may estimate the driver's displeasure index based on the tone of the user's voice over a predetermined period. For example, the estimation unit 112 may set a range of tones considered normal (for example, a predetermined range from the median) based on the tone of the user's voice collected over a certain period, and increase the displeasure index by 1 when a user voice with a tone outside that range is recognized. The estimation unit 112 may also decrease the displeasure index set for the driver by 1 if a predetermined period of time passes during which no user voice with a specific tone as described above is recognized.

[0047] The estimation unit 112 may also estimate the displeasure index based on, for example, the speaking speed of the user's voice. Here, the speaking speed is evaluated by the number of Japanese characters recognized from the voice, which is the number of characters per minute, but it is not limited to this example as long as the speed can be evaluated in a similar way. The estimation unit 112 may set a displeasure index for the driver corresponding to the speaking speed if the speaking speed is above a predetermined threshold (or increase the displeasure index set for the driver by 1). The threshold for speaking speed can be arbitrarily set according to the expected user group, etc. The estimation unit 112 may also estimate the driver's displeasure index based on the speaking speed of the user's voice over a predetermined period. For example, the estimation unit 112 may set a threshold for speaking speed based on the speaking speed of the user's voice collected over a certain period (for example, as the maximum value within that certain period), and increase the displeasure index by 1 if a user's voice at a speed above that threshold is recognized. Furthermore, the estimation unit 112 may lower the displeasure index set for the driver by 1 if a predetermined period of time occurs during which no user voice at a speed exceeding the threshold described above is recognized.

[0048] The estimation unit 112 may also estimate an unpleasant emotion index based on the content of the utterances included in the user's voice. For example, if the estimation unit 112 determines that the driver is cursing, such as when a sigh is recognized in the user's voice, it may set an unpleasant emotion index for the driver corresponding to the cursing (or increase the unpleasant emotion index already set for the driver by 1). Here, the content recognized as cursing is not particularly limited, and the determination may also be made using a facial image. Furthermore, for example, if the user's voice contains a specific statement, the estimation unit 112 may set an unpleasant emotion index for the driver corresponding to that statement (or increase the unpleasant emotion index already set for the driver by 1). The "specific statement" here can be arbitrarily set according to the expected user group, and may include statements that a nervous driver might utter, such as "this is bad," "this is bad," or "wow." Furthermore, for example, the estimation unit 112 may set the driver's unpleasant emotion index based on the volume of the user's voice. For example, the estimation unit 112 may set a volume threshold (for example, as a median) based on the volume of voices collected over a certain period, and increase the discomfort index by 1 if user voices exceeding that threshold are recognized. Alternatively, the estimation unit 112 may decrease the discomfort index set for the driver by 1 if a predetermined period of time is not observed during which specific statements or user voices exceeding the set volume threshold are not recognized.

[0049] Furthermore, the estimation unit 112 can estimate the driver's discomfort index from the user's voice based on a machine learning model that has been pre-trained to output a corresponding discomfort index as input to human voice. In this embodiment, a machine learning model can be used that recognizes user voice containing the occurrence of a specific tone, specific statements, or voice waveforms of statements made when feeling uncomfortable, and sets the discomfort index. With such a configuration, it becomes possible to set the discomfort index based on the user's voice while driving.

[0050] Furthermore, the discomfort index according to this embodiment may be estimated based on the driver's biometric information. Here, biometric information refers to, for example, the driver's pulse, heart rate, blood pressure, amount of sweat, respiratory rate, electroencephalogram, body temperature, or information that can be derived from any of these, but is not limited to these as long as it is information that is generally acquired as biometric information. The driver's biometric information may be measured by sensors provided in the vehicle V, by external devices (not shown) installed in the vehicle V, or by devices worn or held by the driver. For example, the amount of sweat, heart rate, or body temperature measured by sensors provided on the steering wheel ST may be used as biometric information, or the pulse or blood pressure measured by a device worn on the user's wrist may be used.

[0051] The evaluation unit 113 evaluates the driver's level of discomfort. Here, discomfort level is an evaluation value that assesses the discomfort the driver is feeling. The evaluation unit 113 can evaluate the level of discomfort based, for example, on the discomfort emotion index estimated by the estimation unit 112 and the elapsed time.

[0052] The evaluation unit 113 according to this embodiment can evaluate the level of discomfort in two ways: 1, which indicates that the driver is uncomfortable, or 0, which indicates that the driver is not uncomfortable. In this case, the evaluation unit 113 can, for example, set the initial value of the level of discomfort to 0, and set the level of discomfort to 1 if the discomfort index remains above a predetermined threshold (e.g., 3) (hereinafter referred to as the evaluation threshold) for a predetermined threshold time (e.g., 1 minute). Alternatively, if the level of discomfort is 1, the evaluation unit 113 can set the level of discomfort to 0 if the discomfort index remains below the evaluation threshold for a predetermined threshold time (e.g., 1 minute).

[0053] Furthermore, if the passenger is someone who requires safer driving than usual, someone the driver doesn't want to embarrass, or someone the driver isn't very close to, the driver's driving skills are more likely to decline than normal. From this perspective, if the passenger meets these predetermined conditions (safety conditions), the threshold time used to evaluate the degree of discomfort as described above may be reduced, or the evaluation threshold may be reduced. The safety conditions used here may include, for example, the passenger being under a certain age, the passenger being of the opposite sex to the driver, or the passenger being someone riding in vehicle V for the first time, but other conditions can be set arbitrarily. This processing makes it easier to perform driving assistance at a more appropriate timing according to the attributes of the passenger.

[0054] Alternatively, the evaluation unit 113 may evaluate the degree of discomfort by multiplying the discomfort index by the elapsed time. For example, if the discomfort index is updated every 30 seconds, the evaluation unit 113 may calculate an evaluation value of the discomfort index × 0.5 (minutes) for each such update unit, and evaluate the degree of discomfort by summing up these evaluation values ​​for the most recent predetermined period (for example, 3 minutes). In this case, for example, if the most recent 3 minutes consist of 1 minute with an discomfort index of 0, 0.5 minutes with an index of 2, 0.5 minutes with an index of 4, and 1 minute with an index of 3, the degree of discomfort would be calculated as 6.

[0055] The execution unit 114 performs driving assistance for vehicle V if there is a passenger in vehicle V and the level of discomfort is above a threshold (execution threshold). The driving assistance can be the one described with reference to Figure 2, but any other arbitrary processing used as a driving assistance function for a vehicle may be executed. For example, if the level of discomfort is evaluated as 1 or 0, the execution unit 114 may perform driving assistance for vehicle V if the level of discomfort is 1 (setting the execution threshold to 1).

[0056] For example, if the degree of discomfort is evaluated by multiplying the discomfort index by the elapsed time, the execution unit 114 can set the execution threshold to, for example, 5. In this case, the execution threshold may be set in advance, or it may be set based on the degree of discomfort over a predetermined period, such as setting the execution threshold to the maximum value of the degree of discomfort evaluated during driving in the most recent predetermined period (for example, one week) + 1. Furthermore, if the passengers meet the safety conditions described above, the execution threshold may be reduced.

[0057] The following describes the execution process of the driver support performed by the information processing device 100 according to this embodiment, with reference to Figure 4. Figure 4 is a flowchart showing an example of information processing by the information processing device 100. The process shown in Figure 4 is started by the processing unit 201 of the information processing device 100 when an operation start instruction is given on the application.

[0058] In S401, the determination unit 111 determines whether or not there is a passenger in the vehicle V. If there is a passenger, the process proceeds to S402; otherwise, the process ends.

[0059] In S402, the estimation unit 112 estimates the driver's displeasure index. Here, the displeasure index is estimated based on the driver's facial image and voice. In S403, the evaluation unit 113 evaluates the driver's level of displeasure. Here, the level of displeasure is evaluated based on the displeasure index estimated in S402, the elapsed time, and other factors.

[0060] In S404, the execution unit 114 determines whether the level of discomfort evaluated in S403 is equal to or greater than the execution threshold. If the level of discomfort is equal to or greater than the threshold, the process proceeds to S405; otherwise, the process ends. In S405, the execution unit 114 performs driving assistance and terminates the process shown in Figure 4.

[0061] This process evaluates the driver's level of discomfort and enables the vehicle's driving assistance to be activated if there are passengers in the vehicle and the level of discomfort exceeds a certain threshold. Therefore, it becomes possible to activate driving assistance while taking into account the decline in driving skills due to the presence of passengers. Furthermore, it becomes possible to blame the vehicle for any embarrassment caused by activating driving assistance mid-drive.

[0062] The values ​​exemplified in this embodiment are merely examples, and any values ​​may be used as long as they allow for the execution of each process in the same manner. For example, in this embodiment, the level of discomfort is set to a 5-point scale from 1 to 5, but these could also be set to a 500-point scale from 0.01 to 5.00, using two decimal places.

[0063] [Summary of Embodiments] The above embodiments disclose at least the following information processing device, information processing method, and program.

[0064] 1. The information processing device of the above embodiment is: A determination means for determining whether or not there are passengers other than the driver in the vehicle, An evaluation means for evaluating the degree of driver discomfort, An execution means for performing driving assistance for the vehicle when the passenger is in the vehicle and the level of discomfort is equal to or greater than a first threshold, It is equipped with. According to this embodiment, driving assistance is performed taking into account the decline in driving skills due to the presence of a passenger, and it becomes possible to blame the embarrassment caused by performing driving assistance in the middle of driving on the vehicle.

[0065] 2. In the information processing device of the above embodiment, The first threshold described above decreases when the passenger meets a predetermined condition. According to this embodiment, it becomes possible to perform driving assistance at a more appropriate timing depending on whether or not the passenger meets the conditions.

[0066] 3. In the information processing device of the above embodiment, The aforementioned predetermined conditions include at least one of the following: the passenger is under a certain age; the passenger is of the opposite sex to the driver; or the passenger is a person riding in the vehicle for the first time. According to this embodiment, it becomes possible to perform driving assistance at a more appropriate timing depending on the attributes of the passenger.

[0067] 4. In the information processing device of the above embodiment, The system further comprises estimation means for estimating the driver's discomfort index, The evaluation means evaluates the degree of discomfort based on the discomfort index and the elapsed time. According to this embodiment, it is possible to evaluate the degree of discomfort based on the discomfort index estimated for the driver.

[0068] 5. In the information processing device of the above embodiment, The level of discomfort becomes equal to or greater than the first threshold when the state in which the discomfort index is equal to or greater than the second threshold continues for a threshold period of time or longer. According to this embodiment, safe driving is made possible by providing driving assistance in response to the continued state of discomfort experienced by the driver.

[0069] 6. In the information processing device of the above embodiment, The threshold time decreases when the passenger meets predetermined conditions. According to this embodiment, it becomes possible to perform driving assistance at a more appropriate timing depending on whether or not the passenger meets the conditions.

[0070] 7. In the information processing device of the above embodiment, The aforementioned predetermined conditions include at least one of the following: the passenger is under a certain age; the passenger is of the opposite sex to the driver; or the passenger is a person riding in the vehicle for the first time. According to this embodiment, it becomes possible to perform driving assistance at a more appropriate timing depending on the attributes of the passenger.

[0071] 8. In the information processing device of the above embodiment, The estimation means estimates the unpleasant emotion index based on at least one of the driver's facial expression, tone of voice, speaking speed, and content of speech. According to this embodiment, it is possible to accurately estimate the displeasure index based on information obtained from the driver inside the vehicle.

[0072] 9. In the information processing apparatus of the above embodiment, The estimation means estimates the unpleasant emotion index based on information acquired via an imaging device or a sound collection device. According to this embodiment, it becomes possible to estimate the unpleasant emotion index with greater accuracy.

[0073] 10. The information processing method of the above embodiment is: A determination process to determine whether or not there are passengers other than the driver in the vehicle, An evaluation step to evaluate the degree of driver discomfort, An execution step of performing driving assistance for the vehicle when the passenger is in the vehicle and the level of discomfort is equal to or greater than a first threshold, It is equipped with. According to this embodiment, driving assistance is performed taking into account the decline in driving skills due to the presence of a passenger, and it becomes possible to blame the embarrassment caused by performing driving assistance in the middle of driving on the vehicle.

[0074] 11. The program of the above embodiment causes the computer to function as one of the means of the information processing device described in any one of items 1 to 9. According to this embodiment, driving assistance is performed taking into account the decline in driving skills due to the presence of a passenger, and it becomes possible to blame the embarrassment caused by performing driving assistance in the middle of driving on the vehicle.

[0075] Although embodiments of the invention have been described above, the invention is not limited to the above embodiments, and various modifications and changes are possible within the scope of the gist of the invention. [Explanation of Symbols]

[0076] 111: Judgment unit, 112: Estimation unit, 113: Evaluation unit, 114: Execution unit

Claims

1. A determination means for determining whether or not there are passengers other than the driver in the vehicle, Estimation means for estimating the driver's discomfort index, An evaluation means for evaluating the driver's level of discomfort based on the discomfort index and the elapsed time, An execution means for performing driving assistance for the vehicle when the passenger is in the vehicle and the level of discomfort is equal to or greater than a first threshold, Equipped with, The aforementioned level of discomfort becomes equal to or greater than the first threshold when the state in which the discomfort emotion index is equal to or greater than the second threshold continues for a threshold time or longer.

2. The information processing apparatus according to claim 1, characterized in that the first threshold decreases when the passenger satisfies a predetermined condition.

3. The information processing apparatus according to claim 2, characterized in that the predetermined conditions include at least one of the following: the passenger is below a certain age; the passenger is of the opposite sex to the driver; or the passenger is a person riding in the vehicle for the first time.

4. The information processing apparatus according to claim 1, characterized in that the threshold time decreases when the passenger satisfies a predetermined condition.

5. The information processing apparatus according to claim 4, characterized in that the predetermined conditions include at least one of the following: the passenger is below a certain age; the passenger is of the opposite sex to the driver; or the passenger is a person riding in the vehicle for the first time.

6. The information processing apparatus according to claim 1, characterized in that the estimation means estimates the unpleasant emotion index based on at least one of the driver's facial expression, tone of voice, speaking speed, and content of speech.

7. The information processing apparatus according to claim 6, wherein the estimation means estimates the unpleasant emotion index based on information acquired via an imaging device or a sound collection device.

8. A determination means for determining whether or not there are passengers other than the driver in the vehicle, An evaluation means for evaluating the degree of driver discomfort, An execution means for performing driving assistance for the vehicle when the passenger is in the vehicle and the level of discomfort is equal to or greater than a first threshold, Equipped with, The information processing device is characterized in that the first threshold decreases when the passenger satisfies a predetermined condition including at least one of the following: the passenger is below a certain age; the passenger is of the opposite sex to the driver; or the passenger is a person riding in the vehicle for the first time.

9. An information processing method performed by an information processing device, A determination process to determine whether or not there are passengers other than the driver in the vehicle, An evaluation step to evaluate the degree of driver discomfort, An execution step of performing driving assistance for the vehicle when the passenger is in the vehicle and the level of discomfort is equal to or greater than a first threshold, An information processing method comprising:

10. A program for causing a computer to function as one of the means of an information processing device according to any one of claims 1 to 8.

Images