Information processing apparatus, information processing method, and program

Abstract

To execute driving support in consideration of a decrease in driving skill due to the presence of a fellow passenger, and to cause a vehicle to feel embarrassed by executing the driving support in the middle of driving.SOLUTION: To determine whether or not a fellow passenger other than a driver is on a vehicle. To evaluate the discomfort degree of a driver. When a fellow passenger is riding in the vehicle and the discomfort degree is equal to or greater than the first threshold value, driving support of the vehicle is executed.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is a technology that provides driving assistance according to the emotional state of the driver while driving a vehicle. Patent Document 1 describes a technology that sets an emotional map that uses environmental difficulty and driving skill as parameters and divides the driver's state into a fun area where the driver feels happy, a bored area where the driver feels bored, and an anxiety area where the driver feels anxious, and that provides driving assistance so that the driver's state becomes the fun area when the driving skill is above a predetermined value and the driver's state is in the anxiety area, while performing automatic driving when the driving skill is below the predetermined value regardless of the environmental difficulty. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-110417 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when there is a passenger, it is conceivable that the driver's driving skill may fluctuate due to tension, impatience, etc. Furthermore, when there is a passenger, the driver does not want to make driving mistakes, but may feel embarrassed to turn on driving assistance in the middle of driving, and the technology described in Patent Document 1 has a problem in that it is not able to take into consideration such driver's feelings.

[0005] Therefore, an object of the present invention is to provide driving assistance that takes into account the decline in driving skill due to the presence of a passenger, and to blame the embarrassment of providing driving assistance in the middle of a drive on the vehicle. [Means for solving the problem]

[0006] According to the present invention, a determination means for determining whether or not a passenger other than the driver is riding in the vehicle; evaluation means for evaluating the degree of discomfort of the driver; an execution means for executing driving assistance for the vehicle when the passenger is in the vehicle and the discomfort level is equal to or greater than a first threshold; An information processing device is provided. [Effects of the Invention]

[0007] According to the present invention, driving assistance is performed taking into consideration the decline in driving skill due to the presence of a passenger, and it is also possible to blame the vehicle for the embarrassment caused by performing driving assistance while driving. [Brief explanation of the drawings]

[0008] [Figure 1] Block diagram of a vehicle and its control device [Figure 2] A diagram showing the driving assistance provided in each mode [Figure 3] FIG. 1 is a block diagram showing an example of a functional configuration of an information processing apparatus. [Figure 4] 10 is a flowchart showing an example of information processing by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be combined in any desired manner. Furthermore, the same reference numerals are used to designate identical or similar components, and redundant descriptions will be omitted.

[0010] An embodiment of the present invention will be described. FIG. 1 is a block diagram of a vehicle V and its control device CNT according to this embodiment. FIG. 1 shows an outline of the vehicle V in plan view and side view. The vehicle V of this embodiment is, as an example, a four-wheeled sedan-type passenger vehicle, and may be, for example, a parallel hybrid vehicle. Note that the vehicle V is not limited to a four-wheeled passenger vehicle, and may be a saddle-type vehicle (motorcycle, motor tricycle), or a large vehicle such as a truck or bus.

[0011] [Configuration of vehicle control device] The control device CNT includes a controller 1, which is an electronic circuit that controls the vehicle V, including driving assistance for the vehicle V. The controller 1 includes multiple ECUs (Electronic Control Units). An ECU is provided, for example, for each function of the control device CNT. Each ECU includes a processor represented by a CPU (Central Processing Unit), a storage device such as a semiconductor memory, an interface with an external device, etc. The storage device stores programs executed by the processor or data used by the processor for processing, etc. The interfaces include an input / output interface and a communication interface. Each ECU may include multiple processors, multiple storage devices, and multiple interfaces.

[0012] The controller 1 controls the drive (acceleration) of the vehicle V by controlling a power unit (power plant) 2. The power unit 2 is a traveling drive unit that outputs drive force to rotate the drive wheels of the vehicle V, and may include an internal combustion engine, a motor, and an automatic transmission. The motor can be used as a drive source to accelerate the vehicle V, and can also be used as a generator during deceleration (regenerative braking).

[0013] In this embodiment, the controller 1 controls the output of the internal combustion engine or the motor, or controls shifting of the automatic transmission, in response to the driver's driving operation and vehicle speed detected by the operation detection sensor 2a provided on the accelerator pedal AP or the operation detection sensor 2b provided on the brake pedal BP. The automatic transmission is provided with a rotation speed sensor 2c that detects the rotation speed of the output shaft of the automatic transmission as a sensor that detects the running state of the vehicle V. The vehicle speed of the vehicle V can be calculated from the detection result of the rotation speed sensor 2c.

[0014] The controller 1 controls the braking (deceleration) of the vehicle V by controlling the hydraulic device 3. The driver's braking operation on the brake pedal BP is converted into hydraulic pressure in the brake master cylinder BM and transmitted to the hydraulic device 3. The hydraulic device 3 is an actuator that can control the hydraulic pressure of the hydraulic oil supplied to the brake devices 3a (e.g., disc brake devices) provided on each of the four wheels based on the hydraulic pressure transmitted from the brake master cylinder BM.

[0015] The controller 1 can control the braking of the vehicle V by controlling the driving of an electromagnetic valve or the like provided in the hydraulic device 3. The controller 1 can also configure an electric servo brake system by controlling the distribution of braking force by the brake device 3a and braking force by regenerative braking of the motor provided in the power unit 2. The controller 1 may also turn on the brake lamps 3b during braking.

[0016] The controller 1 controls the steering of the vehicle V by controlling the electric power steering device 4. The electric power steering device 4 includes a mechanism for steering the front wheels in response to the driver's driving operation (steering operation) with respect to the steering wheel ST. The electric power steering device 4 includes a drive unit 4a including a motor that generates a driving force (sometimes referred to as steering assist torque) for assisting the steering operation or for automatically steering the front wheels, a steering angle sensor 4b, and a torque sensor 4c that detects the steering torque borne by the driver (called steering burden torque, to be distinguished from steering assist torque).

[0017] The controller 1 controls an electric parking brake device 3c provided on the rear wheels. The electric parking brake device 3c has a mechanism for locking the rear wheels. The controller 1 can control the electric parking brake device 3c to lock and unlock the rear wheels.

[0018] The controller 1 controls an information output device 5 that notifies the passengers of information inside the vehicle. The information output device 5 includes, for example, a display device 5a that notifies the driver of information by image and / or an audio output device 5b that notifies the driver of information by audio. The display device 5a may be provided, for example, on an instrument panel or a steering wheel ST. The display device 5a may be a head-up display. The information output device 5 may notify the passengers of information by vibration or light.

[0019] The controller 1 receives instruction inputs from a passenger (e.g., the driver) via the input device 6. The input device 6 is arranged in a position operable by the driver, and includes, for example, a group of switches 6a through which the driver issues instructions to the vehicle V, and / or a turn signal lever 6b that activates a turn signal (blinker).

[0020] The controller 1 recognizes and determines the current position and course (attitude) of the vehicle V. In this embodiment, the vehicle V is provided with a gyro sensor 7a, a GNSS (Global Navigation Satellite System) sensor 7b, and a communication device 7c. The gyro sensor 7a detects the rotational motion (yaw rate) of the vehicle V. The GNSS sensor 7b detects the current position of the vehicle V. The communication device 7c wirelessly communicates with a server that provides map information and traffic information to acquire this information. In this embodiment, the controller 1 determines the course of the vehicle V based on the detection results of the gyro sensor 7a and the GNSS sensor 7b, and sequentially acquires high-precision map information related to the course from the server via the communication device 7c and stores it in a database 7d (storage device). The vehicle V may also be provided with sensors for detecting the state of the vehicle V, such as a speed sensor that detects the speed of the vehicle V and an acceleration sensor that detects the acceleration of the vehicle V.

[0021] The controller 1 performs driving assistance for the vehicle V based on the detection results of various detection units provided in the vehicle V. The vehicle V is provided with surrounding detection units 8a to 8b, which are external sensors that detect the outside of the vehicle V (surrounding conditions), and interior detection units 9a to 9b, which are interior sensors that detect the conditions inside the vehicle V (the driver's state). The controller 1 is able to grasp the surrounding conditions of the vehicle V based on the detection results of the surrounding detection units 8a to 8b, and perform driving assistance in accordance with the surrounding conditions. Furthermore, the controller 1 is able to determine, based on the detection results of the interior detection units 9a to 9b, whether the driver is performing a predetermined operational obligation imposed on the driver when driving assistance is performed.

[0022] The surroundings detection unit 8a is an imaging device that captures images ahead of the vehicle V (hereinafter, sometimes referred to as the front camera 8a), and is attached, for example, to the inside of the passenger compartment of the windshield at the front of the roof of the vehicle V. The controller 1 can extract the contours of targets and lane markings (such as white lines) on the road by analyzing the images captured by the front camera 8a.

[0023] The surroundings detection unit 8b is a millimeter wave radar (hereinafter, may be referred to as radar 8b), and uses radio waves to detect targets around the vehicle V, and detect (measure) the distance to the target and the direction (azimuth) of the target relative to the vehicle V. In the example shown in FIG. 1, five radars 8b are provided: one in the center of the front of the vehicle V, one at each of the left and right corners of the front, and one at each of the left and right corners of the rear.

[0024] The surrounding detection unit installed in the vehicle V is not limited to the above configuration, and the number of cameras and the number of radars may be changed, or a lidar (Light Detection and Ranging: LIDAR) that detects targets around the vehicle V may be installed.

[0025] The in-vehicle detection unit 9a is an imaging device that captures images of the interior of the vehicle (hereinafter, sometimes referred to as in-vehicle camera 9a), and is attached, for example, to the inside of the vehicle cabin at the front of the roof of the vehicle interior V. In this embodiment, the in-vehicle camera 9a is a driver monitor camera that captures images of the driver (for example, the driver's eyes and face). The controller 1 can determine the driver's line of sight and facial direction by analyzing the image (image of the driver's face) captured by the in-vehicle camera 9a.

[0026] The in-vehicle detection unit 9b is a grip sensor that detects the driver's grip of the steering wheel ST (hereinafter, may be referred to as grip sensor 9b), and is provided, for example, on at least a part of the steering wheel ST. Note that the torque sensor 4c that detects the driver's steering torque may be used as the in-vehicle detection unit.

[0027] Examples of driving assistance for the vehicle V include acceleration / deceleration assistance, lane keeping assistance, lane change assistance, and parking assistance. Acceleration / deceleration assistance is driving assistance (ACC: Adaptive Cruise Control) that controls the acceleration / deceleration of the vehicle V within a predetermined vehicle speed while maintaining a distance from a preceding vehicle by controlling the power unit 2 and the hydraulic device 3. Lane keeping assistance is driving assistance (LKAS: Lane Keeping Assist System) that keeps the vehicle V within the lane by controlling the electric power steering device 4. Lane change assistance is driving assistance (ALC: Auto Lane Changing, ALCA: Active Lane Change Assist) that changes the driving lane of the vehicle V to an adjacent lane by controlling the electric power steering device 4. Parking assistance (APA: Advanced parking assist) is driving assistance that parks the vehicle V at a predetermined position by controlling the power unit 2, the hydraulic device 3, and the electric power steering device 4. The parking assistance also includes presenting instructions to the driver in manual driving mode regarding operation details based on the detection results of the surroundings detection units 8a-8b (for example, displaying an image behind the vehicle or notifying the driver of the parking position accuracy by a notification sound, etc.) The driving assistance performed by the controller 1 may also include a collision mitigation brake, an ABS function, traction control, and / or attitude control of the vehicle V that assists in avoiding a collision with an object on the road (for example, a pedestrian, another vehicle, or an obstacle) by controlling the hydraulic device 3.

[0028] Driving assistance for the vehicle V (acceleration / deceleration assistance, lane keeping assistance, lane change assistance, and parking assistance) is performed in a plurality of driving modes including a manual driving mode, a normal assistance mode, and an enhanced assistance mode. FIG. 2 shows driving assistance performed in each of the manual driving mode, the normal assistance mode, and the enhanced assistance mode of this embodiment. In the manual driving mode, acceleration / deceleration assistance, lane keeping assistance, and lane change assistance are not performed, and the driver manually drives the vehicle V. In the manual driving mode, when parking assistance is performed, the driver manually drives the vehicle V along with the parking assistance.

[0029] In manual driving mode, if the driver inputs an instruction to set acceleration / deceleration assist (ACC) via the input device 6 (for example, the switch group 6a), acceleration / deceleration assist is started and the system transitions from manual driving mode to normal assistance mode. In normal assistance mode, lane keeping assist (LKAS) can be executed in addition to acceleration / deceleration assist. Lane keeping assist is started when the driver inputs an instruction to set lane keeping assist via the input device 6 (for example, the switch group 6a) while acceleration / deceleration assist is set. Acceleration / deceleration assist and lane keeping assist are terminated when the driver inputs an instruction to cancel the setting via the input device 6 (for example, the switch group 6a).

[0030] In the normal assistance mode, the driver is required to perform certain actions, such as monitoring the surroundings and gripping the steering wheel. If it is determined based on the detection result of the in-vehicle detection unit 9b that the driver is not performing the certain actions, a notification is sent via the information output device 5 to urge the driver to perform the certain actions.

[0031] When driving on a specific road begins during normal assistance mode, high-precision map information is acquired by the communication device 7c. If matching between the high-precision map information and the image captured by the front camera 8a is successful, the mode automatically transitions from normal assistance mode to extended assistance mode. A specific road is a road for which high-precision map information is provided, such as a highway or a motorway. The high-precision map information includes not only standard information such as the route and location of the specific road, but also information regarding the detailed shape of the specific road, such as the presence or absence of curves and their curvature, the number of lanes, and gradients. When transitioning from normal assistance mode to extended assistance mode, the information output device 5 issues a notification indicating the transition to extended assistance mode, for example, by changing the color of the light emitted by the display device 5a provided on the steering wheel ST.

[0032] In the extended assistance mode, acceleration / deceleration assistance (and lane keeping assistance) is performed in cooperation with highly accurate map information. For example, the controller 1 can perform more advanced acceleration / deceleration assistance than in the normal assistance mode, such as slowing down the vehicle V before a curve or before a point where the lane width decreases, or adjusting the speed of the vehicle V according to the curvature of the curve, based on the highly accurate map information. Note that in the extended assistance mode, as in the normal assistance mode, the driver is required to perform certain actions, such as monitoring the surroundings and holding the steering wheel. If it is determined based on the detection result of the in-vehicle detection unit 9b that the driver is not performing the certain action, a notification is sent via the information output device 5 to urge the driver to perform the certain action.

[0033] Furthermore, in the extended assistance mode, lane change assistance can also be performed. The lane change assistance according to this embodiment includes system-driven lane change assistance (ALC: Auto Lane Changing), which automatically changes lanes based on the judgment of the controller 1, and driver-driven lane change assistance (ALCA: Active Lane Change Assist), which automatically changes lanes in response to an instruction input by the driver. In both system-driven lane change assistance (ALC) and driver-driven lane change assistance (ALCA), the driver is required to perform certain actions, such as monitoring the surroundings and holding the steering wheel, when lane change assistance is performed.

[0034] System-driven lane change assistance (ALC) is initiated when the driver inputs an instruction to set ALC in the enhanced assistance mode via the input device 6 (e.g., the switch group 6a). While ALC is set, the controller 1 sequentially determines whether a lane change is necessary to arrive at a destination previously set by the driver, based on highly accurate map information (information such as lane increase / decrease or lane branching), and automatically performs a lane change when it is determined that a lane change is necessary. While ALC is set, one or more lane changes can be performed depending on the determination of the controller 1. ALC ends when the destination is reached or when the specific road ends. Alternatively, ALC may end when the driver inputs an instruction to cancel the setting via the input device 6 (e.g., the switch group 6a).

[0035] Driver-controlled lane change assistance (ALCA) performs a single lane change in response to a driver's instruction input, and is executed when the driver inputs an instruction to execute ALCA via the input device 6 (for example, the turn signal lever 6b) in the enhanced assistance mode. In ALCA, the driver can input an instruction for the direction in which a lane change is required via the input device 6 (the turn signal lever 6b), and the controller 1 automatically changes the lane to an adjacent lane in the direction instructed by the driver. ALCA can also be executed when system-controlled lane change assistance (ALC) is set.

[0036] Furthermore, parking assistance can be further performed in the normal assistance mode and the extended assistance mode. The parking assistance according to this embodiment includes parking assistance (APA) in the normal assistance mode and the extended assistance mode, which automatically executes parking based on the judgment of the controller 1, and parking assistance in the manual driving mode (manual parking assistance), which presents instructions to the driver when parking. When performing parking assistance, the driver is required to perform certain actions, such as monitoring the surroundings and holding the steering wheel.

[0037] The parking assistance (APA) in the normal assistance mode and the enhanced assistance mode is initiated when the driver inputs an instruction to set up the APA via the input device 6 (e.g., the switch group 6a). Alternatively, for example, the APA may be initiated automatically without an instruction input when the vehicle reaches its destination. During APA execution, the controller 1 calculates a target parking position and a driving control amount for the target parking position based on, for example, the detection results of the radar 8b, and automatically parks the vehicle at the target parking position. Furthermore, the manual parking assistance executes, for example, audio or image display to assist in the execution of such driving control amounts. In this parking assistance, the target parking position is highlighted along with an image of the rear of the vehicle, or the parking operation to the target parking position is instructed by visual display or audio. Note that the manual parking assistance may limit the control of the electric power steering device 4 and the like so as not to exceed such driving control amounts.

[0038] The information processing device 100 according to this embodiment will be described below with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a determination unit 111, an estimation unit 112, an evaluation unit 113, and an execution unit 114. Each of these functional units will be described below. The information processing device 100 may be implemented as a control device CNT, or may be implemented as a server or external device capable of communicating with the control device CNT, and there is no limitation on the form as long as it is capable of similarly executing driving assistance.

[0039] The determination unit 111 determines whether or not a passenger other than the driver is riding in the vehicle V. For example, the determination unit 111 can determine that a passenger is riding in the vehicle V when a seat belt sensor provided in the vehicle V detects that a seat belt of a seat other than the driver's seat is fastened. Furthermore, for example, the determination unit 111 may determine whether or not a passenger is riding in the vehicle V by detecting a face in each seat based on an image captured by a camera that captures an image of the interior of the vehicle. Alternatively, the presence of a passenger can be detected by any method, such as determining the presence or absence of a passenger by a weight sensor provided in each seat.

[0040] The estimation unit 112 can estimate the discomfort index of the driver. The discomfort index of the driver is a value that evaluates the degree of discomfort felt by the driver among his / her emotions. In this embodiment, the discomfort index is expressed as a value of 1 to 5, but this value is not particularly limited to this. An example of the discomfort index related to this embodiment will be described below.

[0041] The estimation unit 112 can estimate the discomfort index (hereinafter, sometimes referred to as the image discomfort index) based on, for example, a captured image of the driver. Alternatively, for example, the designation unit 112 may estimate the discomfort index (hereinafter, sometimes referred to as the voice discomfort index) based on a voice (hereinafter, simply referred to as the user voice) collected by a sound collection device arranged to collect the driver's voice. Hereinafter, several examples of the image discomfort index and the voice discomfort index will be described. However, the estimation unit 112 may select one of these multiple discomfort indexes as the overall discomfort index, and may estimate the discomfort index based on these multiple discomfort indexes (for example, by adding all of them). Alternatively, the estimation unit 112 may calculate the amount of discomfort index added per time for each item for which the discomfort index is to be estimated, and perform an addition process for each of the calculated amounts to estimate the overall discomfort index.

[0042] For example, the estimation unit 112 may estimate an uncomfortable feeling index corresponding to a facial expression based on a facial image of the driver captured by the in-vehicle camera 9 a. In this case, the estimation unit 112 may output an uncomfortable feeling index corresponding to the facial expression, or may add a fluctuation amount corresponding to the facial expression to an initial value of the uncomfortable feeling index (for example, 0 or 2).

[0043] The estimation unit 112 can estimate the discomfort index of the driver from a facial image based on a machine learning model that has been trained in advance to input a human facial image and output a corresponding discomfort index (or its variation). In this embodiment, a machine learning model can be used that recognizes facial expressions that are preset as being uncomfortable or nervous, such as sweating, pale complexion, furrowed brows, or wide-open eyes, and sets the discomfort index. For example, the estimation unit 112 may increase the discomfort index by 1 every 30 seconds while a nervous facial expression is recognized, and decrease the discomfort index by 1 every 30 seconds while no facial expression that increases the discomfort index is recognized.

[0044] The estimation unit 112 may also estimate the discomfort index based on the driver's posture recognized from an image of the driver. Here, if a specific posture, such as a leaning forward posture or stiff shoulders, is detected, the discomfort index corresponding to the posture may be set for the user (or the discomfort index set for the driver's vehicle may be increased by 1). The estimation unit 112 can estimate the driver's discomfort index from an image based on a machine learning model that has been trained in advance to input an image of a person (particularly an image of the upper body) and output a corresponding discomfort index. This configuration makes it possible to perform estimation processing such that the discomfort index is high when the driver's posture is tense and low when the driver's posture is relaxed.

[0045] Furthermore, the estimation unit 112 may estimate the discomfort index based on the user's voice, as described above. The sound collection device used here is not particularly limited as long as it is capable of collecting the driver's voice. For example, the sound collection device may be provided in the in-vehicle camera 9a, in-vehicle equipment such as the steering wheel ST, a dedicated microphone worn by the driver, or a mobile terminal such as a smartphone connected to the vehicle V.

[0046] The estimation unit 112 may estimate the discomfort index based on, for example, the tone of the user's voice. When the tone of the user's voice reaches a specific tone, the estimation unit 112 may set an discomfort index corresponding to the tone for the driver (or increase the discomfort index set for the driver by 1). The "specific tone" here can be set arbitrarily depending on the expected user demographic, etc., and may be, for example, a tone above or below a specific tone that is expected to be emitted by an abnormal person. Furthermore, for example, the estimation unit 112 may estimate the discomfort index of the driver based on the tone of the user's voice over a predetermined period. For example, the estimation unit 112 may set a range of tones considered to be normal (e.g., a predetermined range from the median) based on the tones of the user's voice collected over a certain period, and may increase the discomfort index by 1 when a user's voice with a tone outside that range is recognized. Furthermore, the estimation unit 112 may decrease the discomfort index set for the driver by 1 when a period in which a user's voice with the above-mentioned specific tone is not recognized continues for a predetermined period.

[0047] The estimation unit 112 may also estimate the discomfort index based on, for example, the speaking speed of the user's voice. Here, the speaking speed is evaluated based on the number of Japanese characters recognized from the voice per minute. However, the present invention is not limited to this example as long as the speed can be evaluated in a similar manner. If the speaking speed is equal to or higher than a predetermined threshold, the estimation unit 112 may set an discomfort index corresponding to the speed for the driver (or increase the discomfort index set for the driver by 1). The threshold for the speaking speed can be set arbitrarily depending on the expected user demographic, etc. For example, the estimation unit 112 may also estimate the driver's discomfort index based on the speaking speed of the user's voice over a predetermined period. For example, the estimation unit 112 may set a speaking speed threshold (e.g., as the maximum speaking speed within the predetermined period) based on the speaking speed of the user's voice collected over a certain period, and increase the discomfort index by 1 when a user's voice is recognized at a speed equal to or higher than the threshold. Furthermore, the estimation unit 112 may be configured to decrease the discomfort index set for the driver by 1 if a period in which a user voice at a speed equal to or faster than the threshold as described above is not recognized continues for a predetermined period of time.

[0048] The estimation unit 112 may also estimate the discomfort index based on the content of an utterance included in the user voice. For example, when the estimation unit 112 determines that the driver is swearing, for example, when a sigh is recognized in the user voice, the estimation unit 112 may set an discomfort index corresponding to the swearing for the driver (or increase the discomfort index set for the driver by 1). Here, the recognized content that is determined to be swearing is not particularly limited, and the determination may be made using a facial image as well. Furthermore, for example, when a specific utterance is included in the user voice, the estimation unit 112 may set an discomfort index corresponding to the utterance for the driver (or increase the discomfort index set for the driver by 1). Here, the "specific utterance" can be set arbitrarily depending on the expected user demographic, and may be, for example, a utterance that a nervous driver might utter, such as "This is bad," "This is terrible," or "Wow." Furthermore, for example, the estimation unit 112 may set the discomfort index of the driver based on the volume of the user voice. For example, the estimation unit 112 may set a volume threshold (for example, as a median) based on the volume of voices collected over a certain period of time, and when a user voice with a volume exceeding the threshold is recognized, increase the discomfort index by 1. Furthermore, the estimation unit 112 may decrease the discomfort index set for the driver by 1 when a period in which the above-mentioned specific utterance or a user voice with a volume exceeding the set volume threshold is not recognized continues for a predetermined period of time.

[0049] Furthermore, the estimation unit 112 can estimate the discomfort index of the driver from the user's voice based on a machine learning model that has been trained in advance to input human voice and output a corresponding discomfort index. In this embodiment, a machine learning model can be used that recognizes user voice including the generation of a specific tone, a specific statement, or a voice waveform of a statement made when the user feels uncomfortable, and sets the discomfort index. With this configuration, it is possible to set the discomfort index based on the user's voice while driving.

[0050] The discomfort index according to this embodiment may be estimated based on the driver's biological information. Here, the biological information may be, for example, the driver's pulse, heart rate, blood pressure, sweat rate, respiratory rate, brain waves, body temperature, or brain waves, or information that can be derived from any of these. However, the biological information is not limited to these, as long as it is information that is generally acquired as biological information. The driver's biological information may be measured by a sensor provided in the vehicle V, by an external device (not shown) mounted in the vehicle V, or by a device worn or held by the driver. For example, the biological information may be the sweat rate, heart rate, or body temperature measured by a sensor provided on the steering wheel ST, or the pulse or blood pressure measured by a device worn on the user's wrist.

[0051] The evaluation unit 113 evaluates the discomfort level of the driver. Here, the discomfort level is an evaluation value that evaluates the discomfort felt by the driver. The evaluation unit 113 can evaluate the discomfort level based on, for example, the discomfort index estimated by the estimation unit 112 and the elapsed time.

[0052] The evaluation unit 113 according to this embodiment can evaluate the discomfort level as, for example, one of two levels: 1, which indicates that the driver is uncomfortable, and 0, which indicates that the driver is not uncomfortable. In this case, the evaluation unit 113 can set the initial value of the discomfort level to 0, and set the discomfort level to 1 when a predetermined threshold time (for example, 1 minute) has passed in a state where the discomfort index is equal to or greater than a predetermined threshold value (for example, 3) (hereinafter, this will be referred to as the evaluation threshold). Furthermore, when the discomfort level is 1, the evaluation unit 113 can set the discomfort level to 0 when a predetermined threshold time (for example, 1 minute) has passed in a state where the discomfort index is less than the evaluation threshold.

[0053] Furthermore, if a passenger is required to drive more safely than usual, if the passenger is someone the driver does not want to embarrass, or if the passenger is not very close to the driver, it is considered that the passenger's driving skills are more likely to deteriorate than usual. From this perspective, if the passenger satisfies such predetermined conditions (safety conditions), the threshold time used for evaluating the discomfort level as described above may be reduced, or the evaluation threshold may be reduced. The safety conditions used here may be, for example, if the passenger is under a certain age, if the passenger is of the opposite sex to the driver, or if the passenger is riding in the vehicle V for the first time, but other conditions may also be set as desired. This processing makes it easier to perform driving assistance at a more appropriate time depending on the passenger's attributes.

[0054] Furthermore, for example, the evaluation unit 113 may evaluate the discomfort level by multiplying the discomfort index by the elapsed time. For example, if the discomfort index is updated every 30 seconds, the evaluation unit 113 may calculate an evaluation value of the discomfort index value x 0.5 (minutes) for each such update unit, and evaluate the discomfort level as the sum of such evaluation values ​​for the most recent predetermined period (e.g., 3 minutes). In this case, for example, if the most recent 3 minutes consist of 1 minute of a discomfort index state of 0, 0.5 minutes of a state of 2, 0.5 minutes of a state of 4, and 1 minute of a state of 3, in that order, the discomfort level is calculated to be 6.

[0055] The execution unit 114 executes driving assistance for the vehicle V when a passenger is in the vehicle V and the discomfort level is equal to or greater than a threshold value (execution threshold value). The driving assistance can be that described with reference to FIG. 2 or the like, but any other processing used as a driving assistance function of the vehicle may also be executed. For example, when the discomfort level is evaluated as 1 or 0, the execution unit 114 may execute driving assistance for the vehicle V if the discomfort level is 1 (the execution threshold value is set to 1).

[0056] Furthermore, for example, if the discomfort level is evaluated by multiplying the discomfort index by the elapsed time, the execution unit 114 can set the execution threshold to, for example, 5. In this case, the execution threshold may be set in advance, or may be set based on the discomfort level over a predetermined period (e.g., one week), such as by adding 1 to the maximum discomfort level evaluated during driving over the most recent predetermined period. Furthermore, if the passenger satisfies the above-mentioned safety conditions, the execution threshold may be reduced.

[0057] Hereinafter, the driving assistance execution process performed by the information processing device 100 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of information processing by the information processing device 100. The process shown in Fig. 4 is started when an operation start instruction is issued on an application by the processing unit 201 of the information processing device 100.

[0058] In S401, the determination unit 111 determines whether or not a fellow passenger is riding in the vehicle V. If a fellow passenger is riding in the vehicle V, the process proceeds to S402, and if not, the process ends.

[0059] In S402, the estimation unit 112 estimates the discomfort index of the driver. Here, the discomfort index is estimated based on the face image and voice of the driver. In S403, the evaluation unit 113 evaluates the discomfort level of the driver. Here, the discomfort level is evaluated based on the discomfort index estimated in S402 and the elapsed time.

[0060] In S404, the execution unit 114 determines whether the discomfort level evaluated in S403 is equal to or greater than the execution threshold. If the discomfort level is equal to or greater than the threshold, the process proceeds to S405; if not, the process ends. In S405, the execution unit 114 executes driving assistance, and the process in FIG. 4 ends.

[0061] According to this process, the discomfort level of the vehicle driver is evaluated, and if a passenger is in the vehicle and the discomfort level is equal to or greater than the execution threshold, driving assistance for the vehicle can be performed. Therefore, driving assistance can be performed while taking into consideration the decline in driving skill due to the presence of a passenger. Furthermore, it is possible to blame the vehicle for the embarrassment caused by performing driving assistance while driving.

[0062] Note that the values ​​exemplified in this embodiment are merely examples, and any value may be used as long as it is possible to execute each process in the same way. For example, in this embodiment, the discomfort level is expressed as a five-level value from 1 to 5, but it may also be expressed as a 500-level value from 0.01 to 5.00 using two decimal places.

[0063] [Summary of the embodiment] The above-described embodiments disclose at least the following information processing device, information processing method, and program.

[0064] 1. The information processing device of the above embodiment is a determination means for determining whether or not a passenger other than the driver is riding in the vehicle; evaluation means for evaluating the degree of discomfort of the driver; an execution means for executing driving assistance for the vehicle when the passenger is in the vehicle and the discomfort level is equal to or greater than a first threshold; Equipped with. According to this embodiment, driving assistance is performed taking into account the decline in driving skills due to the presence of a passenger, and it is possible to blame the vehicle for the embarrassment caused by performing driving assistance while driving.

[0065] 2. In the information processing device of the above embodiment, The first threshold is decreased when the passenger meets a predetermined condition. According to this embodiment, it is possible to perform driving assistance at a more appropriate timing depending on whether the passenger satisfies the conditions.

[0066] 3. In the information processing device of the above embodiment, The predetermined conditions include at least one of the following: the passenger is under a certain age; the passenger is of the opposite sex to the driver; or the passenger is riding in the vehicle for the first time. According to this embodiment, it is possible to perform driving assistance at more appropriate timing depending on the attributes of the passengers.

[0067] 4. In the information processing device of the above embodiment, Further, an estimation means for estimating an uncomfortable feeling index of the driver is provided, The evaluation means evaluates the discomfort level based on the discomfort index and the elapsed time. According to this embodiment, it is possible to evaluate the discomfort level of the driver based on the discomfort index estimated for the driver.

[0068] 5. In the information processing device of the above embodiment, The discomfort level becomes equal to or greater than the first threshold when the discomfort index remains equal to or greater than the second threshold for a threshold time or longer. According to this embodiment, safe driving is made possible by providing driving assistance in response to the continuation of a state that the driver finds uncomfortable.

[0069] 6. In the information processing device of the above embodiment, The threshold time is decreased if the passenger meets a predetermined condition. According to this embodiment, it is possible to perform driving assistance at a more appropriate timing depending on whether the passenger satisfies the conditions.

[0070] 7. In the information processing device of the above embodiment, The predetermined conditions include at least one of the following: the passenger is under a certain age; the passenger is of the opposite sex to the driver; or the passenger is riding in the vehicle for the first time. According to this embodiment, it is possible to perform driving assistance at more appropriate timing depending on the attributes of the passengers.

[0071] 8. In the information processing device of the above embodiment, The estimation means estimates the uncomfortable emotion index based on at least one of the driver's facial expression, tone of voice, speaking speed, and spoken content. According to this embodiment, it is possible to estimate the discomfort index with high accuracy based on information obtained from the driver inside the vehicle.

[0072] 9. In the information processing device of the above embodiment, The estimation means estimates the uncomfortable feeling index based on information acquired via an imaging device or a sound collection device. According to this embodiment, it is possible to estimate the uncomfortable feeling index with higher accuracy.

[0073] 10. The information processing method of the above embodiment is a determination step of determining whether or not a passenger other than the driver is riding in the vehicle; an evaluation step of evaluating the discomfort level of the driver; an execution step of executing driving assistance for the vehicle when the passenger is in the vehicle and the discomfort level is equal to or greater than a first threshold; Equipped with. According to this embodiment, driving assistance is performed taking into account the decline in driving skills due to the presence of a passenger, and it is possible to blame the vehicle for the embarrassment caused by performing driving assistance while driving.

[0074] 11. The program of the above embodiment causes a computer to function as each of the means of the information processing device described in any one of items 1 to 9. According to this embodiment, driving assistance is performed taking into account the decline in driving skills due to the presence of a passenger, and it is possible to blame the vehicle for the embarrassment caused by performing driving assistance while driving.

[0075] Although the embodiments of the invention have been described above, the invention is not limited to the above-described embodiments, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0076] 111: Determination unit, 112: Estimation unit, 113: Evaluation unit, 114: Execution unit

Claims

1. a determination means for determining whether or not a passenger other than the driver is riding in the vehicle; evaluation means for evaluating the degree of discomfort of the driver; an execution means for executing driving assistance for the vehicle when the passenger is in the vehicle and the discomfort level is equal to or greater than a first threshold; An information processing device comprising:

2. The information processing device according to claim 1 , wherein the first threshold value is decreased when the passenger satisfies a predetermined condition.

3. 3. The information processing device according to claim 2, wherein the predetermined conditions include at least one of the following: the passenger is under a certain age; the passenger is of the opposite sex to the driver; or the passenger is riding in the vehicle for the first time.

4. Further, an estimation means for estimating an uncomfortable feeling index of the driver is provided, 2. The information processing apparatus according to claim 1, wherein the evaluation means evaluates the discomfort level based on the discomfort index and an elapsed time.

5. The information processing device according to claim 4 , wherein the discomfort level becomes equal to or greater than the first threshold value when the discomfort index remains equal to or greater than a second threshold value for a threshold time period or longer.

6. The information processing device according to claim 5 , wherein the threshold time is decreased when the passenger satisfies a predetermined condition.

7. 7. The information processing device according to claim 6, wherein the predetermined conditions include at least one of the following: the passenger is under a certain age; the passenger is of the opposite sex to the driver; or the passenger is riding in the vehicle for the first time.

8. 5. The information processing device according to claim 4, wherein the estimation means estimates the uncomfortable emotion index based on at least one of the facial expression, tone of voice, speaking speed, and contents of speech of the driver.

9. The information processing device according to claim 8 , wherein the estimation means estimates the uncomfortable feeling index based on information acquired via an imaging device or a sound collecting device.

10. a determination step of determining whether or not a passenger other than the driver is riding in the vehicle; an evaluation step of evaluating the discomfort level of the driver; an execution step of executing driving assistance for the vehicle when the passenger is in the vehicle and the discomfort level is equal to or greater than a first threshold; An information processing method comprising:

11. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 9.

Images