Information providing support method, information providing support device, information providing support program, and information providing system

Abstract

To provide an information provision support method, information provision support device, information provision support program and information provision support system which can induce a prescribed operation that should be performed by a driver in order to obtain information provided from a vehicle or on the basis of the information provided from the vehicle.SOLUTION: An information provision support method provides information to a driver 10 by using an information provision support device 101 that supports information provision from a vehicle 100 to the driver 10. In the information provision support method, information to be provided to the driver 10 is acquired from the vehicle 100, the operation of the driver 10 is recognized, and a response operation that has been executed by the information provision support device 101 according to a prescribed operation when the driver 10 performs the prescribed operation in order to obtain the information or on the basis of the information is executed by the information provision support device 101 before the driver 10 performs the prescribed operation after the information is acquired.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information providing support method, an information providing support device, an information providing support program, and an information providing system for supporting the provision of information from a vehicle to a driver.

Background Art

[0002] Patent Document 1 describes a toy that recognizes the actions of a player and performs actions that reflect the actions of the player.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, vehicles support drivers by providing various information to the drivers according to the situations inside and outside the vehicles, so that a comfortable experience can be obtained. The content of the information provided by the vehicle to the driver is diverse, and among them, for example, for ensuring safety or the like, in order to obtain the information, or based on the information, there may be cases where it is desirable for the driver to perform a predetermined action. In this case, it is desirable for the vehicle to induce the driver's predetermined action by providing information in a manner that the driver can intuitively or very naturally recognize the predetermined action to be performed.

[0005] On the other hand, even if a vehicle or a device mounted on the vehicle can recognize the actions of the driver and perform flexible actions according to the actions of the driver, like the toy described in Patent Document 1, just performing passive actions cannot induce the predetermined actions that the driver should perform.

[0006] An object of the present invention is to provide an information provision support method, an information provision support device, an information provision support program, and an information provision system that can acquire information provided from a vehicle or induce a predetermined action to be performed by a driver based on the information provided from the vehicle.

Means for Solving the Problems

[0007] An aspect of the present invention is an information provision support method for providing information to a driver using an information provision support device that supports information provision from a vehicle to the driver. In this information provision support method, information to be provided to the driver is acquired from the vehicle. Also, the driver's actions are recognized. And, in order to acquire information or when a predetermined action to be performed by the driver based on the information is performed, a response action that may have been executed by the information provision support device according to the predetermined action is executed by the information provision support device after the information is acquired and before the driver performs the predetermined action.

Effects of the Invention

[0008] According to the present invention, it is possible to provide an information provision support method, an information provision support device, an information provision support program, and an information provision system that can induce a predetermined action to be performed by a driver.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0011] [First Embodiment] FIG. 1 is a block diagram showing a schematic configuration of a vehicle 100 and an information-providing support device 101. The vehicle 100 can support driving operations that the driver 10 should originally perform, such as automatically following other vehicles traveling ahead. Further, the vehicle 100 supports driving by providing information to the driver 10 using the information-providing support device 101.

[0012] As shown in FIG. 1, the vehicle 100 includes a driving operation unit 11, a car navigation system 12, an agent 13, an agent controller 14, a vehicle controller 15, a sensor 16, and the like.

[0013] The driving operation unit 11 is an operation unit that the driver 10 operates for driving the vehicle 100. The driving operation unit 11 is, for example, a steering wheel 42 (see FIG. 5 etc.), an accelerator pedal, and a brake pedal.

[0014] The car navigation system 12 is a guiding device that guides the driver 10 on the route to the destination and the like. The car navigation system 12 is provided with a unique display and operation keys. The driver 10 uses the operation keys of the car navigation system 12 to perform operations necessary for setting the destination, route guidance, and the like. Also, the driver 10 can receive the provision of information including route guidance and the like by the sound or voice output from the display of the car navigation system 12 and / or the speaker installed in the car navigation system 12 or the vehicle 100.

[0015] The agent 13 is a user interface that supports the provision of information from the vehicle 100 to the driver 10. That is, the agent 13 is an information-providing assistant that provides the driver 10 with the information that the vehicle 100 should provide to the driver 10, together with the vehicle 100 or on behalf of the vehicle 100.

[0016] The agent 13 is represented physically or virtually in the form of a mascot (also referred to as a mascot character or simply a character), such as a person, an animal, a plant, an object, or other symbols. Therefore, the agent 13 is anthropomorphic and can express emotions and the emotional changes, which are the changes in emotions, through its "facial expressions" and the like. Also, the agent 13 can express emotions and emotional changes through colors, sounds or voices, or actions such as gestures. That is, the agent 13 expresses its pseudo-emotions and emotional changes through facial expressions, colors, sounds or voices, or actions.

[0017] Agent 13 performs, for example, a response operation or an action-inducing operation by means of an expression or the like that expresses the above-mentioned emotions and feelings. The response operation is an operation performed in response to the operation of the driver 10. "Action induction" in the present embodiment means inducing an operation (action) that the driver 10 should perform by teaching an operation (action) that the driver 10 should perform in an indirect expression form or an expression form that appeals to intuition. For example, Agent 13 suggests, implies, or tacitly indicates an operation that the driver 10 should perform by means of an action-inducing operation. That is, the action-inducing operation executed by Agent 13 is an operation performed to induce the operation of the driver 10, and is executed before the operation that the driver 10 should perform. In particular, in the present embodiment, Agent 13 induces the operation of the driver 10 by executing, before the operation that the driver 10 should perform, a "response operation" that has been executed in response to the operation of the driver 10. That is, the response operation and the action-inducing operation are operations of the same form, but the timing of execution is different. The response operation is executed after the associated operation of the driver 10, and the action-inducing operation is executed before the associated operation of the driver 10.

[0018] In the present embodiment, Agent 13 is constituted by a robot having a physical form and is disposed within the field of view of the driver in the driving state, for example, near the driver's seat. However, when the vehicle 100 employs a head-up display, Agent 13 can be virtually constituted by the display of the head-up display. Further, when a device possessed by the driver 10, such as a smartphone, is linked with the vehicle 100, Agent 13 can be virtually displayed or the like by means of a device such as a smartphone.

[0019] The agent controller 14 is a control unit that controls the operation of the agent 13. More specifically, the agent controller 14 is a computer programmed with the expression, color, sound or voice, or movement of the agent 13 according to the information provided by the vehicle 100 to the driver 10, the driver 10's actions, etc. The agent 13 and the agent controller 14 constitute an information provision support device 101 that supports the provision of information from the vehicle 100 to the driver 10. Further, the information provision support device 101 constitutes an information provision system 102 together with the whole vehicle 100 or the components related to information provision in the vehicle 100. The specific configuration of the agent controller 14 will be described in detail later.

[0020] Note that in the present embodiment, the agent controller 14 is configured separately from the vehicle controller 15, but some or all of the parts constituting the agent controller 14 may be configured integrally with the vehicle controller 15. Also, in the present embodiment, since the agent 13 is formed as one of the devices mounted on the vehicle 100, the agent controller 14 for controlling this is also provided in the vehicle 100. However, when the agent 13 is a cooperation device independent of the vehicle 100, such as a smartphone, that device can constitute the agent controller 14.

[0021] The vehicle controller 15 is a control unit that comprehensively controls the operation of the vehicle 100. Based on the input from the driving operation unit 11, the vehicle controller 15 controls an electric motor and / or an engine that drives the vehicle 100, as well as brakes and the like. The vehicle controller 15 is constituted by, for example, one or a plurality of computers in which the operations of each part constituting the vehicle 100 are programmed. The vehicle controller 15 may refer to signals output by the sensor 16 or the like for the control of the vehicle 100. The vehicle controller 15 acquires operations on the vehicle 100 and operations on various devices mounted on the vehicle 100 (hereinafter referred to as device operation information 33. See FIG. 2). In addition, the vehicle controller 15 obtains information to be provided to the driver 10 from the vehicle 100 through calculations using signals or the like acquired from the sensor 16.

[0022] The information provided by the vehicle 100 to the driver 10 includes various types such as information for alerting attention to objects that should be noted, such as traffic signs and pedestrians, information for route guidance, and information for requesting operations of the vehicle 100 or devices mounted on the vehicle 100. In any case, the information provided by the vehicle 100 to the driver 10 is provided according to the situation inside and / or outside the vehicle. Therefore, the information provided by the vehicle 100 to the driver 10 is referred to as situation information 32 (see FIG. 2).

[0023] In the present embodiment, the situation information 32 provided by the vehicle 100 to the driver 10 is classified into two types: information that requires the operation of the driver 10 and the rest (that is, information that does not require the operation of the driver 10). Whether various situation information 32 is information that requires the operation of the driver 10 is determined in advance according to its type and nature.

[0024] In order to obtain the information provided by the vehicle 100, or when there is a predetermined operation that the driver 10 should perform based on the information provided by the vehicle 100, such information is the information that requires the driver 10's operation. Also, the operation of the driver 10 that should be performed in order to obtain the information provided by the vehicle 100, or based on the information provided by the vehicle 100, is hereinafter simply referred to as a predetermined operation. The predetermined operations that the driver 10 should perform include, in addition to the driver 10 moving their hands, feet, etc., simply the driver 10 moving their line of sight.

[0025] Examples of the information that requires the driver 10's operation include, for example, alerting attention to a target to be noted outside or inside the vehicle (hereinafter simply referred to as the target to be noted 41; see FIG. 4 etc.), an instruction or request to grip the steering wheel 42, and a suggestion to operate various devices mounted on the vehicle 100 such as the car navigation system 12, etc. And the movement of the line of sight to the target to be noted 41, the gripping of the steering wheel 42, and the operation of various devices, etc. are each predetermined operations that the driver 10 should perform.

[0026] The sensor 16 is one or more sensors that measure the status of each part constituting the vehicle 100 and the status inside or outside the vehicle 100. The sensor 16 is, for example, an acceleration sensor, an in-vehicle camera 17 for photographing inside the vehicle, a camera for photographing outside the vehicle, a microphone for collecting sounds inside or outside the vehicle, a GNN (Global Navigation Satellite System) sensor, and a LiDAR (Light Detection and Ranging) sensor, etc. In the present embodiment, the sensor 16 is assumed to include at least the in-vehicle camera 17.

[0027] The in-vehicle camera 17 continuously or intermittently photographs the inside of the vehicle 100, particularly the area including the driver 10. The image or video photographed by the in-vehicle camera 17 (hereinafter referred to as the in-vehicle image 31; see FIG. 2) is used in the agent controller 14 to recognize the operation of the driver 10.

[0028] FIG. 2 is a block diagram showing the configuration of the agent controller 14. As shown in FIG. 2, it includes an information acquisition unit 21, a driver action recognition unit 22, an action induction determination unit 23, an agent action control unit 24, and an action pattern storage unit 25. The program that causes the agent controller 14 to function as these respective units constitutes an information provision support program for operating the information provision support device 101 and the information provision system 102. The information provision support program is pre-stored in the information provision support device 101 or the information provision system 102. Also, the information provision support program can be provided, for example, in the form of a recording medium that can be read by the information provision support device 101, the information provision system 102, or other computers. Further, after being installed in the information provision support device 101 or the information provision system 102 using the recording medium or by means of so-called OTA (Over The Air) technology, part or all of the information provision support program can be updated.

[0029] The information acquisition unit 21 acquires an in-vehicle image 31, situation information 32, and device operation information 33. The information acquisition unit 21 acquires the in-vehicle image 31 directly from the in-vehicle camera 17 or indirectly via the vehicle controller 15. Also, the information acquisition unit 21 acquires the situation information 32 and the device operation information 33 from the vehicle controller 15. Note that the information acquisition unit 21 can acquire a signal or the like output from the sensor 16 and generate the situation information 32 by itself instead of the vehicle controller 15.

[0030] The driver action recognition unit 22 recognizes the actions of the driver 10. Also, the driver action recognition unit 22 functions as a driver action monitoring unit that monitors the actions of the driver 10 by continuously or intermittently recognizing the actions of the driver 10. In the present embodiment, the driver action recognition unit 22 recognizes the actions of the driver 10 using the in-vehicle image 31 and / or based on the operation status of the devices mounted on the vehicle 100. Specifically, the driver action recognition unit 22 includes an action detection unit 34, a gaze detection unit 35, and a device operation detection unit 36.

[0031] The motion detection unit 34 detects the overall motions of the driver 10, such as the hands and face of the driver 10. For example, the motion detection unit 34 recognizes the driver 10 from the in-vehicle image 31, and detects the physical motions of the driver 10 based on the temporal changes in the shape and the like.

[0032] The gaze detection unit 35 detects the gaze of the driver 10. For example, the gaze detection unit 35 detects the gaze of the driver 10 and its transition (hereinafter referred to as the gaze motion) by recognizing the eyeballs (pupils) of the driver 10 and their motions from the in-vehicle image 31.

[0033] The device operation detection unit 36 detects an operation on the vehicle 100 or an operation on various devices mounted on the vehicle 100 (hereinafter referred to as the device operation motion) based on the device operation information 33. For example, when the car navigation system 12 is operated, the device operation detection unit 36 detects the device operation motion based on the device operation information 33.

[0034] Hereinafter, unless otherwise specifically mentioned, the physical motions, gaze motions, and device operation motions of the driver 10 are collectively referred to simply as the motions of the driver 10. Among the physical motions, gaze motions, and / or device operation motions of the driver 10, those performed to obtain the situation information 32 or based on the situation information 32 are the predetermined motions as described above.

[0035] The action induction determination unit 23 determines, based on the situation information 32, whether it is necessary to induce a predetermined motion of the driver 10 when providing the situation information 32 to the driver 10. The action induction determination unit 23 determines in advance whether it is necessary to induce a predetermined motion of the driver 10, for example, using a data table or the like, according to the type and nature of the situation information 32. For this reason, the action induction determination unit 23 determines whether it is necessary to induce a predetermined motion of the driver 10 according to the type and nature of the individual and specific situation information 32.

[0036] In addition, when it is necessary to induce a predetermined operation of the driver 10, the action induction determination unit 23 sets parameters for determining the operation mode of the specific agent 13 that induces the predetermined operation of the driver 10. For this purpose, the action induction determination unit 23 includes an importance determination unit 37 and a learning status determination unit 38.

[0037] The importance determination unit 37 determines the importance of the situation information 32. The importance of the situation information 32 is preset based on the urgency of the situation information 32 and / or the risk level when a predetermined operation based on the situation information 32 is not performed. That is, the importance of the situation information 32 is preset according to the type and nature of the situation information 32. In particular, when the situation information 32 is information related to the driving operation of the vehicle 100, its importance is determined to be high. On the other hand, when the situation information 32 is information other than information related to the driving operation of the vehicle 100, its importance is determined to be low. To give a more specific example, when the situation information 32 takes traffic signs, pedestrians, etc. as the attention target 41 and has the content of calling for attention necessary for ensuring safety, the importance of the situation information 32 is determined to be "high". On the other hand, when the situation information 32 specifically takes the equipment installed in the vehicle 100, etc. as the attention target 41 and has the content of calling for attention to prompt its operation, the importance of the situation information 32 is determined to be "low". Also, for example, when the situation information 32 is information that requires gripping the steering wheel 42, the importance of the situation information 32 is determined to be "high". The importance of the situation information 32 determined by the importance determination unit 37 is one of the parameters for determining the operation mode of the agent 13.

[0038] The learning status determination unit 38 determines the learning (cognition) status of the driver 10 with respect to the operation of the agent 13.

[0039] Specifically, the learning status determination unit 38 acquires the actions of the driver 10 from the driver action recognition unit 22. Also, when the driver 10 performs a predetermined action according to the situation information 32, the agent 13 performs a response action according to the situation information 32. Therefore, the learning status determination unit 38 receives a notification from the agent action control unit 24 that the agent 13 has performed a response action. Based on this information, the learning status determination unit 38 determines whether the driver 10 has recognized the response action executed by the agent 13. For example, after the agent 13 executes a corresponding response action according to the predetermined action of the driver 10, when the driver 10 visually recognizes the agent 13, the learning status determination unit 38 determines that the driver 10 has recognized the response action of the agent 13 corresponding to the predetermined action.

[0040] In addition, the learning status determination unit 38 counts the number of times (hereinafter referred to as the response action recognition count) that the driver 10 has recognized the response action of the agent 13 corresponding to the predetermined action. When the response action recognition count is equal to or greater than a predetermined threshold value determined in advance, the learning status determination unit 38 determines that the learning of the driver 10 (hereinafter referred to as user learning) for the response action of the agent 13 corresponding to the predetermined action has been completed. The above determination result as to whether the user learning has been completed is one of the parameters for determining the operation mode of the agent 13.

[0041] In the present embodiment, the learning status determination unit 38 counts the number of times the response action has been recognized as the response action recognition count, but it is not limited to this. The learning status determination unit 38 can count the number of trials of performing the response action instead of the response action recognition count. In this case, the learning status determination unit 38 determines that the user learning has been completed when the number of trials of the response action is equal to or greater than a predetermined threshold value.

[0042] The agent operation control unit 24 controls the operation of the agent 13. Based on the situation information 32 and the like, the agent operation control unit 24 causes the agent 13 to execute a response operation, an action induction operation, and other operations. For this reason, the agent operation control unit 24 includes at least a response operation control unit 39 and an action induction operation control unit 40.

[0043] The response operation control unit 39 controls the response operation of the agent 13. Based on the situation information 32 and the operation of the driver 10 acquired from the driver operation recognition unit 22, when the driver 10 performs a predetermined operation corresponding to the situation information 32, the response operation control unit 39 causes the agent 13 to execute the corresponding response operation. The response operation corresponding to the predetermined operation and other operation modes of the agent 13 are determined in advance according to the type and nature of the situation information 32 and are stored in the operation pattern storage unit 25. For this reason, when the driver 10 performs a predetermined operation, the response operation control unit 39 can cause the agent 13 to execute the response operation corresponding to the predetermined operation in real time by referring to the operation pattern storage unit 25.

[0044] The action induction operation control unit 40 controls the action induction operation of the agent 13 when the action induction determination unit 23 determines that it is necessary to induce a predetermined operation of the driver 10. In addition, the action induction operation control unit 40 causes the agent 13 to execute an action induction operation corresponding to the situation information 32 based on the situation information 32 and the determination result of the learning situation determination unit 38. Specifically, when it is determined that the user learning is completed according to the determination result of the learning situation determination unit 38, the action induction operation control unit 40 refers to the operation pattern storage unit 25 without waiting for the driver 10 to perform a predetermined operation corresponding to the situation information 32, and causes the agent 13 to execute the response operation corresponding to the situation information 32 as an action induction operation. For this reason, substantially before the driver 10 performs a predetermined operation corresponding to the situation information 32, the agent 13 executes an action induction operation.

[0045] When the agent 13 executes an action-inducing operation, the driver 10 who has completed user learning regarding the action-inducing operation can intuitively recognize that they should perform a predetermined operation. Also, when the agent 13 executes an action-inducing operation, the driver 10 who has completed user learning can perform a predetermined operation almost reflexively in response to this.

[0046] In addition, when the action-inducing operation control unit 40 causes the agent 13 to execute an action-inducing operation, the response operation control unit 39 does not cause the agent 13 to execute a response operation. That is, the agent 13 selectively executes either an action-inducing operation or a response operation in one provision of the situation information 32. When the driver 10 has little experience using the vehicle 100, the agent 13 generally executes a response operation, and when the driver 10 has a certain degree of rich experience using the vehicle 100, the agent 13 generally executes an action-inducing operation.

[0047] Also, as described above, after causing the agent 13 to execute an action-inducing operation, the action-inducing operation control unit 40 determines whether or not the driver 10 has performed a predetermined operation. That is, the action-inducing operation control unit 40 functions as an action-inducing success / failure determination unit that determines the success or failure of action induction by the agent 13. The action-inducing operation control unit 40 determines the success or failure of action induction by monitoring at least the actions of the driver 10 after the agent 13 has executed the action-inducing operation. For example, the action-inducing operation control unit 40 measures the elapsed time (hereinafter simply referred to as the elapsed time) after the agent 13 has executed the action-inducing operation. Then, when the driver 10 performs a predetermined operation before the elapsed time has elapsed a predetermined time (threshold value) set in advance, the action-inducing operation control unit 40 determines that the action induction has been successful. On the other hand, when the elapsed time exceeds the predetermined time, the action-inducing operation control unit 40 determines that the action induction has failed.

[0048] After that, the action induction operation control unit 40 causes the agent 13 to execute an additional operation according to the success or failure of the action induction. In the present embodiment, the additional operation after the action induction operation is an operation that expresses the emotion or sentiment of the agent 13 (hereinafter referred to as emotion), or an emphasis on the action induction operation.

[0049] The emotions that the action induction operation control unit 40 additionally causes the agent 13 to execute include, for example, a first emotion and a second emotion. The first emotion is an operation in which the agent 13 expresses a positive emotion or sentiment (hereinafter referred to as a positive sentiment) by the expression or the like of the agent 13. For example, an emotion that is generally perceived as the agent 13 generally expressing joy, such as the agent 13 making an expression or gesture expressing joy, emitting a relatively high sound or voice, or presenting a warm color system, is the first emotion. The second emotion is an operation in which the agent 13 expresses a negative emotion or sentiment (hereinafter referred to as a negative sentiment) by the expression or the like of the agent 13. For example, an emotion that is generally perceived as the agent 13 generally expressing sadness, such as the agent 13 making an expression or gesture expressing sadness, emitting a relatively low sound or voice, or presenting a cool color system, is the second emotion.

[0050] After causing the agent 13 to execute the action induction operation, when the driver 10 performs a predetermined operation, the action induction operation control unit 40 causes the agent 13 to execute the positive first emotion. That is, when the action induction is successful, the agent 13 executes the first emotion. By perceiving the positive first emotion of the agent 13, the driver 10 recognizes that the agent 13 has suggested or instructed the predetermined operation that the driver 10 should perform by the action induction operation of the agent 13, and is more likely to have gratitude or other positive emotions towards the agent 13. Therefore, the action induction in subsequent times is more likely to be successful.

[0051] On the other hand, after causing the agent 13 to perform the action-inducing operation, when the driver 10 does not perform the predetermined action, the action-inducing operation control unit 40 changes the additional action to be performed by the agent 13 according to the importance of the situation information 32. That is, when the action induction fails, the agent 13 performs an additional action according to the importance of the situation information 32.

[0052] Specifically, when the action induction fails and the importance of the situation information 32 is low, the action-inducing operation control unit 40 causes the agent 13 to execute a negative second emotion. By perceiving the negative second emotion of the agent 13, the driver 10 can recognize that there was an action that the driver 10 should perform. As a result, the driver 10 will naturally pay attention to the actions of the agent 13, so that action induction will be more likely to succeed in subsequent times.

[0053] On the other hand, when the action induction fails and the importance of the situation information 32 is high, the action-inducing operation control unit 40 causes the agent 13 to execute an emphasized action-inducing operation. The emphasis of the action-inducing operation means changing and / or adding the expression, color, sound or voice, or action of the agent 13 related to the action-inducing operation to a relatively exaggerated expression form. For example, the action-inducing operation is emphasized by increasing the volume of the sound or voice emitted by the agent 13, or by greatly expressing the changes in the expression and gesture of the agent 13. Since the driver 10 can easily recognize the action-inducing operation of the agent 13, the action-inducing operation is preferably emphasized in a form of adding the types of the five senses (so-called modalities) that are perceived by the driver 10 in the action-inducing operation. For example, the action-inducing operation that was represented only by visual information such as a change in expression is emphasized in a form that appeals to the sense of hearing in addition to vision, for example, by further emitting sound or voice.

[0054] The operation pattern storage unit 25 stores the operation patterns of the agent 13 according to the types and properties of the situation information 32 and the like. For example, the operation pattern storage unit 25 stores the corresponding response operations and action induction operations for each piece of situation information 32. Further, the operation pattern storage unit 25 stores in advance the first emotion, the second emotion, and the emphasized form of the action induction operation for each piece of situation information 32.

[0055] Hereinafter, the operation of the vehicle 100 configured as described above, that is, the operation related to user learning and action induction in the information providing support device 101 and the information providing system 102 will be described.

[0056] <User learning> FIG. 3 is a flowchart of user learning for causing the driver 10 to recognize the response operation of the agent 13. As shown in FIG. 3, in step S101, the driver action recognition unit 22 recognizes the action of the driver 10. In step S102, the response operation control unit 39 refers to the operation pattern storage unit 25 and determines a response operation corresponding to the recognized action of the driver 10. Further, in step S103, the response operation control unit 39 causes the agent 13 to execute the response operation.

[0057] Thereafter, in step S104, the learning situation determination unit 38 determines whether or not the driver 10 has recognized the response operation of the agent 13. When it is determined that the driver 10 has recognized the response operation of the agent 13, in step S105, the learning situation determination unit 38 increments the number of times the response operation is recognized.

[0058] FIG. 4 is an explanatory diagram showing an example of user learning. As shown in FIG. 4, scene a1 at time t1 is a scene in which the target of attention 41 occurs, the driver 10 notices this by himself / herself, and the line of sight is shifted. When the driver 10 thus shifts the line of sight to the target of attention 41, in the stage of user learning, as shown in scene a2 at time t2, the agent 13 follows the line-of-sight shift of the driver 10, turns toward the direction of the generated target of attention 41, and executes a response action of looking at the target of attention 41. Then, when the driver 10 recognizes the response action of the agent 13, "looking at the target of attention 41", as in scene a3 at time t3, the number of times of recognizing the response action is incremented on the assumption that the driver 10 has learned this response action. And the user learning related to this response action is repeated until the number of times of recognizing the response action becomes equal to or more than the threshold value and it is determined that the user learning is completed.

[0059] FIG. 5 is an explanatory diagram showing another example of user learning. As shown in FIG. 5, scene b1 at time t1 is a scene in which the driver 10 has released his / her hand from the steering wheel 42. In scene b1, the agent 13 expresses an expression as if it is sleeping. Then, in scene b2 at time t2, when the driver 10 grips the steering wheel 42, accordingly, the agent 13 executes a response action of, for example, opening its eyes wide and expressing an expression indicating activation or awakening. Thereafter, when the driver 10 recognizes the response action that "when the steering wheel 42 is gripped, the agent 13 wakes up", as in scene b3 at time t3, the number of times of recognizing the response action is incremented on the assumption that the driver 10 has learned this response action. And the user learning related to this response action is repeated until the number of times of recognizing the response action becomes equal to or more than the threshold value and it is determined that the user learning is completed.

[0060] Hereinafter, it is assumed that the user learning is completed for the response action that the agent 13 looks at this when the target of attention 41 occurs and the response action that the agent 13 wakes up when the steering wheel 42 is gripped.

[0061] <Action Inducement> FIG. 6 is a flowchart of action induction. As shown in FIG. 6, in step S201, the information acquisition unit 21 acquires situation information 32 that the vehicle 100 should provide to the driver 10. Further, in step S202, the action induction determination unit 23 determines whether there is a need for action induction to induce a predetermined action of the driver 10 based on the specific type, nature, etc. of the acquired situation information 32.

[0062] When it is determined that action induction is necessary, in step S203, the learning situation determination unit 38 determines whether user learning has been completed for the response action corresponding to the situation information 32. When the user learning has not been completed, the above-described response action is executed. On the other hand, when the user learning has been completed, in step S204, the action induction operation control unit 40 refers to the operation pattern storage unit 25 and determines an action induction operation corresponding to the situation information 32. Then, in step S205, the action induction operation control unit 40 causes the agent 13 to execute the determined action induction operation.

[0063] Thereafter, in step S206, the action induction operation control unit 40 determines whether the predetermined action of the driver 10 has been induced by the action induction operation of the agent 13. When it is determined that the predetermined action of the driver 10 has been induced by the action induction operation of the agent 13, in step S107, the action induction operation control unit 40 causes the agent 13 to execute a positive first emotion.

[0064] On the other hand, when it is determined that the predetermined action of the driver 10 cannot be induced by the action induction operation of the agent 13, in step S208, the action induction operation control unit 40 refers to the importance of the situation information 32. When the importance of the situation information 32 is high, in step S209, the action induction operation control unit 40 causes the agent 13 to execute an emphasized action induction operation. On the other hand, when the importance of the situation information 32 is low, in step S210, the action induction operation control unit 40 causes the agent 13 to execute a negative second emotion.

[0065] FIG. 7 is an explanatory diagram showing an action-inducing operation and an example of an operation after successful action induction. As shown in FIG. 7, scene A1 at time T1 is a scene at the moment when a target of attention 41 to which the vehicle 100 should provide information occurs to the driver 10. In scene A1, the driver 10 has not noticed this yet and is thinking about something else. Scene A2 at time T2 is a scene that is substantially at the same time as time T1. In scene A2, before the driver 10 notices the target of attention 41, the agent 13 performs an action-inducing operation of looking at the target of attention 41. Since the agent 13 is within the driver 10's field of vision, the driver 10 inevitably perceives the action-inducing operation of the agent 13. Also, since the user learning related to this action-inducing operation is completed, the driver 10 intuitively and almost reflexively understands that the action-inducing operation of the agent 13 suggests a predetermined operation to be performed by the driver 10. Therefore, in scene A3 at the immediately following time T3, under the influence of the action-inducing operation of the agent 13, as if being attracted by the action-inducing operation of the agent 13, the driver 10 reflexively shifts the line of sight to the target of attention 41.

[0066] Thus, when the predetermined operation of the driver 10, "looking at the target of attention 41", is performed by the action-inducing operation of the agent 13, as in scene A4 shown at time T4, the agent 13 performs a first emotion such as smiling.

[0067] FIG. 8 is an explanatory diagram showing an example of an operation after failed action induction. As shown in FIG. 8, scene A1 at time T1 is, as in the case of FIG. 7, a scene at the moment when the target of attention 41 occurs. And in scene A2' at time T2, it is a scene that is substantially at the same time as time T1, and the agent 13 performs an action-inducing operation of looking at the target of attention 41. However, in scene A2', the driver 10 has not noticed the action-inducing operation of the agent 13 and has not performed the predetermined operation.

[0068] Here, for example, when the importance of the situation information 32 is high, such as when the target of attention 41 needs to be visually recognized for ensuring safety or the like and immediate ad-hoc driving operations must be performed, in the scene A3' at time T3, the agent 13 emphasizes the action-inducing operation. Here, while maintaining the essential aspect of the action-inducing operation of "looking at the target of attention 41", the agent 13 changes the expression by changing a part of the face and the color of the eyes, thereby emphasizing the action-inducing operation. As a result, the driver 10 notices the emphasized action-inducing operation by the agent 13 and intuitively and almost reflexively understands the predetermined operation suggested by the action-inducing operation. Therefore, under the influence of the action-inducing operation of the agent 13 and being lured by the action-inducing operation of the agent 13, the driver 10 reflexively shifts the line of sight to the target of attention 41.

[0069] FIG. 9 is an explanatory diagram showing another example of an operation after a failure to induce an action. As shown in FIG. 9, the scene A1 at time T1 is the scene at the moment when the target of attention 41 occurs, similar to the cases of FIGS. 7 and 8. Also, in the scene A2' at time T2, similar to the case of FIG. 8, the agent 13 executes the action-inducing operation of looking at the target of attention 41, but the driver 10 has not noticed the action-inducing operation of the agent 13 and has not performed the predetermined operation.

[0070] Here, for example, assume that the importance of the situation information 32 is low, such as when the target of attention 41 is an in-vehicle device such as the car navigation system 12 and the situation information 32 recommends an operation of the in-vehicle device. In this case, in the scene A3'' at time T3, the agent 13 executes a second emotion such as showing a sad expression, such as showing a crying face. Then, when the driver 10 recognizes the second emotion of the agent 13, the driver 10 can observe that some predetermined operation was required, and thereafter, it becomes easier to notice the target of attention 41. Also, when the driver 10 recognizes the second emotion of the agent 13, a potential sense of guilt or the like towards the anthropomorphic agent 13 is generated, and thereafter, it becomes easier to notice at least the operation of the agent 13. As a result, it becomes easier for the action induction to succeed in subsequent times.

[0071] Figure 10 is an explanatory diagram showing another behavior induction operation and its effect. In Case 1 and Case 2 of Figure 10, it is assumed that the vehicle 100 is automatically running following the vehicle ahead and the driver 10 has released his / her hand from the steering wheel 42.

[0072] Case 1 of Figure 10 is a successful example of behavior induction. First, in scene B1 at time T1, it is assumed that the following driving function for the vehicle ahead is about to be lost and the vehicle 100 needs to provide information to the driver 10 to the effect that the driver 10 should hold the steering wheel 42. At this time, since the driver 10 has completed the user learning that "when the steering wheel 42 is held, the agent 13 wakes up", the agent 13 executes a behavior induction operation expressing a waking-up expression as shown in scene B2 at time T2. Then, when the driver 10 recognizes the agent 13 with a suddenly waking-up expression within the field of vision, the driver 10 intuitively and almost reflexively understands that he / she should hold the steering wheel 42. Therefore, in scene B3 at time T3, the driver 10 almost reflexively holds the steering wheel 42. When such a predetermined operation that the driver 10 holds the steering wheel 42 is performed, as in scene B4 at time T4, the agent 13 executes a first emotion such as smiling.

[0073] On the other hand, Case 2 in FIG. 10 is an example of the operation when the action inducement fails. First, similar to Scene B1, in Scene B1' at time T2, the following vehicle function for following the vehicle ahead is about to fail, and it is a scene where the vehicle 100 needs to provide information to the driver 10 indicating that the driver 10 should grip the steering wheel 42. Then, in Scene B2' at time T2, the agent 13 executes an action inducement operation expressing an awakening expression. However, here, although it is a situation with a high importance regarding the driving operation of gripping the steering wheel 42, it is assumed that the driver 10 did not notice the action inducement operation of the agent 13. In this case, in the subsequent Scene B3' at time T3, the agent 13 emphasizes the action inducement operation, for example, by maintaining the essential aspect of the action inducement operation such as an awakened expression and further changing the facial color. As a result, the driver 10 notices the action inducement operation of the agent 13 and can perform a predetermined operation of gripping the steering wheel 42 in Scene B4' at time T4.

[0074] As described above, the information provision support method according to the present embodiment is an information provision support method for providing information to the driver 10 using the information provision support device 101 that supports the information provision from the vehicle 100 to the driver 10. In this information provision support method, the information to be provided to the driver 10 is acquired from the vehicle 100 as the situation information 32. Also, the operation of the driver 10 is recognized. And in order to obtain the situation information 32 or based on the situation information 32, when the driver 10 performs a predetermined operation that should be performed, a response operation that the information provision support device 101 may have performed according to the predetermined operation is executed as an action inducement operation by the information provision support device 101 after the acquisition of the situation information 32 and before the driver 10 performs the predetermined operation.

[0075] In this way, when the response operation is executed as an action-inducing operation before the driver 10 performs a predetermined operation, the driver 10 can intuitively and almost reflexively perceive the predetermined operation associated with the situation information 32. Therefore, according to the information-providing support method according to the present embodiment, the predetermined operation of the driver 10 is induced. That is, according to the information-providing support method of the present embodiment, the information-providing support device 101 can act on the driver 10 so that the driver 10 naturally wants to perform the predetermined operation. As a result, according to the information-providing support method of the present embodiment, the information-providing support device 101 can very naturally induce the necessary actions of the driver 10 according to individual specific situations.

[0076] For example, when the driver 10 receives a lot of information from the vehicle 100, direct information provision may be felt annoying, and the driver 10 may be slow to respond even to important information provision. However, according to the information provision by the above-described action induction, the predetermined operation that the driver 10 should perform can be elicited earlier and more surely than in the case of direct information provision.

[0077] In the information-providing support method according to the present embodiment, on the premise that when the driver 10 performs a predetermined operation, a response operation is executed by the information-providing support device 101. Also, it is determined whether the driver 10 has recognized the response operation executed by the information-providing support device 101, and the number of times of recognizing the response operation executed by the information-providing support device 101 that the driver 10 has recognized is counted. Then, when the number of times of recognizing the response operation is equal to or greater than a threshold value, before the driver 10 performs the predetermined operation, the information-providing support device 101 executes the response operation as an action-inducing operation.

[0078] In this way, when the learning of the driver 10 as a user (user learning) regarding the response operation is completed, by executing the response operation as an action-inducing operation, the predetermined operation of the driver 10 can be induced particularly surely. This is because, since the user learning is completed, the driver 10 can intuitively perceive the predetermined operation suggested by the action-inducing operation and reflexively perform it without being confused by the action-inducing operation of the information-providing support device 101.

[0079] In the information-providing support method according to the present embodiment, after the execution of the response operation as the action-inducing operation by the information-providing support device 101, when the driver 10 performs a predetermined operation, the information-providing support device 101 executes a first emotion that is an operation expressing a positive emotion. In this way, after the success of the action induction, when the positive first emotion is executed, it is recognized that the predetermined operation to be performed by the driver 10 has been suggested by the action-inducing operation, and it becomes easier for the information-providing support device 101 to have positive feelings. As a result, it becomes easier for the subsequent action inductions to be successful.

[0080] In the above embodiment, the information-providing support device 101 particularly executes the response operation and the action-inducing operation by the agent 13 that is a mascot, and the first emotion is expressed by an expression, color, sound or voice, or an operation or the like. For this reason, it becomes easier to have positive feelings towards the agent 13 due to the first emotion, and the success rate of the action induction is particularly likely to improve.

[0081] In the information-providing method according to the present embodiment, the importance of the situation information 32 to be provided to the driver 10 is determined. Then, after the execution of the response operation as the action-inducing operation by the information-providing support device 101, when the driver 10 does not perform a predetermined operation, the operation of the information-providing support device 101 is changed according to the importance. Thereby, when the action induction fails, it is possible to perform an appropriate operation according to the importance of the situation information 32, such as leading the action induction to success or increasing the success rate of the next action induction.

[0082] In the above embodiment, in particular, when it is determined that the importance of the situation information 32 is high, the information providing support device 101 executes a response operation emphasized as an action inducing operation. For this reason, when the situation information 32 is related to a driving operation or the like and its importance is high, the information providing support device 101 can particularly actively appeal to the driver 10 and successfully lead to the induction of a predetermined operation.

[0083] Also, in the above embodiment, in particular, when it is determined that the importance of the situation information 32 is low, the information providing support device 101 executes a second emotion, which is an operation expressing a spleen-removed emotion. When the driver 10 senses the negative second emotion by the information providing support device 101, the driver 10 naturally becomes more attentive to the operation of the agent 13, so that it becomes easier to succeed in inducing actions after the next time. Further, when the importance of the situation information 32 is low, the information providing support device 101 appeals to the driver 10 moderately by executing the second emotion, so that the annoyance related to information provision is reduced.

[0084] Furthermore, in the above embodiment, the information providing support device 101 particularly executes a response operation and an action inducing operation by the agent 13 which is a mascot, and the second emotion is expressed by an expression, a color, a sound or a voice, or an operation or the like. For this reason, by recognizing the second emotion, the driver 10 becomes more concerned about the operation of the agent 13, so that the success rate of inducing actions after the next time is particularly likely to be improved.

[0085] In the above embodiment, when the situation information 32 is information related to the driving operation of the vehicle 100, its importance is determined to be high, and when it is other information, its importance is determined to be low. In this way, when determining that the importance of the situation information 32 related to the driving operation is high, it is possible to more reliably induce a predetermined operation of the driver 10 regarding the provision of the situation information 32 related to the driving operation. As a result, safety is particularly easily ensured.

[0086] The embodiments of the present invention have been described above. However, the configurations described in the above embodiments merely show some application examples of the present invention and are not intended to limit the technical scope of the present invention.

[0087] For example, the information provided by the vehicle 100 to the driver 10 is not limited to the examples in the above embodiments. Also, the operations to be performed by the driver 10 in order to obtain the information provided by the vehicle 100 or based on the information provided by the vehicle 100 are not limited to the examples in the above embodiments. Therefore, the present invention can be applied to combinations of various information provided by the vehicle 100 to the driver 10 and various operations to be performed by the driver 10.

[0088] In addition, the in-vehicle camera 17 is provided in the vehicle 100 as one of the sensors 16, but is not limited thereto. There may be a case where the information providing support device 101 independently includes a camera, and the interior of the vehicle 100 can be photographed to such an extent that the operation of the driver 10 can be recognized by this camera. In this case, the information providing support device 101 can acquire the in-vehicle image 31 using the camera independently provided by the information providing support device 101 instead of the in-vehicle camera 17 provided in the vehicle 100.

[0089] The situation information 32 in the above embodiment and the corresponding response operations and action inducing operations are merely examples. Therefore, the information providing support method and the like of the present invention can be applied to various scenes other than the scenes described in the above embodiment.

[0090] For example, when the driver 10 is operating the air conditioner and the agent 13 makes a response action of looking at the air conditioner, when the driver 10 or a passenger sitting in the front passenger seat or the rear seat looks cold or hot, the agent 13 can perform an action-inducing operation of looking at the air conditioner. When the driver 10 is operating the car navigation system 12 and the agent 13 makes a response action of looking at the car navigation system 12, the agent 13 can perform an action-inducing operation of looking at the car navigation system 12 when it wants the driver to look at the car navigation system 12 or when it wants the driver to operate it. When locking the door of the vehicle 100 and the agent 13 makes a response action of barking, for example, "wan", the agent 13 can perform an action-inducing operation of barking "wan" when it wants the door of the vehicle 100 to be locked. Further, regardless of whether the vehicle 100 is performing automatic following driving, in a scene where the steering wheel 42 should be gripped, the agent 13 can make a reaction such as waking up.

[0091] The information providing support device 101 of the above embodiment can be implemented by a so-called AI (Artificial Intelligence) program. The AI program may be referred to as a learned model. Further, although the information providing support device 101 of the above embodiment is clearly distinguished from the vehicle 100, the information providing support device 101 can be implemented integrally with the vehicle 100 as a part of the vehicle 100. That is, some or all of the functions of the agent controller 14 can be realized by the vehicle controller 15.

Description of Signs

[0092] 10: Driver, 11: Driving operation unit, 12: Car navigation system, 13: Agent, 14: Agent controller, 15: Vehicle controller, 16: Sensor, 17: In-vehicle camera, 21: Information acquisition unit, 22: Driver action recognition unit, 23: Action induction determination unit, 24: Agent action control unit, 25: Action pattern memory unit, 31: In-vehicle image, 32: Situation information, 33: Equipment operation information, 34: Action detection unit, 35: Gaze detection unit, 36: Equipment operation detection unit, 37: Importance determination unit, 38: Learning status determination unit, 39: Response action control unit, 40: Action induction action control unit, 41: Target of attention, 42: Steering wheel, 100: Vehicle, 101: Information providing support device, 102: Information providing system

Claims

1. An information providing support method for providing information to a driver using an information providing support device that supports information provision from a vehicle to the driver, comprising: acquiring information to be provided to the driver from the vehicle; recognizing the driver's actions; after acquiring the information, before the driver performs the predetermined action, causing the information providing support device to execute a response action that the information providing support device may have executed in response to the predetermined action when the driver performs the predetermined action in order to obtain the information or based on the information; An information providing support method.

2. The information providing support method according to claim 1, comprising: when the driver performs the predetermined action, causing the information providing support device to execute the response action; determining whether the driver has recognized the response action executed by the information providing support device; counting the number of times the driver has recognized the response action executed by the information providing support device; when the number is equal to or greater than a predetermined threshold, causing the information providing support device to execute the response action before the driver performs the predetermined action; An information providing support method.

3. The information providing support method according to claim 1 or 2, comprising: after the information providing support device executes the response action, when the driver performs the predetermined action, causing the information providing support device to execute a first emotion, which is an action expressing a positive emotion; An information providing support method.

4. The information providing support method according to claim 3, comprising: the information providing support device executing the response action by a mascot; the first emotion being expressed by the expression, color, sound or voice, or movement of the mascot; An information providing support method.

5. The information providing support method according to any one of claims 1 to 3, comprising: determining the importance of the information; after the information providing support device executes the response action, when the driver does not perform the predetermined action, changing the operation of the information providing support device according to the importance; An information providing support method.

6. The information providing support method according to claim 5, comprising: when it is determined that the importance is high, causing the information providing support device to execute the emphasized response action; An information providing support method.

7. The information providing support method according to claim 5, comprising: When it is determined that the importance is low, the information providing support apparatus executes a second emotion which is an action expressing a negative emotion. Information providing support method.

8. The information providing support method according to claim 7, wherein the information providing support apparatus executes the response action by a mascot, and the second emotion is expressed by the expression, color, sound or voice, or an action of the mascot. Information providing support method.

9. The information providing support method according to any one of claims 5 to 8, wherein when the information is information related to the driving operation of the vehicle, it is determined that the importance is high, and when the information is information other than the information related to the driving operation of the vehicle, it is determined that the importance is low. Information providing support method.

10. An information providing support apparatus for supporting information provision from a vehicle to a driver, an information acquisition unit that acquires information to be provided to the driver from the vehicle, a driver action recognition unit that recognizes the actions of the driver, and after acquiring the information, before the driver performs the predetermined action, executes a response action that may have been executed in response to the predetermined action when the driver performs the predetermined action to obtain the information or based on the information. Information providing support apparatus.

11. An information providing support program for operating an information providing support apparatus for supporting information provision from a vehicle to a driver, causing the information providing support apparatus to acquire information to be provided to the driver from the vehicle, causing the information providing support apparatus to recognize the actions of the driver, and after acquiring the information, before the driver performs the predetermined action, causing the information providing support apparatus to execute a response action that may have been executed in response to the predetermined action when the driver performs the predetermined action to obtain the information or based on the information. Information providing support program.

12. An information providing system including a vehicle and an information providing support apparatus for supporting information provision from the vehicle to a driver, wherein the information providing support apparatus is used to provide information to the driver, wherein the information providing support apparatus acquires information to be provided to the driver from the vehicle, and the information providing support apparatus recognizes the actions of the driver. Before the driver performs the predetermined operation, in order to obtain the information or based on the information, the driver executes a response operation that the information providing support device may have executed in response to the predetermined operation. An information providing system.

Images