Information notification system
The system addresses user discomfort and distraction by customizing vehicle notifications based on occupant characteristics, allowing user acceptance, thus improving user experience.
Patent Information
- Application Number
- JP2025078771
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2026-11-19
AI Technical Summary
Existing information notification systems in vehicles often notify users in manners undesirable to them, causing confusion and discomfort, and existing solutions that attempt to customize notifications may further distract the user without their knowledge.
An information notification system that detects occupant characteristics, including driving ability, dynamic state, and purpose of travel, to customize the timing and content of notifications, allowing occupants to accept or reject proposals for customization.
Customizes notifications to suit individual occupants, reducing confusion and discomfort by ensuring notifications are tailored to their preferences and needs, thereby enhancing user acceptance and reducing distraction.
Smart Images

Figure 2026182491000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an information notification system.
Background Art
[0002] For example, Patent Document 1 discloses a driving behavior change support device that supports changes in the driving behavior of a vehicle driver. The processor of this device performs providing advance information to the driver to enhance the driver's support acceptability, and determining the mode of actual support according to the driver's reaction to the advance information to support the driver.
[0003] Here, according to the aforementioned Patent Document 1, the processor executes the provision of the advance information before the driver gets in the vehicle and starts driving. The processor also executes the actual support through the HMI device during driving from when the driver starts driving to when driving ends, based on the mode determined according to the advance information.
[0004] Further, Patent Document 2 discloses an information notification system configured to display notifications to a vehicle user on a display device. The processor of this device prohibits displaying a first-type notification related to roads or traffic around the vehicle and a second-type notification different from the first-type notification in an overlapping manner on the display device.
[0005] Here, according to the aforementioned Patent Document 2, the second-type notification includes a user setting notification related to settings customized by the user. The user can customize various settings by using an input device.
[0006] Further, Patent Document 3 discloses an information notification system that notifies occupants of information. This information notification system includes a matching information determination unit that determines information matching the preferences of a vehicle occupant, and a notification unit that notifies the determined information.
[0007] Furthermore, Patent Document 4 discloses an information notification system for notifying a vehicle driver of information. This information notification system includes an information notification control device that stores weighted data in relation to the manner in which the information is notified.
[0008] According to Patent Document 4, the information notification control device updates weighted data based on the estimated result of the driver's emotions and notifies the driver of the information in a notification manner determined based on the weighted data. [Prior art documents] [Patent Documents]
[0009] [Patent Document 1] Japanese Patent Publication No. 2024-157854 [Patent Document 2] Japanese Patent Publication No. 2024-060481 [Patent Document 3] Japanese Patent Publication No. 2020-085568 [Patent Document 4] Japanese Patent Publication No. 2025-014571 [Overview of the Initiative] [Problems that the invention aims to solve]
[0010] Incidentally, the inventions described in each of the aforementioned Patent Documents 1 to 4 all involve a device mounted on a vehicle unilaterally notifying the user of information determined by the device. Therefore, depending on the manner in which the information is notified, it may be notified in a manner undesirable to the user or may cause the user to feel annoyed.
[0011] Regarding the manner in which information is notified, Patent Document 1 discloses changing the manner of actual support in accordance with the driver's reaction to the prior information. Furthermore, Patent Document 4 discloses estimating the emotions of the driver who has been notified of the information and updating the manner in which the information is notified based on the estimation result.
[0012] However, updating notification patterns without the driver's knowledge could confuse and unsettle them. Since this confusion and unsettlement could lead to driver distraction, mitigating it would be beneficial. These challenges are common not only to vehicles but to all mobile devices.
[0013] This disclosure is made in view of the above, and its purpose is to customize the manner in which information is notified to suit the occupants of a vehicle, while reducing the confusion and discomfort caused to the occupants by such customization. [Means for solving the problem]
[0014] A first aspect of this disclosure relates to an information notification system for notifying an occupant of a mobile vehicle. The information notification device includes a notification device for notifying the occupant of information; a detection device mounted on the mobile vehicle for detecting information related to the occupant and the mobile vehicle; an estimation unit that estimates occupant characteristics including one or more of the occupant's driving ability, a dynamically changing dynamic state relating to the occupant, and the occupant's purpose of travel, based on the detection signal of the detection device; and a notification proposal unit that determines the timing for proposing customization of the information to be notified to the occupant and the content of the proposed customization, and executes the proposed customization to the occupant based on the determination result, wherein the notification device executes the notification of the information with the proposed content reflected, depending on whether the occupant accepts or rejects the proposal.
[0015] According to the first embodiment described above, the notification and proposal unit not only determines the content of the customization proposal regarding the information, but also executes the customization proposal at a timing based on the occupant characteristics.
[0016] This approach not only enables the notification of information in a manner that is tailored to the crew, but also allows the crew themselves to decide whether or not to accept it. By reflecting the crew's own will, the confusion and discomfort caused by customization can be reduced.
[0017] In this process, by having the device determine the timing of suggesting customization based on the occupant's characteristics, the confusion and discomfort caused to the occupant by customization can be further reduced.
[0018] Furthermore, according to a second aspect of this disclosure, the estimation unit may estimate the driving capacity, the dynamic state, and the purpose of movement, and the notification proposal unit may determine, based on the combination of the driving capacity, the dynamic state, and the purpose of movement, the content of the proposed customization and the timing of the proposed customization corresponding to each type of information subject to the proposed customization.
[0019] According to the second embodiment described above, a customized approach suitable for each type of information can be proposed for each crew member. This makes it possible to provide information in a manner more tailored to the crew member compared to a configuration that uniformly customizes all information.
[0020] Furthermore, according to a third aspect of this disclosure, the proposed customization may include a proposal for customization relating to one or more of the following: the content of the notification of the information, the means of notifying the information, and the order of notification of the information when the information is present in multiple locations.
[0021] According to the third embodiment described above, by narrowing down the types of customization proposals, the effort required from the crew can be reduced compared to methods that offer an excessively high degree of freedom. This is advantageous in reducing the inconvenience imposed on the crew.
[0022] Further, according to the fourth aspect of the present disclosure, the notification proposing unit may determine the proposal timing for the customization based on the change in the driving capability, and the notification proposing unit may set information related to driving support corresponding to the traffic situation around the mobile object as a proposal target for the customization.
[0023] According to the fourth aspect, by determining the proposal timing with reference to the change in driving capability, the customization can be proposed at a more appropriate timing.
[0024] Further, according to the fifth aspect of the present disclosure, the driving capability may include a combination of the occupant's cognitive capability and the occupant's driving skill.
[0025] In general, it is considered that the notification mode adapted to the occupant, such as the level of detail to which navigation information is notified, may vary depending on the occupant's cognitive capability or driving skill. The present disclosure suitably acts on the driving capability as in the fifth aspect described above.
[0026] Further, according to the sixth aspect of the present disclosure, the notification means includes a visual means for notifying visual information, an auditory means for notifying auditory information, and a tactile means for notifying tactile information, and when the cognitive capability is low, the notification proposing unit may propose to the occupant, as the proposed content of the customization, that the notification frequency by the visual means is reduced and the notification frequency by the auditory means or the tactile means is increased, compared to when the cognitive capability is high.
[0027] In general, it is considered that as the occupant's cognitive capability decreases, the occupant's attention level also decreases. In this case, if visual information is heavily used, the occupant's attention is likely to be scattered along with the movement of the line of sight. Therefore, as in the sixth aspect, by increasing the notification frequency by the auditory means or the tactile means according to the level of cognitive capability, the movement of the occupant's line of sight can be suppressed.
[0028] Furthermore, according to a seventh aspect of this disclosure, the estimation unit may classify the occupant's dynamic state into one or more categories, and the notification suggestion unit may make the information relating to driving assistance corresponding to the categories classified by the estimation unit the subject of the customization suggestion.
[0029] According to the seventh embodiment described above, information corresponding to dynamic states can be included in the proposed customizations. This makes it possible to notify information corresponding to the dynamic states of the occupants.
[0030] Furthermore, according to an eighth aspect of this disclosure, the dynamic state may include, as the plurality of categories, a plurality of categories relating to the emotional and physical state of the occupant.
[0031] According to the eighth embodiment described above, information corresponding to the occupant's emotions or physical condition can be included in the suggestions for customization. This makes it possible to notify the occupant of information that corresponds to their mental and physical state.
[0032] Furthermore, according to a ninth aspect of this disclosure, the detection device may be mounted on the mobile body and include a terminal that receives input of the purpose of movement, the notification and proposal unit may determine that it is time to propose the customization when the purpose of movement is input to the terminal, and the notification and proposal unit may make information related to driving assistance corresponding to the purpose of movement the subject of the customization proposal.
[0033] According to the ninth embodiment described above, by determining the timing of a suggestion when the purpose of movement is entered, customization can be suggested at a more appropriate timing.
[0034] Furthermore, according to a tenth aspect of this disclosure, the information relating to driving assistance corresponding to the purpose of travel may include the travel path of the moving body and the stopping spots or passing spots of the moving body located along the travel path, the estimation unit classifies the occupant's purpose of travel into one or more categories, and the notification suggestion unit notifies the occupant, based on the categories classified by the estimation unit, whether or not to notify the stopping spots or passing spots as a suggested customization.
[0035] Generally, it is conceivable to notify passengers of the locations of landmarks, tourist spots, and other points of interest as information. However, depending on the passengers' purpose of travel, presenting such points of interest may be bothersome to them. Therefore, a configuration that decides not to display such points of interest based on the purpose of travel is conceivable, but if the device performs such processing autonomously, it may confuse the passengers. Therefore, as in the tenth embodiment described above, by having the passengers confirm whether or not they need to be notified of stopping points or passing points, the confusion caused to the passengers can be reduced.
[0036] Furthermore, according to an eleventh aspect of this disclosure, the estimation unit may estimate the degree of interest of the occupant to the notification of the information based on the occupant's response to the notification, and the notification suggestion unit may set the timing of the customization suggestion to a higher frequency when the degree of interest is high compared to when the degree of interest is low.
[0037] According to the 11th embodiment described above, if the level of interest is low, there is a possibility that the customer will not even be interested in the customization proposal in the first place. By varying the frequency of proposals according to the level of interest, customization can be proposed at a more appropriate time.
[0038] Furthermore, according to a twelfth aspect of this disclosure, the estimation unit may estimate the degree of trust with the occupant based on the occupant's response to the notification of the information, and the notification proposal unit may, based on the degree of trust, set the timing of the customization proposal to a lower frequency when the degree of trust is high compared to when the degree of trust is low, and set the process of automatically reflecting the proposed content without requiring the occupant to accept or reject the proposal to a higher frequency.
[0039] According to the 12th embodiment described above, if the reliability is high, it is considered acceptable to the crew even if the customization is automatically reflected without asking whether or not to accept it. By varying the frequency of proposal timing according to the reliability, customization can be proposed at a more appropriate timing. [Effects of the Invention]
[0040] As explained above, this disclosure makes it possible to customize the manner in which information is notified to suit the occupant driving the vehicle, while reducing the confusion and discomfort caused to the occupant as a result of such customization. [Brief explanation of the drawing]
[0041] [Figure 1] Figure 1 is a schematic diagram illustrating a crew support system as an information notification system. [Figure 2] Figure 2 is a block diagram showing an example configuration of an in-vehicle system. [Figure 3A] Figure 3A is a schematic diagram illustrating the input and output in the crew assistance system. [Figure 3B] Figure 3B is a schematic diagram illustrating the display screen of a terminal. [Figure 4] Figure 4 is a schematic diagram illustrating an example of the target of estimation by the estimation unit. [Figure 5] Figure 5 is a diagram illustrating the confidence estimation method. [Figure 6] Figure 6 is a flowchart illustrating the procedure for estimating the level of interest. [Figure 7]Figure 7 is a flowchart illustrating the procedure for calculating the bias index. [Figure 8A] Figure 8A is a diagram illustrating the procedure for calculating bias indicators related to current bias. [Figure 8B] Figure 8B is a graph illustrating the relationship between bias indicators and confidence levels. [Figure 9] Figure 9 is a schematic diagram illustrating the relationship between crew characteristics and the criteria for judgment by the notification proposal unit. [Figure 10] Figure 10 is a table illustrating the decisions made by the notification proposal department. [Figure 11] Figure 11 is a graph illustrating the relationship between the level of confidence and the frequency of proposal timing. [Figure 12] Figure 12 is a graph illustrating the relationship between cognitive ability and the frequency of notifications from each notification method. [Figure 13] Figure 13 is a flowchart illustrating the steps involved in a custom proposal. [Modes for carrying out the invention]
[0042] The embodiments of this disclosure will be described below with reference to the drawings. Note that the following description is illustrative.
[0043] <1. Crew Assistance System> Figure 1 is a schematic diagram illustrating an example of an occupant support system S as an information notification system. Figure 2 is a block diagram showing an example configuration of the in-vehicle system 1. Figure 3A is a schematic diagram illustrating input and output in the occupant support system S. Figure 3B is a schematic diagram illustrating the display screen of terminal 11A.
[0044] (1-1. Overall Structure) As illustrated in Figure 1, the occupant support system S functions as a mobile vehicle control system that controls a mobile vehicle carrying one or more occupants U. The one or more occupants U include the occupant U who drives the mobile vehicle, i.e., the driver U1. In this embodiment, the mobile vehicle is a vehicle V. Controlling the vehicle V includes proposing various services to the occupants U and controlling the vehicle V based on whether or not the occupants U accept or reject the proposals. It should be noted that it is not essential for the occupant support system S to function as a mobile vehicle control system.
[0045] Furthermore, the occupant support system S also functions as an occupant characteristic estimation system that estimates occupant characteristics C. Occupant characteristics C represent the characteristics of the occupants U of the vehicle V as a moving object, for example, the characteristics of the driver U1. In this embodiment, the occupant support system S estimates the quantified occupant characteristics C for the driver U1.
[0046] Here, the terms "characteristics of crew member U," and by extension "crew member characteristics C," are merely illustrative terms introduced for convenience. Instead, the terms "characteristics of crew member U" and "crew member characteristics" may be used, as well as the terms "state of crew member U" and "crew member state." Note that "state of crew member U" here refers to a concept that includes dynamic states other than "dynamic state C5," which will be discussed later, and sexual states in general.
[0047] As illustrated in Figure 1, the occupant support system S according to this embodiment also functions as an information notification system that notifies the driver U1 of information In. This notification is made by the notification proposal unit 112, which will be described later.
[0048] Here, the information In notified by the crew support system S, which acts as an information notification system, includes a proposal to the crew member U. In other words, the crew support system S can also be considered a proposal system that executes the proposal to the crew member U. The proposal to the crew member U is notified as information In in the form of visual information (e.g., text information), audio information, and tactile information. Hereinafter, "information In notified to the crew member U" will also be simply referred to as "notification information In".
[0049] The suggestions made by the occupant support system S include suggestions for occupant actions to be performed by the occupant U themselves. These suggestions for occupant actions are notified to the occupant U in the form of text information or the like. Occupant actions include the act of moving to a predetermined destination Lp. As illustrated in Figure 3B, the suggestions for the act of moving include, for example, at least one of a suggestion of a specific destination Lp and a suggestion of a travel route Tp to said destination Lp. The act of moving to the destination Lp may be an act performed by the occupant U driving the vehicle V, or it may be an act performed by the occupant U allowing the vehicle V to be driven automatically.
[0050] Here, the term "destination Lp" is used in a broad sense. That is, in this embodiment, destination Lp may be a specific point such as a building, or it may be an entire region including that specific point (so to speak, the target area). Furthermore, the term destination Lp includes events held at a specific location (for example, tourist events).
[0051] Furthermore, in this embodiment, the suggestion of movement includes, in place of or in addition to, the suggestion of the movement route Tp of vehicle V, a suggestion of a driving operation by occupant U. The suggestion of a driving operation includes one or more suggestions for lane changes, merging, and right or left turns of vehicle V. The suggestion of a right or left turn includes, for example, a suggestion for a right turn operation at an intersection.
[0052] Each of the suggested crew actions can be rephrased as "provision of services." Thus, the suggested crew actions include the provision of a destination suggestion service configured to suggest the aforementioned travel activities, and the provision of a tourism support service, which provides crew member U with tourism support information, as an alternative to the suggested travel activities. Hereafter, the suggested crew actions may be referred to as "action suggestions." Details of action suggestions will be described later.
[0053] Furthermore, the suggestions made by the crew support system S include suggestions for customizing the notification information In itself. By customizing this notification information In, various suggestions, including action suggestions, can be sent to the crew member U with different notification content, notification methods, and priorities. Hereafter, suggestions for customizing the notification information In may be referred to as "custom suggestions." Details of this function will be described later.
[0054] Furthermore, the vehicle V as a mobile entity may be an automobile using an internal combustion engine as its power unit, an electric vehicle using an electric motor, or a hybrid vehicle using both an internal combustion engine and an electric motor. The fuel used in the internal combustion engine may include any fuel. In this embodiment, vehicle V is a four-wheeled vehicle. Vehicle V is an SDV (Software Defined Vehicle) and can provide various functions to the occupant U, such as the services described later.
[0055] As shown in Figure 1, the occupant support system S comprises an in-vehicle system 1 mounted on the vehicle V, and an external server 1000 connected to the in-vehicle system 1 via a cloud network N1. The in-vehicle system 1 and the external server 1000 are connected to each other so that data can be sent and received from each other.
[0056] Here, the in-vehicle system 1, as shown in Figure 2, includes an in-vehicle device 10, an in-vehicle computer 100, and an in-vehicle database 200. The in-vehicle device 10 is electrically connected to the in-vehicle computer 100 and the in-vehicle database 200 via an in-vehicle network. The in-vehicle database 200 is an example of an "internal database" in this embodiment.
[0057] The in-vehicle device 10 includes a notification device 11, a vehicle control device 12, a biometric information detection device 13, and a movement information detection device 14. All of these devices are mounted on the vehicle V.
[0058] (1-2. Notification Devices) The notification device 11 is one or more devices that notify the driver U1 of information In. The notification device 11 includes a terminal 11A and a vibration device 11B. The terminal 11A and the vibration device 11B each exemplify the "notification unit" in this embodiment in that they can notify the occupant U of information.
[0059] (1-2-A. Terminal) Terminal 11A notifies crew member U of the proposal from the notification / proposal unit 112 as information In, which is composed of visual and auditory information. More specifically, terminal 11A has notification functions for action suggestions and custom suggestions. Furthermore, terminal 11A also has functions for presenting and answering questionnaires for crew member U.
[0060] It is not mandatory for terminal 11A to have a questionnaire presentation function and a questionnaire response function. The presentation function may be implemented via a navigation device separate from terminal 11A, or via a communication terminal owned by crew member U, such as a Cell Phone.
[0061] Specifically, the terminal 11A according to this embodiment has a display 15 such as a liquid crystal panel or an organic EL panel, and a speaker 16 (shown only in Figure 3A).
[0062] The display 15 displays the proposed content from the notification / proposal unit 112 as visual information on the display screen Scr. The speaker 16 emits the proposed content from the notification / proposal unit 112 as auditory information (sound information).
[0063] It is not mandatory to configure the terminal 11A with the display 15 and speaker 16. The terminal 11A may be configured with the display 15 alone, or with the speaker 16 alone.
[0064] For example, terminal 11A may display text indicating the proposed content, or locations or routes superimposed on a map, on the display screen Scr, or emit sound corresponding to the text. In addition to such displays or sounds, terminal 11A can also visualize and display the survey response items using text or the like.
[0065] Terminal 11A is also configured as an interactive terminal device. Specifically, terminal 11A is configured to accept or reject a proposal from crew member U, to receive modifications from crew member U to the proposal, and to receive the crew member's evaluation of the proposal. Here, the term "proposal" can be replaced with "service provision content" or simply "service content."
[0066] More specifically, terminal 11A can also receive responses to a first questionnaire A1 that characterizes crew characteristics C, particularly preference characteristics C7 described later, and responses to a second questionnaire A2 that characterizes the crew member U's purpose of travel C4 (see Figure 4 below).
[0067] Specifically, the terminal 11A according to this embodiment is configured by using a touch panel on the display 15 described above. This terminal 11A can also be considered as a "reception unit" that receives input and operations performed by the crew member U.
[0068] It is not mandatory to configure the terminal 11A using a touch panel on the display 15. The reception unit of the terminal 11A may be configured using a separate microphone.
[0069] Terminal 11A then receives responses regarding crew member U's age, gender, hobbies, height, weight, build, and family structure as the first questionnaire A1. Note that including age and gender in the response items is not mandatory. As described later, depending on the usage flag, questionnaires regarding age, gender, height, weight, and build may be omitted.
[0070] Terminal 11A also accepts responses related to the occupant U's destination Lp and the purpose of travel to that destination Lp, as the second questionnaire A2. Specifically, the second questionnaire A2 includes questions that can determine whether or not the purpose of travel is to a particular destination Lp. Terminal 11A functions as a device (detection device) mounted on the vehicle V that accepts input of the purpose of travel C4.
[0071] If the purpose is to travel to a specific destination Lp, terminal 11A will accept a response regarding the purpose of travel to that destination Lp as the second questionnaire A2. If the purpose is not to travel to a specific destination Lp, terminal 11A may accept a response regarding the region that the occupant U considers to be the destination Lp.
[0072] The content I1 of the conversation between occupant U and terminal 11A is converted into an electrical signal corresponding to the content I1 and transmitted to the in-vehicle database 200. The in-vehicle database 200 stores the content I1 (see Figures 2 and 4). This content I1 includes responses to the first questionnaire A1 and the second questionnaire A2, as well as the acceptance, rejection, modification, and evaluation of proposals by occupant U. The content I1 may be stored in the storage 103 of the in-vehicle computer 100 instead of, or in addition to, storage in the in-vehicle database 200.
[0073] The content of the dialogue between occupant U and terminal 11A includes information indicating the relationship between occupant U and vehicle V. In this embodiment, terminal 11A, along with the biometric information detection device 13 and the movement information detection device 14, exemplifies the "detection device" in this embodiment in that it can output detection signals related to both occupant U and vehicle V.
[0074] (1-2-B. Vibration Devices) The vibration device 11B notifies the occupant U of the suggestion from the notification suggestion unit 112 as notification information In, which is composed of tactile information. Specifically, the vibration device 11B, like the terminal 11A, has notification functions for action suggestions and custom suggestions.
[0075] Specifically, the vibration device 11B according to this embodiment is located on at least one of the steering wheel and the seat of the vehicle V, and vibrates in response to electrical signals from the on-board computer 100. In this embodiment, the intensity of vibration of the vibration device 11B can be changed via electrical signals from the on-board computer 100.
[0076] (1-3. Vehicle control devices) The vehicle control device 12 includes lighting equipment 121, air conditioning equipment 122, and sound equipment 123. Each of these devices functions as an actuator that controls the vehicle V by receiving control signals from the onboard computer 100.
[0077] Lighting equipment 121 functions as lighting to illuminate the interior of the vehicle. Air conditioning equipment 122 performs air conditioning such as heating, cooling, and ventilation to maintain a comfortable environment inside the vehicle. Audio equipment 123 controls the sound inside the vehicle. For example, audio equipment 123 can play music inside the vehicle or provide voice notifications of various suggestions to the occupant U. Voice notifications of suggestions are not mandatory.
[0078] The audio device 123 can be used in addition to, or in place of, the speaker 16 of the terminal 11A, as a notification unit that notifies auditory information (voice information) indicating the content of the proposal from the notification proposal unit 112.
[0079] (1-4. Biometric Information Detection Devices) The biometric information detection device 13 includes an in-vehicle camera 131, an in-vehicle microphone 132, and a vital sensor 133. These devices function as sensing devices that detect the biometric information I2 of the occupant U. The biometric information detection device 13 can be rephrased as the "biometric information detection unit" in this embodiment. The biometric information detection device 13, as the biometric information detection unit, exemplifies the "detection device" in this embodiment in that it detects information related to the occupant U.
[0080] The biometric information I2 includes a body image of the occupant U acquired via the in-vehicle camera 131, which acts as an imaging unit; voice data of the occupant U acquired via the in-vehicle microphone 132; and vital data of the occupant U acquired via the vital sensor 133.
[0081] The in-vehicle camera 131 captures images of the interior of the vehicle. More specifically, the in-vehicle camera 131 captures images of the body of the occupant U of the vehicle V. More specifically, the in-vehicle camera 131 captures one or more of the occupant U's facial expression, clothing, gaze, blinking, and body movements. Information such as facial expression and gaze can be obtained through processing performed by the in-vehicle computer 100. The in-vehicle camera 131 operates by receiving control signals from the in-vehicle computer 100 and generates electrical signals indicating the captured images.
[0082] The in-vehicle microphone 132 collects sound from inside the vehicle. For example, the in-vehicle microphone 132 collects conversations taking place inside the vehicle. The in-vehicle microphone 132 operates by receiving control signals from the in-vehicle computer 100 and generates electrical signals indicating the collected sound. Based on these electrical signals, the in-vehicle computer 100 can extract the semantic content, frequency, volume, intonation, etc., of the sound.
[0083] The vital sensor 133 includes one or more of the following: a heart rate sensor for detecting the heart rate of occupant U, an electromyography (EMG) sensor for detecting the electromyography (EMG) of occupant U, an electroencephalogram (EEG) sensor for detecting the electroencephalogram (EEG) of occupant U, and a temperature sensor for detecting the body temperature of occupant U. The vital sensor 133 detects one or more of the vital data, such as heart rate, EMG, EEG, and body temperature, and generates an electrical signal corresponding to the detected signals. The vital sensor 133 may also acquire the occupant U's biometric information I2 by communicating with a user-owned terminal such as a smartwatch.
[0084] The electrical signals generated by each sensing device constituting the biometric information detection device 13 are transmitted to the in-vehicle database 200 as electronic data representing the biometric information I2 of the occupant U. The in-vehicle database 200 stores the biometric information I2 (see Figure 2). The biometric information I2 may be stored in the in-vehicle computer 100 instead of, or in addition to, storage in the in-vehicle database 200.
[0085] Furthermore, the biometric information I2 includes, in addition to the information detected by the sensing device described above, information contained in or inferred from the dialogue content I1 via the terminal 11A. The biometric information I2 based on the dialogue content I1 includes physical information of the occupant U, such as height, weight, build, and gender, and information indicating the occupant U's physical condition (the occupant U's own subjective judgment). The occupant U's physical information and physical condition may be determined based on images captured by the in-vehicle camera 131.
[0086] (1-5. Movement Information Detection Device) The movement information detection device 14 includes an external camera 141 and a driving environment sensor 142. These devices function as sensing devices that detect movement information I3 that characterizes the characteristics of the vehicle V itself and the driving conditions of the vehicle V. The movement information detection device 14 and the biometric information detection device 13 can be rephrased as the "movement information detection unit" in this embodiment. The movement information detection device 14 as the movement information detection unit exemplifies the "detection device" in this embodiment in that it detects information related to a moving object (vehicle V).
[0087] The movement information I3 can be rephrased as vehicle information or the occupant's driving history. As vehicle information, movement information I3 acquires at least the location information of vehicle V as information that characterizes the driving status of vehicle V. The location information of vehicle V includes the GPS history information of vehicle V and the starting point Ps of vehicle V and the arrival point Pg of vehicle V corresponding to the destination Lp when moving along a predetermined movement path Tp. The location information of vehicle V includes, for example, the movement trajectory of vehicle V.
[0088] The external camera 141 captures images of the area outside the vehicle. More specifically, the external camera 141 captures images of the surrounding environment of vehicle V. Here, the term "surrounding environment of vehicle V" includes preceding vehicles, following vehicles, oncoming vehicles, pedestrians, various structures, and the traffic environment of vehicle V. The external camera 141 operates by receiving control signals from the onboard computer 100 and generates electrical signals indicating the captured images.
[0089] The driving environment sensor 142 detects the weather outside the vehicle, the location information of the vehicle V, and the surrounding environment. The driving environment sensor 142 consists of an illuminance sensor, a raindrop sensor, a GPS sensor, a LiDAR (Light Detection And Ranging) sensor, an outside temperature sensor, and a sound-collecting microphone. The illuminance sensor detects the illuminance of sunlight. The raindrop sensor detects raindrops caused by rainfall. The GPS sensor detects the location information of the vehicle V. The LiDAR emits laser light and detects the distance to an object or the shape of an object based on the reflected light. A radar that emits radio waves or millimeter waves may be used instead of LiDAR. Each of these devices generates an electrical signal indicating the detection result. The outside temperature sensor detects the temperature outside the vehicle. The sound-collecting microphone detects the presence and volume of noise outside the vehicle.
[0090] The electrical signals generated by each sensing device constituting the movement information detection device 14 are transmitted to the in-vehicle database 200 as electronic data indicating the movement information I3 of the vehicle V. The in-vehicle database 200 stores the movement information I3 (see Figure 2). The movement information I3 may be stored in the in-vehicle computer 100 instead of, or in addition to, storage in the in-vehicle database 200.
[0091] In addition, the mobility information detection device 14 may include a fuel level sensor for detecting the remaining amount of fuel or battery, a history sensor for detecting the riding history of occupant U, a vehicle speed sensor for detecting the vehicle speed of vehicle V, an accelerator pedal position sensor for detecting the degree of accelerator pedal opening, and the like.
[0092] Furthermore, the movement information I3 includes, in addition to the information detected by the sensing device described above, information contained in the dialogue content I1 via terminal 11A or information estimated from the dialogue content I1. The movement information I3 obtained or estimated from the dialogue content I1 includes information indicating the purpose of movement C4, such as the destination specified by the crew member U.
[0093] Furthermore, the mobility information I3 includes a usage flag indicating whether the vehicle V, as a mobile entity, is a shared item (a so-called "shared car") or an item temporarily leased (a so-called "rental car"). This usage flag is mobility information I3 that characterizes the characteristics of the vehicle V itself and is stored in advance in the in-vehicle database 200 or the in-vehicle computer 100.
[0094] Furthermore, the movement information I3 includes a time flag indicating the date and time of use of the vehicle V as a moving object. This time flag is movement information that characterizes the driving status of the vehicle V and is counted by a timer 105 provided in or connected to the onboard computer 100.
[0095] In addition, the movement information I3 includes the operation history of vehicle V. The operation history of vehicle V includes, for example, the accelerator pedal opening degree, brake pedal opening degree, and steering operation history.
[0096] (1-6. In-vehicle database) The in-vehicle database 200 is installed in vehicle V. The in-vehicle database 200 stores the driving history (movement information I3) of occupant U. More specifically, the in-vehicle database 200 stores the content I1 of the conversation between terminal 11A and occupant U, occupant U's biometric information I2, and the movement information I3 of occupant U and vehicle V. The in-vehicle database 200 is composed of storage capable of continuously storing various types of information. The in-vehicle database 200 can be composed of storage based on any standard, such as storage employing flash memory.
[0097] The in-vehicle database 200 is located inside the vehicle V and stores the content of the dialogue between the occupant U and the terminal 11A. Together with the external database 1010 of the external server 1000, it constitutes the "storage unit" according to this embodiment.
[0098] (1-7. Onboard Computers) The in-vehicle computer 100 is a computer in which a control unit 101 and an input / output interface (I / O interface) 102 are interconnected via a system bus.
[0099] The control unit 101 includes one or more processors 101a that execute various programs, a read-only ROM 101b that stores the BIOS and the like, and RAM 101c that functions as main memory.
[0100] In addition to the aforementioned terminal 11A, vehicle control device 12, biometric information detection device 13, movement information detection device 14, timer 105, and in-vehicle database 200, the input / output interface 102 is also connected to the storage 103 and the communication control unit 104. The connection via the input / output interface 102 may also be a connection via an in-vehicle network such as CAN.
[0101] The storage 103 stores various information related to the vehicle V and occupant U, and various programs for processing said information. The in-vehicle database 200 may also serve as the storage 103.
[0102] The communication control unit 104 is connected to the external server 1000 via the cloud network N1 based on a predetermined communication standard. The in-vehicle computer 100 sends and receives data to and from the external server 1000 via the communication control unit 104 and the cloud network N1.
[0103] Here, the external server 1000 includes a processor, ROM, RAM, and a communication interface, as well as an external database 1010.
[0104] The external database 1010 is located outside the vehicle V and stores information about other user groups U' besides the occupant U. Specifically, the external database 1010 stores a destination database 1011 associated with occupant characteristics C, an acoustic database 1012 corresponding to occupant characteristics C, and a tourism database 1013 containing regional information including event information and tourist information for each destination Lp. Occupant characteristics C will be described later.
[0105] The external database 1010 also stores the content I1 of conversations I1 between the terminal 11A of another vehicle V' and a group of other users U' other than the occupant U, biometric information I2 of the other group of users U', and movement information I3 of the other group of users U' and the other vehicle V'.
[0106] The external database 1010 is located outside the vehicle V' and stores the content of interactions between other user groups U' and other terminals 11A. Together with the aforementioned in-vehicle database 200, it constitutes the "storage unit" according to this embodiment. The external server 1000 is an example of an "external computer" in this embodiment.
[0107] As illustrated in Figure 3A, the in-vehicle computer 100 according to this embodiment comprises an estimation unit 111, a notification suggestion unit 112, and a notification control unit 113. Details of these elements will be described below. These elements are, for example, comprised of the control unit 101 of the in-vehicle computer 100.
[0108] <2. Estimation part> Figure 4 is a schematic diagram illustrating the target of estimation by the estimation unit 111. Figure 5 is a diagram illustrating the estimation method for confidence level C1. Figure 6 is a flowchart illustrating the estimation procedure for interest level C2. Figure 7 is a flowchart illustrating the calculation procedure for the bias index Rb that characterizes uniqueness C3. Figure 8A is a diagram illustrating the calculation procedure for the bias index Rb related to current bias. Figure 8B is a graph illustrating the relationship between the bias index Rb and confidence level C1.
[0109] Here, the estimation unit 111 estimates occupant characteristics C, which include one or more of the following: occupant U's driving ability C8, a dynamic state C5 that changes dynamically with respect to occupant U, and occupant U's purpose of movement C4, based on the detection signals of the terminal 11A as a detection device, the biometric information detection device 13, and the movement information detection device 14, respectively. Specifically, in this embodiment, the estimation unit 111 estimates the driving ability C8, the dynamic state C5, and the purpose of movement C4, respectively.
[0110] As illustrated in Figure 4, the occupant characteristics C are divided into dynamic characteristics Cd, which can change in real time, and static characteristics Cs, which can change over a longer period of time compared to the dynamic characteristics Cd.
[0111] Here, the dynamic characteristic Cd includes, in addition to the movement purpose C4 and dynamic state C5, the degree of trust C1 between the system and the occupant U and the degree of interest C2 of the occupant U to the proposal from the notification / proposal unit 112.
[0112] In contrast, static characteristics Cs include, in addition to the driving ability C8, preference characteristics C7 that characterize the occupant U's preferences and the uniqueness of occupant U compared to other user groups U' C3. Preference characteristics C7 are related to the occupant U's life stage and lifestyle.
[0113] As shown in Figure 4, reliability C1, interest C2, uniqueness C3, preference characteristics C7, and driving ability C8 constitute personal characteristics C0 that characterize the individual occupant U.
[0114] Note that the definitions of occupant characteristics C, dynamic characteristics Cd, static characteristics Cs, and personal characteristics C0 shown in Figure 4 are merely illustrative. The various characteristics constituting occupant characteristics C, dynamic characteristics Cd, static characteristics Cs, and personal characteristics C0 can be modified as appropriate.
[0115] The estimation unit 111 then takes the detection signals from the terminal 11A, the biometric information detection device 13, and the movement information detection device 14 as inputs, and estimates the occupant characteristics C as exemplified in Figure 4 based on a state estimation model M0 that outputs occupant characteristics C. It should be noted that estimating the reliability C1, interest level C2, uniqueness C3, and preference characteristics C7 among the characteristics exemplified in Figure 4 is not mandatory.
[0116] The state estimation model M0 according to this embodiment is a trained machine learning model. The storage 103 stores information that can identify these models, such as the coefficients of each node in the neural network. Note that different state estimation models M0 may be used for each type of characteristic that constitutes the occupant characteristics C. The second, third, and fourth state estimation models described later are also included in such state estimation models M0.
[0117] (2-1. Estimation of confidence level) The estimation unit 111 estimates the confidence level C1 with crew member U based on the crew member U's response to the notification information In. More specifically, the estimation unit 111 estimates the confidence level C1 of crew member U based on the crew member U's response to the proposal (service provision) from the notification proposal unit 112. Estimation based on the response can be performed based on the dialogue content I1. Note that the proposal here includes action proposals, custom proposals, and both.
[0118] The confidence level C1 estimated by the estimation unit 111 represents the level of confidence between the occupant U and the vehicle V as a mobile entity, the services provided by the occupant support system S, or the manufacturer brand of the vehicle V.
[0119] The term "services provided by the crew support system S" used here is used in a broad sense. In other words, the term "trust level C1 between crew member U and the services provided by the crew support system S" includes both crew member U's trust level C1 in the proposed content (contents and services provided) from the crew support system S, and crew member U's trust level C1 in the crew support system S itself.
[0120] Furthermore, the term "manufacturing brand" used here is used in a broad sense. The phrase "trust level C1 between occupant U and the manufacturing brand of vehicle V" includes both occupant U's trust level C1 with the manufacturer of vehicle V and occupant U's trust level C1 with the manufacturing brand.
[0121] Here, the estimation unit 111 calculates the degree of update of the individual characteristic C0 (for example, the number of updates). Then, the estimation unit 111 updates the confidence level C1 according to the calculated degree of update. Specifically, as illustrated in graph G1 of Figure 5, the estimation unit 111 sets a higher confidence level C1 the more updates are calculated.
[0122] Furthermore, the estimation unit 111 calculates the frequency or amount of modification of the proposal (content provided) via the terminal 11A. The estimation unit 111 then updates the confidence level C1 according to the calculated frequency or amount of modification. Specifically, as illustrated in graph G2 of Figure 5, the estimation unit 111 sets the confidence level C1 lower the higher the calculated frequency or the larger the amount of modification.
[0123] Furthermore, the estimation unit 111 calculates the frequency or number of times a proposal (content offered) is adopted via the terminal 11A. The estimation unit 111 then updates the confidence level C1 according to the calculated frequency or number of adoptions. Specifically, as illustrated in graph G3 of Figure 5, the estimation unit 111 sets a higher confidence level C1 the higher the calculated frequency or number of adoptions.
[0124] Furthermore, the estimation unit 111 aggregates the evaluations of the proposals (offerings) received by the terminal 11A. The estimation unit 111 then updates the confidence level C1 according to the aggregated evaluation results. Specifically, as illustrated in graph G4 of Figure 5, the estimation unit 111 sets a higher confidence level C1 the more highly the proposal is evaluated.
[0125] Furthermore, the estimation unit 111 calculates the cumulative operating period (valid period) since the terminal 11A activated the proposed function (service provision function). The estimation unit 111 then updates the confidence level C1 according to the length of the valid period. Specifically, as illustrated in graph G5 of Figure 5, the estimation unit 111 sets the confidence level C1 higher as the valid period lengthens.
[0126] Thus, the estimation unit 111 according to this embodiment calculates the confidence level C1 using five variables that characterize the crew member U's response to the proposal from the notification / proposal unit 112: a first variable that quantifies the degree of update of personal characteristics C0; a second variable that quantifies the frequency or amount of modification of the proposal; a third variable that quantifies the frequency or number of times the proposal is adopted; a fourth variable that quantifies the evaluation of the proposal; and a fifth variable that quantifies the cumulative operating period of the terminal 11A.
[0127] In detail, the storage device 103, which acts as a memory device, has a first state estimation model pre-stored in it, which takes the five variables as inputs and outputs a confidence score C1. The estimation unit 111 calculates the confidence score C1 by inputting the five variables into the first state estimation model.
[0128] The notification / proposal unit 112 then executes a proposal (service provision) as described below, based on the confidence level C1 calculated (updated) in this manner. The estimation unit 111 further updates the confidence level C1 of crew member U based on the crew member U's response to the proposal.
[0129] In this way, the notification suggestion unit 112 and the estimation unit 111 perform feedback control via confidence level C1. Through this feedback control, the suggestion from the notification suggestion unit 112 is optimized according to the confidence level C1 of the crew member U.
[0130] (2-2. Estimation of the level of interest) The estimation unit 111 estimates the level of interest C2 of occupant U based on the occupant U's response to the notification information In. More specifically, the estimation unit 111 estimates the level of interest C2 of occupant U based on the occupant U's response to the proposal from the notification proposal unit 112. This level of interest C2 may be considered as the level of interest in the notification proposal unit 112 or the occupant support system S itself. The proposal referred to here includes action proposals, custom proposals, and both.
[0131] Specifically, the estimation unit 111 according to this embodiment determines the level of interest C2 of the occupant U based on the stages of behavioral change of the occupant U, which are represented by multiple stages. Here, the stages of behavioral change are modeled by five stages, as is well known. The five stages include the "precontemplation stage," the "contemplation stage," the "preparation stage," the "action stage," and the "maintenance stage."
[0132] The "precontemplation stage" is the stage in which a person does not intend to change their "behavior" within a predetermined first period (e.g., six months). The "contemplation stage" is the stage in which a person intends to change their "behavior" within a predetermined third period (e.g., one month). The "action stage" is the stage in which a person intends to change their "behavior" within a second period shorter than the aforementioned first period (e.g., one month). The "action stage" is the stage in which a person has changed their "behavior" but has not yet reached the predetermined third period (e.g., six months). The "maintenance stage" is the stage in which a person has changed their "behavior" and has reached the aforementioned third period or longer.
[0133] In this embodiment, "action" includes "adoption of a proposal from the crew support system S." Furthermore, this embodiment utilizes four stages, excluding the "preparation phase."
[0134] More specifically, the estimation unit 111 is configured to determine the progress of the dialogue between terminal 11A and crew member U, and to update the level of interest C2 according to that progress. More specifically, the estimation unit 111 updates the level of interest C2 by referring to the confidence level C1 in addition to the progress.
[0135] As an example, as illustrated in step S101 of Figure 6, the estimation unit 111 first determines whether the suggestion function of the crew support system S (more specifically, the action suggestion function) is enabled. If the determination in step S101 is NO, the estimation unit 111 sets the crew member U's level of interest C2 to "precontemplation" in step S102 and returns the control process to step S101. The estimation unit 111 maintains the level of interest C2 at "precontemplation" until the suggestion function is enabled.
[0136] If the determination in step S101 is YES, the estimation unit 111 proceeds to step S103 of the control process. In step S103, the estimation unit 111 determines whether there is a history of acquiring personal characteristics C0 or whether the acquisition of new personal characteristics C0 has been completed. If the determination in step S103 is NO, the estimation unit 111 sets the interest level C2 of crew member U to "interest period" in step S104 and returns the control process to step S103. If there is no history of acquiring personal characteristics C0, the estimation unit 111 keeps the interest level C2 at "interest period" until the acquisition of new personal characteristics C0 is completed.
[0137] If the determination in step S103 is YES, the estimation unit 111 proceeds to step S105 of the control process. In step S105, the estimation unit 111 determines whether the elapsed time since the proposed function was activated (system validity period) has exceeded a predetermined first period, or whether the number of proposed options calculated based on the confidence level C1 has exceeded a predetermined first threshold. If the determination in step S105 is NO, the estimation unit 111 sets the crew member U's level of interest C2 to "execution period" in step S106 and returns the control process to step S105. The estimation unit 111 maintains the level of interest C2 at "execution period" until the system validity period exceeds the first period or the number of proposed options exceeds the first threshold.
[0138] Here, the system validity period is an example of the "cumulative operating period" in this embodiment. Furthermore, updating the level of interest C2 by referring to the number of proposed options is substantially equivalent to updating the level of interest C2 by referring to the confidence level C1, as will be described later.
[0139] If the determination in step S105 is YES, the estimation unit 111 proceeds to step S107 of the control process. In step S107, the estimation unit 111 determines whether the proposed function has been disabled. If the determination in step S107 is NO, the estimation unit 111 sets the interest level C2 of crew member U to the "maintenance phase" in step S108 and returns the control process to step S107. The estimation unit 111 maintains the interest level C2 in the "maintenance phase" until the proposed function is disabled.
[0140] If the determination in step S107 is YES, the estimation unit 111 sets the interest level C2 of occupant U to "incontemplation" in step S109 and terminates the control process shown in Figure 6. In this case, with the interest level C2 set to "incontemplation," the estimation unit 111 will execute the flow shown in Figure 6 again, starting from step S101.
[0141] (2-3. Estimation of Uniqueness) The estimation unit 111 estimates the bias index of occupant U based on at least one detection signal from the terminal 11A, the biometric information detection device 13, and the movement information detection device 14, which are detection devices. The bias index is an index that shows the behavioral tendencies of occupant U in relation to the driving scene of the vehicle V and in comparison to the behavior of others. Behavioral tendencies are one aspect of "uniqueness" as illustrated in Figure 4, and will be simply referred to as "uniqueness" below.
[0142] Uniqueness C3 represents the "habits (uniqueness)" of occupant U's behavior, which are considered when implementing various proposals from the notification proposal unit 112. For example, depending on the level of occupant U's uniqueness C3, the actions taken by occupant U in a particular driving scene will differ from those taken by other users in the same driving scene. The bias index is one example of an index that can be used to quantify such uniqueness C3.
[0143] More specifically, the estimation unit 111 estimates an index that shows cognitive biases, including present bias, as a bias index that indicates originality C3. Further specifically, the estimation unit 111 estimates an index that shows present bias, loss aversion bias, conformity bias, altruism bias, optimism bias, overconfidence bias, and anchoring effect, respectively.
[0144] Here, present bias is a cognitive bias related to short-term rewards, such as the shortest route in the immediate future or immediate safety risks. Loss aversion bias is a cognitive bias related to the Harding effect, such as anxiety about belonging to a minority group. Conformity bias is a cognitive bias related to conforming to the behavior of others, such as the degree to which one matches the speed of surrounding vehicles. Altruism bias is a cognitive bias related to altruistic behavior, such as the frequency of yielding to others. Optimism bias and overconfidence bias are cognitive biases related to optimism and confidence regarding risky behaviors, such as changes in the frequency of speed limit violations, sudden steering, and sudden braking. The anchoring effect is a cognitive bias that shows the influence of past behavior on present behavior.
[0145] Furthermore, the estimation unit 111 estimates the bias index Rb based on the contents of the in-vehicle database 200, which serves as an internal database. Specifically, the estimation unit 111 estimates the bias index corresponding to each cognitive bias based on the movement information I3, which shows the driving history of the occupant U.
[0146] For example, the estimation unit 111 according to this embodiment estimates the bias index Rb based on the difference between the travel route Tp proposed by the notification proposal unit 112 and the travel route Tp actually adopted by the occupant U. The estimation unit 111 also estimates the bias index Rb based on the difference between the estimated time required for the travel route Tp proposed by the notification proposal unit 112 and the actual time required. The former time required is estimated by the notification proposal unit 112 when proposing the travel route Tp.
[0147] Furthermore, in estimating the bias index Rb, the estimation unit 111 refers to the occupant characteristics C estimated by the estimation unit 111. The occupant characteristics C used in estimating the bias index Rb include, for example, confidence level C1 and mental / physical state C5.
[0148] Next, a specific example of a method for estimating the bias index Rb, which currently indicates bias, will be explained with reference to Figure 7. The process in Figure 7 illustrates a process that is performed based on the difference between the travel route Tp proposed by the notification proposal unit 112 and the travel route Tp actually adopted by the crew member U, but it can also be applied to other processes and other bias indexes Rb.
[0149] First, in step S301 of Figure 7, the estimation unit 111 obtains the crew action proposed by the notification proposal unit 112 and the actual crew action performed by crew member U. For example, the estimation unit 111 obtains the proposed shortest travel route Tp connecting the starting point Ps and the destination point Pg, and the actual travel route Tp adopted by crew member U in relation to that proposed route.
[0150] In the subsequent step S302, the estimation unit 111 obtains one or more parameters that characterize the proposed content. For example, the estimation unit 111 obtains the distance (X1), the number of right and left turns at intersections (X2), and the time required (X3) for the shortest travel route Tp obtained in step S301.
[0151] In the subsequent step S303, the estimation unit 111 obtains one or more parameters that characterize the actual occupant behavior. For example, the estimation unit 111 obtains the distance (Y1), the number of right and left turns at intersections (Y2), and the time taken (Y3) along the actual travel route Tp obtained in step S301.
[0152] In the subsequent step S304, the estimation unit 111 calculates the difference between the proposed occupant behavior (proposed content) and the actual occupant behavior through the ratio between one or more of the parameters. For example, the estimation unit 111 calculates an index that shows the difference between the shortest travel route Tp and the actual travel route Tp. As a further example, the estimation unit 111 calculates a first ratio (R1=Y1 / X1) showing the ratio of distance, a second ratio (R2=Y2 / X2) showing the ratio of the number of right and left turns, and a third ratio (R3=Y3 / X3) showing the ratio of required time. Instead of the first to third ratios, the difference between each parameter (e.g., Y1-X1) may be calculated. More generally, the estimation unit 111 may calculate a total of N ratios (the Nth ratio) or differences between parameters.
[0153] In the subsequent step S305, the estimation unit 111 acquires ancillary conditions associated with the occupant's actions. For example, the estimation unit 111 acquires traffic congestion information and traffic regulation information (Z) for the date and time the travel route Tp is adopted as ancillary conditions. These ancillary conditions are acquired, for example, from an external server 1000.
[0154] In the subsequent step S306, the estimation unit 111 acquires some or more of the occupant characteristics C, excluding the uniqueness C3, that it has estimated. For example, the estimation unit 111 acquires the confidence level C1 and the dynamic state C5.
[0155] In the subsequent step S307, the estimation unit 111 updates the frequency distribution (histogram) of the differences. For example, the estimation unit 111 reflects the values of the first ratio, second ratio, and third ratio calculated in step S304 into three types of histograms Gh, with the first ratio, second ratio, and third ratio on the horizontal axis, respectively (see Figure 8A). This updates each histogram Gh. Note that each histogram Gh in Figure 8A represents the frequency of occurrence of the first ratio, second ratio, and third ratio, respectively.
[0156] More specifically, in step S307, the estimation unit 111 refers to the congestion information and traffic restriction information obtained in step S305. Specifically, the estimation unit 111 reflects the ratios when congestion or traffic restrictions exist in each histogram Gh, while not reflecting the ratios when neither congestion nor traffic restrictions exist in each histogram Gh. In this way, a histogram Gh suitable for analyzing current bias is constructed.
[0157] More specifically, in step S307, the estimation unit 111 decides whether to include the movement information I3 used to estimate the bias index Rb, which currently indicates a bias, based on the dynamic state C5 acquired in step S306. Specifically, if the occupant U is feeling negative or unwell, the estimation unit 111 does not reflect the ratios based on the movement information I3 acquired in that case in the corresponding histogram Gh.
[0158] In the subsequent step S308, the estimation unit 111 estimates a bias index Rb representing the uniqueness C3 of crew member U based on the histogram Gh updated in step S307. For example, the estimation unit 111 estimates the bias index Rb of crew member U with respect to current bias.
[0159] For more details, the value of the bias index Rb corresponding to each cognitive bias, including current bias, is given based on one or more of the values of the first, second, and third ratios (hereinafter also referred to as "peak positions") at which each histogram Gh reaches its maximum value (peak).
[0160] Here, the bias index Rb may be estimated so that the peak position of each ratio contributes independently. For example, the estimation unit 111 estimates the bias index Rb to be larger the larger the peak position related to the first ratio (more specifically, the larger it is than 1), and independently of that estimation, it estimates the bias index Rb to be larger the larger the peak position related to the second ratio (more specifically, the larger it is than 1), and independently of that estimation, it estimates the bias index Rb to be larger the larger the position reached by the peak related to the third ratio (more specifically, the larger it is than 1).
[0161] Alternatively, the value of the bias index Rb may be estimated such that the peak positions of each ratio contribute to each other in relation to one another. For example, if the peak position related to the first ratio (Y1 / X1), the peak position related to the second ratio (Y2 / X2), and the peak position related to the third ratio (Y1 / X1) are all greater than 1, the estimation unit 111 estimates the bias index Rb to be larger the larger those peak positions are.
[0162] As in the latter configuration, when all three peak positions are large, it means that drivers tend to choose detours even though there is no time-saving effect. In this case, it can be concluded that "drivers tend to choose the shortest possible route at the moment." To quantify this tendency, the estimation unit 111 overestimates the current bias relatively when all three peak positions are large.
[0163] Alternatively, instead of the value at which each histogram Gh reaches its peak, the mean or median of each histogram may be used, or any other statistical indicator that characterizes the shape of each histogram Gh may be used. Hereafter, the position at which the peak is reached (peak position) and various statistical indicators will simply be referred to as "magnitude of each ratio," and will be denoted by the symbol Rp (see Figure 8A).
[0164] More specifically, in step S308, the estimation unit 111 adjusts the estimated value of the bias index Rb currently associated with the bias based on the confidence level C1 obtained in step S306. Specifically, when the confidence level C1 is high, the estimation unit 111 estimates the bias index Rb to be larger than when the confidence level C1 is low (see Figure 8B).
[0165] Furthermore, when estimating the bias index Rb, the estimation unit 111 refers to the difference ΔR between the magnitude Rp of each ratio (e.g., peak position) and a predetermined reference value. Based on the magnitude of this difference, the estimation unit 111 estimates the value of the bias index Rb.
[0166] Here, the estimation unit 111 performs estimation of a bias index Rb, which includes indicators currently related to bias, based on the contents stored in the external database 1010, so as to reflect the differences in the behavior of crew member U and other user group U'.
[0167] More specifically, the estimation unit 111 uses the magnitude Rp' of each ratio in other user groups U', stored in the external database 1010, as the reference value. As illustrated in graph G6 of Figure 8A, the estimation unit 111 estimates a larger value for the bias index Rb, which includes indicators related to the current bias, the larger the difference ΔR between the magnitude Rp of each ratio in crew member U and the magnitude Rp' of each ratio in other crew members.
[0168] The aforementioned reference value is not limited to the contents stored in the external database 1010. The reference value Rp' may be a fixed value stored in the in-vehicle database 200 in advance, or it may be the magnitude of each ratio based on the entire behavioral history of the occupant U who is the target of estimation. The magnitude of each ratio based on the entire behavioral history means that data excluded based on traffic congestion information, traffic regulation information, and dynamic state (mental and physical state) C5 is also taken into consideration.
[0169] (2-4. Estimation of Preference Characteristics) The estimation unit 111 estimates the occupant U's preference characteristics C7 based on the stored contents of the in-vehicle database 200 (particularly the dialogue content I1) (see Figure 4). The preference characteristics C7 include an index representing the occupant U's age in multiple stages, an index representing the occupant U's gender, an index representing the occupant U's hobbies, an index representing the occupant U's family structure, an index representing other characteristics of the occupant U, and an index representing the relationship between the occupant U and the vehicle V. The values of these indexes may be quantified.
[0170] Crew member U's age can be represented in six stages, for example, "infancy," "childhood," "youth," "adulthood," "middle age," and "old age." Crew member U's family structure can be represented in multiple stages, for example, "single," "nuclear family," and "multi-generational family."
[0171] Other characteristics of crew member U are represented by one or more flags, such as whether crew member U is a student or not, or whether crew member U is pregnant or raising children.
[0172] An index representing the relationship between occupant U and vehicle V consists of, for example, one or more of the following: frequency of use of vehicle V, duration of use of vehicle V, distance traveled by vehicle V, and location information of vehicle V. These indexes may be further classified by day of the week, time of day, day of the week, or season. These indexes can be calculated based on travel information I3.
[0173] The estimation unit 111 estimates the occupant U's preference characteristics C7 based on at least one of the following: the content of the conversation with terminal 11A I1, biometric information I2, and mobility information I3. The information referenced may differ for each type of preference characteristic C7. For example, the occupant U's age can be estimated based on the response to the first questionnaire A1. On the other hand, an index representing the relationship between occupant U and vehicle V can be estimated, for example, based on mobility information I3 as described above.
[0174] The preference characteristics C7 of crew member U can also be estimated based on crew member U's body image. For example, the estimation unit 111 extracts a face image of crew member U based on the body image of crew member U. The estimation unit 111 then estimates the gender, age, etc., of crew member U based on the face image of crew member U. In other words, preference characteristics C7 can be estimated based on biometric information I2.
[0175] In detail, the storage device 103, which functions as a memory device, has a second state estimation model pre-stored in it. This model takes the detection signal corresponding to at least one of the following as input: the dialogue content I1 with the terminal 11A, the biometric information I2, and the movement information I3, and outputs the values of each index that constitutes the preference characteristic C7. The estimation unit 111 calculates the value of the preference characteristic C7 by inputting the at least one piece of information into the second state estimation model.
[0176] (2-5. Estimation of the purpose of movement) The estimation unit 111 obtains the purpose of travel to vehicle V C4 based on at least one of the following: dialogue content I1, biometric information I2, and movement information I3 (see Figure 4). When the moving object is vehicle V, the term "purpose of travel" C4 may be read as "purpose of boarding."
[0177] The estimation unit 111 classifies the purpose of travel C4 into one of several categories. These categories include one or more of the following: sightseeing, driving, visiting family, funerals, celebrations, business, daily life, and other purposes. The degree of each of these categories may also be quantified.
[0178] More specifically, the estimation unit 111 refers to the response to the second questionnaire A2 as dialogue content I1, and estimates the purpose of crew member U's travel C4 based on that response. For example, if the response to the second questionnaire A2 includes the fact that crew member U intends to travel to a specific tourist destination, the estimation unit 111 classifies the purpose of travel C4 as tourism. On the other hand, if the response to the second questionnaire A2 includes the fact that the purpose of travel is to a specific wedding venue, the estimation unit 111 classifies the purpose of travel C4 as a celebratory event.
[0179] In addition to, or instead of, referring to the responses to the second questionnaire A2, the estimation unit 111 can estimate the purpose of crew member U's movement C4 based on the crew member U's body image as biometric information I2. For example, the estimation unit 111 estimates the crew member U's clothing based on the crew member U's body image. The estimation unit 111 then performs a classification of the purpose of movement C4 according to the estimated clothing.
[0180] To give a further example, if crew member U is wearing a business suit, the estimation unit 111 can classify the purpose of travel C4 as business. If crew member U is wearing a tie, the estimation unit 111 can classify the purpose of travel C4 as business, funeral, or celebration based on the pattern and color of the tie.
[0181] Furthermore, in addition to, or instead of, referring to the responses to the second questionnaire A2, the estimation unit 111 can estimate the purpose of crew member U's movement C4 based on the crew member U's voice data as biometric information I2. This estimation can be performed in combination with estimation based on body images.
[0182] In addition, the estimation unit 111 can estimate the purpose of travel C4 based on the time flag of the vehicle V. For example, if the vehicle V is used on a weekday, the estimation unit 111 determines that the purpose of travel C4 is likely to be business or daily life, and if the vehicle V is used on a Saturday or Sunday, it determines that the purpose of travel C4 is likely to be for a purpose other than business or daily life.
[0183] The estimation unit 111 can improve the accuracy of estimating the movement purpose C4 by combining the judgment based on the time flag with the judgment based on the second questionnaire A2, body image, and voice data.
[0184] In addition, the estimation unit 111 can estimate the purpose of travel C4 based on the vehicle V's usage flag. For example, if the vehicle V is a rental car, the estimation unit 111 can determine that the purpose of travel C4 is likely to be for a purpose other than business or daily life.
[0185] The estimation unit 111 can improve the accuracy of estimating the purpose of movement C4 by combining the judgment based on the purpose flag with the judgment based on the second questionnaire A2, body image, voice data, and time flag.
[0186] The usage flag can be used in the response items of the first questionnaire A1. The estimation unit 111 determines whether vehicle V is a shared car or a rental car based on the usage flag. If it is determined to be a shared car or a rental car, the estimation unit 111 reduces the number of response items in the first questionnaire A1 compared to when it is not determined to be a shared car or a rental car. More specifically, if it is determined that vehicle V is a shared car or a rental car, the estimation unit 111 reduces the number of response items other than the usage flag in the first questionnaire A1 compared to when it is not determined to be a shared car or a rental car.
[0187] In detail, the storage device 103, which functions as a memory device, has a third state estimation model pre-stored in it. This model takes the detection signal corresponding to at least one of the following as input: the content of the interaction with the terminal 11A I1, the biometric information I2, and the movement information I3, and outputs the degree of each category that constitutes the movement purpose C4. The estimation unit 111 calculates a category value indicating the movement purpose C4 by inputting at least one of the aforementioned pieces of information into the third state estimation model.
[0188] (2-6. Estimation of Dynamic State) The estimation unit 111 estimates the dynamic state C5 of the occupant U based on at least one of the dialogue content I1, biometric information I2, and movement information I3 (see Figure 4). The dynamic state C5 estimated by the estimation unit 111 may include the occupant U's physical state in addition to the occupant U's emotions as described above. As illustrated in Figure 4, the occupant U's physical state may be the occupant U's physical condition. In this embodiment, the dynamic state C5 can be rephrased as "mental and physical state" as described above.
[0189] The estimation unit 111 classifies the occupant U's dynamic state C5 into one or more categories. The dynamic state C5 includes multiple categories related to the occupant U's emotional and physical state.
[0190] More specifically, the estimation unit 111 classifies the crew member U's emotions and physical state into one of several categories. These categories include one or more of the following: "normal," "joy (happiness)," "anger," "sadness," "fun," "pleasure," "displeasure," "active (awakened)," "inactive (sleep)," "excitement," "calm," "depression," "peace of mind," "anxiety," "fulfillment," "emptiness," "disappointment," "fatigue," "weariness," and "exhaustion." The state estimation model M0 used for estimation may differ between categories related to emotions and categories related to physical state.
[0191] More specifically, the estimation unit 111 estimates the dynamic state C5 of crew member U based on the crew member U's body image as biological information I2. For example, the estimation unit 111 extracts a face image of crew member U based on the crew member U's body image. The estimation unit 111 then estimates the dynamic state C5 of crew member U based on the facial expression shown in the crew member U's face image.
[0192] Furthermore, in addition to, or instead of, the body image of crew member U, the estimation unit 111 can estimate the dynamic state C5 of crew member U based on the voice data of crew member U as biometric information I2. This estimation can be performed in combination with estimation based on the body image. Moreover, the estimation unit 111 can also estimate the dynamic state C5 of crew member U based on other biometric information I2, such as heart rate.
[0193] In addition, the estimation unit 111 can estimate the emotions of the occupants U based on the purpose of travel C4 of the vehicle V. For example, if the purpose of travel C4 is a funeral, the estimation unit 111 can determine that the emotion is likely to be classified as "sadness." Also, if the purpose of travel C4 is sightseeing, the estimation unit 111 can determine that the emotion is likely to be classified as "joy" or "happiness."
[0194] The estimation unit 111 can improve the accuracy of estimating the dynamic state C5 by combining the judgment based on the movement purpose C4 with the judgment based on the body image and voice data.
[0195] In addition, the estimation unit 111 can estimate the dynamic state C5 based on the time flag of the vehicle V. For example, if the vehicle V is used in the early morning or late at night, the estimation unit 111 determines that there is a high probability that the occupant U is feeling sleepy.
[0196] The estimation unit 111 can improve the accuracy of estimating the dynamic state C5 by combining decisions based on time flags with decisions based on body images, audio data, and movement purpose C4.
[0197] In detail, the storage device 103, which functions as a memory device, pre-stores a fourth state estimation model that takes the detection signal corresponding to at least one of the following as input: the content of the interaction with the terminal 11A I1, the biometric information I2, and the movement information I3, and outputs the degree of each category that constitutes the dynamic state C5. The estimation unit 111 calculates a category value indicating the dynamic state C5 by inputting at least one of the above information into the fourth state estimation model. The fourth state estimation model may be configured so that the detection signal is indirectly input by taking the degree of the movement purpose C4 as input.
[0198] (2-7. Estimation of driving ability) The estimation unit 111 estimates the occupant U's driving ability C8 based on at least one of the following: dialogue content I1, biometric information I2, and movement information I3 (see Figure 4). The driving ability C8 is indexed by either a quantified value of the driving ability C8, or a category value when the driving ability C8 is classified into one of several categories.
[0199] As shown in Figure 4, the driving ability C8 according to this embodiment includes a combination of the occupant U's cognitive ability C81 and the same occupant U's driving skills C82. Cognitive ability C81 and driving skills C82 are each individually indexed. The estimation of driving ability C8 can be applied, for example, to a driver U1 as an occupant U.
[0200] Here, cognitive ability C81 indicates the degree of cognitive function. The degree of cognitive function includes, for example, one or more of the crew member U's attention level and memory level.
[0201] The estimation unit 111 can estimate cognitive ability C81 based on the dialogue content I1. For example, the estimation unit 111 can estimate cognitive ability C81 based on the content of notification information In from terminal 11A. In this case, the estimation unit 111 estimates a lower cognitive ability C81 the more frequently notification information In, which corresponds to a warning, is issued.
[0202] The estimation unit 111 can also estimate cognitive ability C81 based on dialogue content I1 and movement information I3. For example, the estimation unit 111 can estimate cognitive ability C81 by comparing the content of the action suggestion proposed as notification information In with the content of the action actually taken by the occupant U. In this case, the greater the difference between the content of the action suggestion and the content of the action actually taken, the lower the cognitive ability C81 will be estimated by the estimation unit 111. The content of the action actually taken can be determined, for example, based on movement information I3.
[0203] Furthermore, the latter estimation becomes even more favorable when combined with confidence level C1 and interest level C2. For example, if confidence level C1 is below a certain level (the proposal is not trusted), or if interest level C2 is in the "precontemplation stage" (there is no interest in the proposal), the system may be configured not to perform estimation that takes into account the difference between the content of the proposed action and the content of the action actually taken.
[0204] The estimation unit 111 can estimate cognitive ability C81 based on the dialogue content I1. For example, the estimation unit 111 can estimate cognitive ability C81 based on the content of notification information In from terminal 11A. In this case, the estimation unit 111 estimates a lower cognitive ability C81 the more frequently notification information In, which corresponds to a warning, is issued.
[0205] The estimation unit 111 may refer to biometric information I2 when estimating cognitive ability C81. For example, the estimation unit 111 may estimate a lower cognitive ability C81 the older the driver U1 is.
[0206] In contrast, driving skill C82 indicates the degree of skill level in driving operations. The skill level in driving operations includes one or more of the following, for example, the smoothness of steering operations and the clarity of acceleration and deceleration in the vehicle V (for example, an indicator showing whether clear acceleration and deceleration are performed).
[0207] The estimation unit 111 can estimate driving skills C82 based on movement information I3. For example, the estimation unit 111 can estimate driving skills C82 based on the vehicle V's driving trajectory and operation history stored in the in-vehicle database 200 as movement information I3. To illustrate further, the estimation unit 111 may determine the smoothness of steering operations and the clarity of acceleration and deceleration in the vehicle V based on the vehicle V's driving trajectory and operation history, and estimate a higher driving skill C82 the smoother the steering operations are, and set a higher driving skill C82 the clearer the acceleration and deceleration are.
[0208] <3.Notification proposal section> Figure 9 is a schematic diagram illustrating the relationship between occupant characteristics C and the object of judgment by the notification suggestion unit 112. Figure 10 is a schematic diagram illustrating the content of judgment by the notification suggestion unit 112. Figure 11 is a graph illustrating the relationship between the level of confidence C1 and the level of the frequency of the suggestion timing J3. Figure 12 is a graph illustrating the relationship between the level of cognitive ability C81 and the level of notification frequency by each notification means constituting the notification device 11.
[0209] The notification proposal unit 112 according to this embodiment is composed of a trained notification proposal model M1 (see Figure 3A). The notification proposal model M1 may be a neural network or a rule-based model. It is not mandatory to configure the notification proposal unit 112 with the notification proposal model M1. In addition, multiple models may be prepared depending on the type of object controlled by the notification proposal unit 112.
[0210] (3-1.Basic concept) The notification suggestion unit 112 has a function to suggest crew actions as notification information In to crew member U (a function to make action suggestions) and a function to suggest customization of notification information In to crew member U (a function to make custom suggestions).
[0211] In relation to the latter function, as illustrated in Figure 9, the notification suggestion unit 112 determines the timing J3 for suggesting customization of notification information In and the content J2 of the suggested customization based on the estimation result of the estimation unit 111, and then executes the customization suggestion to the crew member U based on the determination result. (Figure 3B shows an example of notification information In).
[0212] More specifically, the notification proposal unit 112 determines, based on the combination of driving capability C8, dynamic state C5, and movement purpose C4, the customization proposal content J2 and the customization proposal timing J3 for each type J1 of notification information In that is subject to customization proposal.
[0213] More specifically, the notification proposal unit 112 executes the proposal by referring to one or more of the following in addition to the combination of driving ability C8, dynamic state C5, and purpose of movement C4: reliability C1, level of interest C2, uniqueness as a behavioral tendency C3, preference characteristics C7, cognitive ability C81, and driving skills C82.
[0214] As illustrated in Figure 9, the type J1 of notification information In that is subject to customization proposals includes notification information related to driving assistance for vehicle V. The notification information In related to driving assistance for vehicle V is divided into three parts: first notification information In1 related to driving assistance corresponding to the traffic conditions around vehicle V, second notification information In2 related to driving assistance corresponding to the dynamic state C5 of driver U1, and third notification information In3 related to driving assistance corresponding to the purpose of travel C4.
[0215] The first notification information In1, the second notification information In2, and the third notification information In3 each correspond to notification information In that proposes a crew action. Notifying crew member U of the first notification information In1, the second notification information In2, and the third notification information In3 corresponds to the notification proposal unit 112 proposing a crew action.
[0216] Of these, the first notification information In1 is notification information In for proposing to occupant U the act of moving to a predetermined destination Lp as an occupant action. The act of moving to a predetermined destination Lp performs at least one of the following: proposing the destination Lp and proposing the travel route Tp to said destination Lp. The first notification information In1 is notified by visual information, such as images and messages to assist in turning right or left, and images and messages to assist in parking operations, taking into account congestion information and traffic regulation information around the vehicle V.
[0217] Furthermore, the second notification information In2 is notification information In for suggesting crew actions in a broad sense. The phrase "suggesting crew actions in a broad sense" includes providing support corresponding to the dynamic state C5 of the crew member U, such as encouraging a change from an "inactive (sleeping)" state to an "active (awake)" state, or encouraging a change from an "excited" state to a "calm" state. The second notification information In2 includes, for example, one or more combinations of visual information represented by a warning display, auditory information represented by an alert sound, and tactile information represented by vibration.
[0218] Furthermore, the third notification information In3 is notification information In intended to propose a broad range of travel actions to the occupant U as an occupant action. The phrase "proposal of a broad range of travel actions" includes, in addition to proposing a travel route Tp, presenting stopping spots Pa or passing spots Pb for the vehicle V located along the travel route Tp (see Figure 3B). Stopping spots Pa indicate spots where the vehicle V may stop as needed, such as tourist spots, commercial facilities, public facilities, and parking lots. Passing spots Pb indicate spots where the vehicle V may pass as needed, such as tourist spots, landmarks, buildings or objects that serve as markers for heading towards a predetermined destination Lp. The third notification information In3 is communicated through visual information, such as images and messages to assist with right and left turns, and images and messages to assist with parking operations.
[0219] Note that the classification of notification information In shown in Figure 9 is merely illustrative. For example, since the third notification information In3 is related to movement, just like the first notification information In1, it may be considered an aspect of the first notification information In1.
[0220] Furthermore, the proposed customization for notification information In (J2) includes a proposal for customization relating to one or more of the following: the notification content of notification information In, the notification means of notification information In, and the notification order of notification information In when multiple notifications of notification information In coexist.
[0221] Here, customization of notification content includes setting the content of the notification information In itself and setting the amount of information in the notification information In. Increasing the amount of information in the notification information In allows you to increase the number of characters in the text notified as visual information, increase the number of images notified as visual information, or make the displayed content in the images more detailed. Setting the amount of information includes setting the notification frequency and number of notifications for stop spot Pa or passing spot Pb in the third notification information In3. In addition, increasing the amount of information in the notification information In allows you to increase the amount of audio message emitted as auditory information or make the vibration waveform emitted as tactile information more complex.
[0222] Customization of notification content includes changing the notification format of Notification Information In. Changing the notification format of Notification Information In includes changing the font size, layout, font type, or color of the text notified as visual information, or changing the size, layout, shape, or color of objects in images notified as visual information. Changing the notification format of Notification Information In also includes changing the size, layout, shape, or whether or not icons accompanying the text and images as visual information are displayed, changing the volume of the sound notified as auditory information, or changing the amplitude of vibrations generated as tactile information.
[0223] Furthermore, customization of the notification means includes selecting at least one of the following: a visual means for notifying visual information, an auditory means for notifying auditory information, and a tactile means for notifying tactile information. The visual means is comprised of, for example, a display 15. The auditory means is comprised of, for example, a speaker 16. The tactile means is comprised of, for example, a vibration device 11B.
[0224] Furthermore, the customization of priorities includes setting the order in which crew member U should be notified when two or more of the aforementioned first notification information In1, second notification information In2, and third notification information In3 are given simultaneously. For example, in the initial state, second notification information In2, which relates to the physical and mental state of crew member U, is set to be notified with higher priority than first notification information In1 and third notification information In3.
[0225] Furthermore, the customization proposal timing J3 for notification information In indicates the timing at which customization of notification information In is proposed. The proposal timing J3 can be determined based on the occupant characteristics C. It should be noted that the proposal timing J3 is the timing at which the "customization proposal" is made, and is determined independently of the timing at which notification information In is actually notified.
[0226] Furthermore, setting the customization suggestion timing J3 includes setting whether or not to reflect the suggestions from the notification suggestion unit 112 in the notification information In (automatic setting), regardless of whether or not the crew member U accepts them, and how often.
[0227] (3-2. Processing related to reliability) As illustrated in Figures 9 and 10, the notification suggestion unit 112 controls the customization suggestion timing J3 according to the confidence level C1.
[0228] In detail, the notification and proposal unit 112 sets the timing of customization proposals to a lower frequency when the reliability level C1 is high compared to when the reliability level C1 is low (see graph G11 in Figure 11). In conjunction with this, the notification and proposal unit 112 sets the process of automatically reflecting the proposed content J2 without accepting or rejecting the proposal by the crew member U (automatic setting) to a higher frequency.
[0229] Furthermore, it is preferable that the suggested content J2 subject to automatic configuration be settings for items other than the content of the suggested information itself (e.g., the amount of information), such as the color of the notification information In.
[0230] Specifically, if the confidence level C1 is below a predetermined threshold, the notification suggestion unit 112 will execute custom suggestions relatively frequently and will not perform the automatic settings. On the other hand, if the confidence level C1 is above the threshold, the notification suggestion unit 112 will execute custom suggestions relatively infrequently and will perform the automatic settings.
[0231] Alternatively, the notification suggestion unit 112 may set the execution frequency of custom suggestions to be higher and the execution frequency of automatic settings to be lower as the confidence level C1 decreases, while setting the execution frequency of custom suggestions to be lower and the execution frequency of automatic settings to be higher as the confidence level C1 increases.
[0232] (3-3. Processing related to the level of interest) As illustrated in Figures 9 and 10, the notification suggestion unit 112 controls the suggestion timing J3 for customization according to the level of interest C2. For example, the notification suggestion unit 112 controls the suggestion timing J3 according to the stage of behavioral change.
[0233] In more detail, the notification suggestion unit 112 sets the customization suggestion timing J3 to a higher frequency when the level of interest C2 is high, compared to when the level of interest C2 is low.
[0234] More specifically, the notification suggestion unit 112 does not execute custom suggestions (custom suggestion: OFF) if interest level C2 is in the "precontemplation" stage. The notification suggestion unit 112 allows the execution of custom suggestions (custom suggestion: ON) if interest level C2 is in a stage other than the "precontemplation" stage.
[0235] (3-4. Processing related to uniqueness) As illustrated in Figures 9 and 10, the notification suggestion unit 112 controls the suggested customization content J2 for each type J1 of notification information that is subject to customization, according to its uniqueness C3.
[0236] More specifically, the notification suggestion unit 112 changes the first notification information In1 or the third notification information In3 based on the value of the bias index Rb. The items to be changed include at least one of the suggested content of the vehicle V's travel route Tp and the suggested content of the occupant U's driving operations.
[0237] For example, consider the case where the bias index Rb currently relates to the bias is referenced. If the value of the bias index Rb is large (when short-term rewards are emphasized), the notification proposal unit 112 proposes a travel route Tp that may result in the shortest travel time, although there is a risk of delay. On the other hand, if the value of the bias index Rb is small (when long-term rewards are emphasized), the notification proposal unit 112 proposes a travel route Tp that has the shortest expected travel time.
[0238] Similarly, if the value of the bias index Rb is large, the notification suggestion unit 112 suggests driving operations, including lane changes, merging, right turns at intersections, and general right / left turns, in order to maximize the safety risk within a predetermined tolerance range while keeping the safety risk within that tolerance range (for example, by setting the safety risk to the upper limit of the tolerance range). On the other hand, if the value of the bias index Rb is small, the notification suggestion unit 112 suggests driving operations, including lane changes, merging, and right turns at intersections, in order to minimize the safety risk (for example, by setting the safety risk to the lower limit of the tolerance range).
[0239] (3-5. Processing related to the purpose of movement) As illustrated in Figures 9 and 10, the notification suggestion unit 112 controls, according to the purpose of movement C4, the suggested content J2 and the timing of the customization suggestion J3 for each type J1 of notification information that is the subject of the customization suggestion.
[0240] The notification suggestion unit 112 determines that it is customization suggestion timing J3 when a travel purpose C4 is entered or estimated in terminal 11A. For example, the notification suggestion unit 112 activates the travel navigation function of terminal 11A, and when a destination Lp or travel purpose C4 is entered into terminal 11A after activation, it executes custom suggestions related to the first notification information In1 and the third notification information In3, respectively, according to the content of the input.
[0241] The notification suggestion unit 112 proposes customizations related to the first notification information In1, which provides a sense of security according to traffic conditions such as congestion information and traffic regulation information, including a suggestion of a travel route Tp that matches the current traffic conditions, and a suggestion of driving operations that are also suitable for the current traffic conditions.
[0242] The notification suggestion unit 112 performs the following actions as third notification information In3 corresponding to the travel purpose C4: for example, it suggests a travel route Tp that matches the current destination Lp, and notifies the vehicle V of a stopping spot Pa or passing spot Pb located along that travel route Tp.
[0243] At that time, the notification proposal unit 112 executes the following as proposal content J2 corresponding to the first notification information In1 and the third notification information In3: a proposal for customization regarding the amount of information and a proposal for customization regarding the notification manner.
[0244] The former proposal includes whether or not to be notified of stopping spots Pa or passing spots Pb, and the number of spots to be notified (notification amount). The latter proposal includes the degree of navigation along the travel path Tp, the color of the notification information In, and whether or not to display the navigation agent (a character displayed on the screen). The stronger the degree of navigation, the more detailed the indicators (e.g., arrows) showing the direction of travel along the travel path Tp will be displayed.
[0245] For example, the notification suggestion unit 112 notifies the crew member U of the necessity of notification of a stopping spot Pa as a customized suggestion J2, based on the category of travel purpose C4 classified by the estimation unit 111.
[0246] For example, if the purpose of travel C4 is classified as "daily life," including round-trip travel between home and work, setting home and work as stop spots Pa may be inconvenient for some passengers U. By allowing passengers U to decide whether or not they need to be notified of stop spots Pa, it becomes possible to provide notification information In that is appropriate for each passenger U.
[0247] (3-6. Processing related to dynamic states) As illustrated in Figures 9 and 10, the notification suggestion unit 112 controls the suggested content J2 according to the type J1 of the notification information to be suggested for customization, in accordance with the dynamic state C5.
[0248] At that time, the notification proposal unit 112 executes a proposal for customization regarding the notification means as a proposal content J2 corresponding to the second notification information In2.
[0249] More specifically, the notification suggestion unit 112 proposes customization by selecting a notification method according to the dynamic state C5. More specifically, the notification suggestion unit 112 proposes customization by notifying the dynamic state C5 when it is classified into a specific category (when the second notification information In2 is notified) using a different notification method than when it is classified into another category (when the first notification information In1 or third notification information In3 is notified).
[0250] Specifically, the notification suggestion unit 112 suggests to crew member U whether it is necessary to notify the second notification information In2 by auditory means or by tactile means, in lieu of or in addition to visual means, when the dynamic state C5 is classified as either an "inactive (sleep)" state or an "excited" state.
[0251] (3-7. Processing related to preference characteristics) As illustrated in Figures 9 and 10, the notification suggestion unit 112 controls the suggested content J2 according to the type J1 of the notification information In that is the subject of the customization suggestion, according to the preference characteristics C7.
[0252] For example, the notification suggestion unit 112 proposes customization of the amount of information and notification method for each notification information In, based on the preference characteristics C7 estimated by the estimation unit 111. For example, the left side of Figure 10 shows that occupant U prefers simpler information. In this case, the notification suggestion unit 112 proposes to occupant U to reduce the amount of information in notification information In and to notify notification information In in a simpler manner (for example, omitting the display of icons and agents, or displaying notification information In as monochrome text).
[0253] Furthermore, the right side of Figure 10 shows that crew member U prefers more complex information. In this case, the notification suggestion unit 112 suggests to crew member U that the amount of information in notification information In be increased and that notification information In be presented in a more elaborate manner (for example, by actively displaying icons and agents, or by displaying notification information In in multi-colored text).
[0254] (3-8. Processing related to driving ability) As illustrated in Figures 9 and 10, the notification suggestion unit 112 controls, according to the driving ability C8, the suggested content J2 and the timing of the customization suggestion J3 for each type J1 of notification information In that is the subject of the customization suggestion. The following explanation is common to both cognitive ability C81 and driving skill C82.
[0255] The notification and proposal unit 112 determines that the timing for suggesting customization is J3 when the driving capability C8 changes. Specifically, when the vehicle V stops after it has been determined that the driving capability C8 has increased or decreased by more than a predetermined amount, the notification and proposal unit 112 executes custom suggestions related to the first notification information In1 and the third notification information In3, respectively.
[0256] For example, the notification suggestion unit 112 suggests customization of the notification means in the corresponding notification information In based on the driving ability C8 estimated by the estimation unit 111. More specifically, the notification suggestion unit 112 adjusts the combination of visual means, auditory means, and tactile means, as well as the frequency of use of each means, according to the level (superiority) of the driving ability C8.
[0257] More specifically, as illustrated in Figure 12, the notification suggestion unit 112 proposes to the crew member U, as a customized suggestion J2, that if, for example, cognitive ability C81 is low, the frequency of notifications G121 by visual means be reduced and the frequency of notifications G122 by auditory or tactile means be increased compared to when cognitive ability C81 is high.
[0258] (3-9. Specific examples of processing related to customization proposals) Figure 13 is a flowchart illustrating the procedure for a custom proposal. First, in step S201 of Figure 13, the estimation unit 111 estimates the occupant characteristics C.
[0259] In the following step S202, the notification and proposal unit 112 determines, based on the occupant characteristics C, whether or not it is time to execute a custom proposal (proposal timing). If the determination in step S202 is YES, the notification and proposal unit 112 proceeds to step S203. On the other hand, if the determination in step S202 is NO, the notification and proposal unit 112 returns the control process to step S201.
[0260] Furthermore, the determination in step S202 is not limited to a judgment based on occupant characteristics C. The notification suggestion unit 112 may determine that the suggestion timing J3 applies when occupant U requests customization of notification information In (when customization is manually entered), or when it is the first time riding in a vehicle (when switching to a new vehicle, or when riding in a rental or shared car).
[0261] In the subsequent step S203, the notification proposal unit 112 determines the proposal content J2 for each type J1 of notification information In. This determination is performed based on the occupant characteristics C, as described above.
[0262] In the subsequent step S204, the notification suggestion unit 112 notifies the occupant U of the custom suggestion via the notification control unit 113 and the display 15. This notification includes the details of the suggestion content J2 described above.
[0263] In the subsequent step S205, the notification proposal unit 112 accepts the acceptance or rejection of the custom proposal via the terminal 11A, which acts as a detection device. If the determination in step S205 is YES, the notification proposal unit 112 proceeds to step S206. On the other hand, if the determination in step S205 is NO, the notification proposal unit 112 returns the control process to step S201.
[0264] In the subsequent step S206, the notification suggestion unit 112 reflects the suggested content J2 determined in step S203 into the notification information In. From this point onward, the notification device 11 will execute notifications of various notification information In, including action suggestions, with the suggested content J2 reflected.
[0265] <4. Notification Control Unit> The notification control unit 113 notifies the occupant U of the notification information In and the customization proposal determined by the notification proposal unit 112 via the notification device 11, respectively. At that time, the notification device 11 executes the notification of the notification information In via the notification device 11 in a state where the content of the customization proposal is reflected, according to whether the occupant U accepts or rejects the customization proposal.
[0266] According to the response of the occupant U to the notification, the action proposal via the notification information In is accepted or rejected. Along with this, factors that influence the proposal from the notification proposal unit 112, such as the reliability C1, are updated each time.
[0267] <5. Effects and the Like> As described above, as exemplified in FIG. 9 and FIG. 10, the notification proposal unit 112 not only determines the customization proposal content J2 related to the notification information In, but also executes the customization proposal at a timing based on the occupant characteristic C.
[0268] By doing so, it is possible not only to realize information notification in a mode adapted to the occupant U, but also to allow the occupant U himself / herself to decide whether to accept or reject it. By reflecting the occupant U's own intention, it is possible to reduce the confusion and discomfort given to the occupant U accompanying the customization.
[0269] At that time, by causing the apparatus to determine the timing itself for proposing customization based on the occupant characteristic C, it is possible to further reduce the confusion and discomfort given to the occupant U accompanying the customization.
[0270] Further, as exemplified in FIG. 9 and FIG. 10, the notification proposal unit 112 can propose customization suitable for the occupant U for each type J1 of the notification information In. Thereby, compared with a configuration in which customization is uniformly applied to all information, it is possible to realize information notification in a mode more adapted to the occupant U.
[0271] Further, as illustrated in FIG. 9, by narrowing down the types of the customization proposal content J2, it is possible to save the effort of the occupant U compared with a method having an excessively high degree of freedom. This is advantageous in reducing the annoyance given to the occupant U.
[0272] Further, as illustrated in FIG. 10, the notification proposing unit 112 determines the customization proposal timing J3 based on a change in the driving capability C8. In this way, the customization can be proposed at a more appropriate timing.
[0273] Further, in general, it is considered that the mode of information notification adapted to the occupant U, such as the level of detail at which navigation information is notified, may vary according to the cognitive ability C81 or the driving skill C82 of the occupant U. The configuration according to the present embodiment suitably acts on the driving capability C8 as illustrated in FIG. 4 and the like.
[0274] Further, in general, it is considered that as the cognitive ability C81 of the occupant decreases, the attention of the occupant U decreases. In this case, if visual information is heavily used, the attention of the occupant U is considered to be distracted as the line of sight of the occupant U moves. Accordingly, as illustrated in FIG. 10, the movement of the line of sight of the occupant U can be suppressed by increasing the notification frequency by the auditory means or the tactile means according to the level of the cognitive ability.
[0275] Further, as illustrated in FIG. 9 and FIG. 10, the notification information In corresponding to the dynamic state C5 can be included in the customization proposal targets. This makes it possible to notify information corresponding to the dynamic state C5 of the occupant U.
[0276] Further, as illustrated in FIG. 4, FIG. 9 and FIG. 10, as an example of the dynamic state C5, information corresponding to the mental and physical state (emotion or physical state) of the occupant U can be included in the customization proposal targets. This makes it possible to notify information corresponding to the mind and body of the occupant U.
[0277] Furthermore, as illustrated in Figure 10, the notification suggestion unit 112 determines the suggestion timing J3 when the movement purpose C4 is input or estimated. This allows for suggestion of customization at a more appropriate timing.
[0278] Furthermore, it is conceivable to present the occupant U with stop spots Pa or passing spots Pb, such as the locations of landmarks and tourist spots, as notification information In. However, depending on the occupant U's purpose of travel C4, presenting such spots may be bothersome to the occupant U. Therefore, a configuration that decides not to display such spots based on the purpose of travel C4 is conceivable, but if the device performs such processing autonomously, it may confuse the occupant U.
[0279] In contrast, as illustrated in Figure 10, the confusion caused to the crew member U can be reduced by having them confirm whether or not they need to be notified of a stopping spot Pa or passing spot Pb.
[0280] Furthermore, if the level of interest C2 is low (for example, the "precontemplation stage"), there is a possibility that the customer will not even be interested in the suggestion of customization. By varying the frequency of the suggestion timing J3 according to the level of interest C2, it is possible to suggest customization at a more appropriate time.
[0281] Furthermore, if the confidence level C1 is high, it is considered acceptable to the crew member U even if the customization is automatically reflected without questioning whether or not to accept it. By varying the frequency of the suggestion timing J3 according to the confidence level C1, customization can be suggested at a more appropriate time.
[0282] <6. Other Embodiments> In the above embodiment, a vehicle V was used as the moving body, but the moving body according to this disclosure is not limited to a vehicle V. The moving body may be a vessel, airplane, or other vehicle other than a vehicle V.
[0283] Furthermore, the notification device 11 is not limited to the combination of the terminal 11A and the vibration device 11B. In place of or in addition to the vibration device 11B, one or more of the vehicle control devices 12 exemplified in FIG. 2 may be used for the notification device 11. That is, the lighting device 121 may be used as a visual means, and the air conditioning device 122 and the acoustic device 123 may be used as a tactile means and an auditory means, respectively.
Description of Reference Numerals
[0284] S Occupant support system (information notification system) V Vehicle (moving body) 1 In-vehicle system 10 In-vehicle device 11 Notification device 11A Terminal (detection device) 15 Display (visual means) 16 Speaker (auditory means) 11B Vibration device (tactile means) 100 In-vehicle computer 101 Control unit 111 Estimation unit 112 Notification proposal unit C Occupant characteristic C0 Personal characteristic C1 Reliability C2 Degree of interest C3 Uniqueness (behavioral tendency) C4 Travel purpose C5 Dynamic state C7 Preference characteristic C8 Driving ability C81 Cognitive ability C82 Driving skill In Notification information (information) I1 Dialogue content I2 Biological information I3 Travel information J1 Type of information targeted for customization proposal J2 Proposal content J3 Proposal timing Rb Bias index Tp Travel route
Claims
1. An information notification system that notifies the occupants of a moving vehicle of information, A notification device that notifies the aforementioned occupant of information, A detection device mounted on the mobile body for detecting information related to the occupants and the mobile body, An estimation unit estimates occupant characteristics, including one or more of the occupant's driving ability, the occupant's dynamically changing dynamic state, and the occupant's purpose of travel, based on the detection signal of the detection device. Based on the estimation results of the estimation unit, the system includes a notification and proposal unit that determines the timing and content of the proposed customization of the information to be notified to the occupant, and, based on the determination result, executes the proposed customization to the occupant. The notification device, depending on whether the occupant accepts or rejects the proposal, will notify the information with the proposed content reflected in it. Information notification system.
2. In the information notification system described in claim 1, The estimation unit estimates the driving capacity, the dynamic state, and the purpose of the movement, respectively. The notification and proposal unit determines, based on the combination of the driving capacity, the dynamic state, and the purpose of movement, the content of the proposed customization and the timing of the proposed customization corresponding to each type of information subject to the proposed customization. Information notification system.
3. In the information notification system described in claim 2, The proposed customizations are as follows: The notification content of the aforementioned information, The means for notifying the aforementioned information, This includes suggestions for customization regarding the notification order of the aforementioned information when such information coexists, and one or more of the following: Information notification system.
4. In the information notification system described in claim 3, The notification and proposal unit determines the timing of the customization proposal based on the change in driving capacity. The notification proposal unit makes information regarding driving assistance corresponding to the traffic conditions around the mobile body the subject of the customization proposal. Information notification system.
5. In the information notification system described in claim 3, The aforementioned driving ability includes a combination of the occupant's cognitive ability and the occupant's driving skills. Information notification system.
6. In the information notification system described in claim 5, The notification means includes a visual means for notifying visual information, an auditory means for notifying auditory information, and a tactile means for notifying tactile information. The notification suggestion unit proposes to the occupant, as part of the customization suggestions, that if the cognitive ability is low, the frequency of notifications by the visual means be reduced compared to when the cognitive ability is high, and the frequency of notifications by the auditory means or the tactile means be increased. Information notification system.
7. In the information notification system described in claim 3, The estimation unit classifies the occupant's dynamic state into one or more of a plurality of categories. The notification proposal unit selects information relating to driving assistance corresponding to the categories classified by the estimation unit as the subject of the customization proposal. Information notification system.
8. In the information notification system described in claim 7, The aforementioned dynamic state includes, as the plurality of categories, a plurality of categories related to the emotional and physical state of the occupant, Information notification system.
9. In the information notification system described in claim 3, The detection device is mounted on the mobile body and includes a terminal that receives input for the purpose of movement. The notification suggestion unit determines that it is time to suggest the customization when the purpose of movement is entered into the terminal. The notification proposal unit makes information regarding driving assistance corresponding to the purpose of travel the subject of the customization proposal. Information notification system.
10. In the information notification system described in claim 9, The information relating to driving assistance corresponding to the purpose of movement includes the movement path of the moving body and the stopping spots or passing spots of the moving body located along the movement path. The estimation unit classifies the purpose of the occupant's movement into one or more categories. The notification suggestion unit notifies the crew whether or not to be notified of the stopping spot or the passing spot, based on the categories classified by the estimation unit, as a suggested customization. Information notification system.
11. In the information notification system described in claim 1, The estimation unit estimates the degree of trust with the occupant based on the occupant's response to the notification of the information, Based on the reliability level, the notification and proposal unit sets the timing of the customization proposal to a lower frequency when the reliability level is high, and sets the process of automatically reflecting the proposed content without requiring the crew to accept or reject the proposal to a higher frequency, compared to when the reliability level is low. Information notification system.
12. In the information notification system described in claim 1, The estimation unit estimates the degree of interest of the occupant in the notification based on the occupant's response to the notification of the information. The notification and proposal unit, based on the level of interest, sets the timing of the customization proposal to a higher frequency when the level of interest is high compared to when the level of interest is low. Information notification system.
Citation Information
Patent Citations
Information notification system
JP2020085568A
Information notification system, information notification method, and information notification program
JP2024060481A
Driving behavior change support device and driving behavior change support program
JP2024157854A
Information notification control device, information notification control method, on-vehicle information notification system, data table generation method, and program
JP2025014571A
