Notification device, notification method, and program

Abstract

To accurately detect the physical condition of a user, and to perform notification matched with the physical condition of the user.SOLUTION: The communication terminal 1 functioning as the notification device includes the intellectual function evaluation unit 12 that evaluates a plurality of intellectual functions related to the physical condition of the driver D on the basis of the biological information Db of the driver D who is a user, the conversation unit 14 that functions as the voice acquisition unit that acquires the voice information of the driver D, the voice evaluation unit 15 that evaluates the second physical condition information including the voice analysis result related to the target of the driver D on the basis of the voice information, and the notification control unit 16 that notifies of the notification information corresponding to the physical condition of the driver D on the basis of the combination of the evaluation result of the intellectual function evaluation unit 12 and the evaluation result of the voice evaluation unit 15.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a notification device, a notification method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have been gaining momentum. To achieve this, efforts are being made to further improve traffic safety and convenience through research and development of preventive safety technologies. For example, Patent Documents 1 and 2 disclose safe driving support devices that use biometric information, including imaging results, respiration, and pulse waves, and vehicle information, to estimate the driver's cognitive state or provide advice or warnings to the driver. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-200544 [Patent Document 2] Japanese Patent Application Publication No. 2023-145868 Summary of the Invention [Problem to be solved by the invention]

[0004] In preventive safety technology, it is desirable to take necessary measures according to the driver's physical condition, which affects the driver's cognitive ability and judgment ability. One of the necessary measures is to accurately detect the user's physical condition and provide notifications according to the detected physical condition. In order to solve the above problems, the present application aims to accurately detect a user's physical condition and provide notifications tailored to the user's physical condition, thereby further improving traffic safety and contributing to the development of sustainable transportation systems. [Means for solving the problem]

[0005] One aspect of the present disclosure is a notification device that includes an intellectual function evaluation unit that evaluates multiple intellectual functions related to the user's physical condition based on the user's biometric information, a voice acquisition unit that acquires the user's voice information, a voice evaluation unit that evaluates voice analysis results related to the user's physical condition based on the voice information, and a notification control unit that notifies the user of notification information corresponding to the user's physical condition based on a combination of the evaluation results of the intellectual function evaluation unit and the evaluation results of the voice evaluation unit.

[0006] Another aspect of the present disclosure is a notification method executed by a notification device, comprising: an intellectual function evaluation step of evaluating multiple intellectual functions related to the user's physical condition based on the user's biometric information; an audio information acquisition step of acquiring the user's audio information; an audio information evaluation step of evaluating audio analysis results related to the user's physical condition based on the audio information; and a notification step of notifying the user of notification information corresponding to the user's physical condition based on a combination of the evaluation results of the intellectual function evaluation step and the evaluation results of the audio information evaluation step.

[0007] Another aspect of the present disclosure is a program that causes a notification device controlled by a computer to function as an intellectual function evaluation unit that evaluates multiple intellectual functions related to the user's physical condition based on the user's biometric information, an audio acquisition unit that acquires the user's audio information, an audio evaluation unit that evaluates audio analysis results related to the user's physical condition based on the audio information, and a notification control unit that notifies the user of notification information corresponding to the user's physical condition based on a combination of the evaluation results of the intellectual function evaluation unit and the evaluation results of the audio evaluation unit. [Effects of the Invention]

[0008] According to the present invention, it is possible to accurately detect the user's physical condition and provide notifications tailored to the user's physical condition. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a usage mode of a communication terminal. [Figure 2] FIG. 2 is a diagram illustrating a configuration of a communication terminal. [Figure 3] FIG. 3 is a diagram showing the relationship between the evaluation target items, a plurality of intellectual functions, and driving advice items. [Figure 4] FIG. 4 is a flowchart illustrating a notification method executed by a communication terminal. [Figure 5] FIG. 5 is a diagram illustrating an example of the dialogue process. [Figure 6] FIG. 6 is a diagram showing the relationship between the evaluation results of the first physical condition information, the evaluation results of the second physical condition information, and notification information during driving. DETAILED DESCRIPTION OF THE INVENTION

[0010] [1. Usage of communication terminal 1] An example of a usage mode of a communication terminal 1 according to an embodiment of a notification device of the present invention will be described with reference to FIG. As shown in FIG. 1, the communication terminal 1 is attached via a terminal holder 150 to a position in front of the driver D, who is the user, inside the vehicle 100. The communication terminal 1 is a portable device such as a smartphone, mobile phone, or tablet terminal, and can be carried by the driver D. The vehicle 100 is a four-wheeled vehicle and is an example of a target vehicle of the present disclosure. Note that the target vehicle may be a moving body other than a four-wheeled vehicle, such as an airplane, a motorcycle, or a ship.

[0011] The driver D wears a wearable device 5. The wearable device 5 is a device such as a smart watch that is worn on the wrist, arm, or head, or a wearable smart innerwear or a sensing device that is attached to the body or epidermis, and includes a vital sensor 6 and a communication unit 7. The vital sensor 6 is a sensor that continuously measures the biological information Db of the driver D at time intervals. The vital sensor 6 includes a sensor that can measure the biological information Db related to the sleep, exercise, and stress of the driver D, and includes a group of sensors that can measure, for example, the heart rate, wrist skin temperature that is an indicator of body temperature, an electrocardiogram, blood oxygen concentration, activity level, and information related to sleep.

[0012] The communication unit 7 is a unit that performs wireless communication with the communication terminal 1, and performs short-range communication in accordance with specifications such as Bluetooth (registered trademark) and UWB (Ultra Wide Band). The wearable device 5 uses the communication unit 7 to transmit measured biological information Db to the communication terminal 1. Information about the communication terminal 1, which is the communication destination, is registered in advance in the communication unit 7.

[0013] The vehicle 100 includes an ECU (Electronic Control Unit) 110 that controls the operation of each part of the vehicle 100, a communication unit 120, a navigation device 121, a front camera 130 that takes images in front of the vehicle 100, a driver monitor camera 131 that takes images of the driver D, a display 132, speakers 133a and 133b, an acceleration sensor 134, and the like. The ECU 110 communicates with the communication terminal 1 via the communication unit 120. The vehicle 100 includes a steering wheel 140, an accelerator pedal 141, a brake pedal 142, and the like as driving operation parts that the driver D operates when driving.

[0014] The communication terminal 1 can detect the behavior of the vehicle 100, which changes in response to the operation of the driving operation unit, that is, the operation of the steering wheel 140, the accelerator pedal 141, and the brake pedal 142 by the driver D, using an acceleration sensor 32 (see FIG. 2 ) built into the communication terminal 1. The notification device 2 may detect the behavior of the vehicle 100 by acquiring the detection result of the acceleration sensor 134 provided in the vehicle 100.

[0015] The communication terminal 1 notifies the user of notification information based on the biological information Db detected by the wearable device 5, etc., on a display 34 provided in the communication terminal 1. Furthermore, the communication terminal 1 has various functions that are generally provided in a smartphone, such as a map display function that displays map information including the current value on the display 34, a route guidance function that provides route guidance from the current value to a destination set by the driver D, and a music playback function. Note that the communication terminal 1 may also have a function that displays the advice information and map information on the display 132 of the vehicle 100.

[0016] [2. Configuration of communication terminal 1] The configuration of the communication terminal 1 will be described with reference to Fig. 2. The communication terminal 1 includes a control unit 3 having a processor 10 and a memory 20, a communication unit 30, a camera 31, an acceleration sensor 32, a GNSS (Global Navigation Satellite System) sensor 33, a display 34, a speaker 35, a microphone 36, and the like.

[0017] The communication unit 30 has a communication module for wireless communication with an external device. The communication unit 30 includes a communication module that performs short-range wireless communication in accordance with specifications such as Bluetooth (registered trademark) and UWB (Ultra Wide Band) between the vehicle 100 and the wearable device 5. The communication unit 30 also includes a communication module that communicates with a traffic information server 210 and a service providing server 211 via a communication network 200 such as a mobile communication network. Information about the vehicle 100 and the wearable device 5, which are the communication destinations, is registered in advance on the communication unit 20 side.

[0018] The camera 31 can capture an image of the driver D when the communication terminal 1 is attached to the terminal holder 150 (see FIG. 1). The acceleration sensor 32 detects, for example, six axes (acceleration in each of the longitudinal, lateral, and vertical directions, and angular velocity about each axis). The GNSS sensor 33 detects the current position of the communication terminal 1. When the communication terminal 1 is attached to the terminal holder 150, the position detected by the GNSS sensor 33 becomes the current position of the vehicle 100. The display 34 is visible to the driver D when the communication terminal 1 is attached to the terminal holder 150 (see FIG. 1). The display 34 is a touch panel, and the driver D can operate the communication terminal 1 by touching the display 34.

[0019] The memory 20 stores an application program 21 for the communication terminal 1, map information 22 including roads, and a biometric information database 23 (hereinafter referred to as biometric information DB23) that stores biometric information Db acquired from the wearable device 5 in chronological order. The processor 10 is a computer that controls the communication terminal 1, and by reading and executing the program 21, it functions as an information acquisition unit 11, an intellectual function evaluation unit 12, an entry detection unit 13, a dialogue unit 14, a voice evaluation unit 15, a notification control unit 16, and a vehicle control unit 17, etc.

[0020] The information acquisition unit 11 acquires the biological information Db measured by the wearable device 5 using the communication unit 30, and stores the biological information Db in the memory 20. The memory 20 also stores the measurement date and time of the biological information Db. Furthermore, the information acquisition unit 11 acquires information acquired by the camera 31, the acceleration sensor 32, the GNSS sensor 33, and the microphone 36, and stores the information in the memory 20. The information acquired by the camera 31 is, for example, facial image data of the driver D, and the facial image data may be used as biometric information Db that identifies the physical condition, etc., of the driver D. Furthermore, the information acquired by the microphone 36 is, for example, voice information of the driver D.

[0021] The information acquired by the acceleration sensor 32 is information indicating the behavior of the vehicle 100 in response to the operation of the steering wheel 140, the accelerator pedal 141, and the brake pedal 142 by the driver D, that is, driving behavior information Dv indicating the driving behavior of the driver D. The driving behavior information Dv is continuously acquired at time intervals and stored in the memory 20. In addition, the information acquisition unit 11 is not limited to a configuration in which it acquires driving behavior information Dv using the acceleration sensor 32 of the communication terminal 1, and may acquire driving behavior information Dv acquired by the acceleration sensor 134 of the vehicle 100 using short-range communication.

[0022] Furthermore, the information acquisition unit 11 can acquire information on the communication network 200 by using the communication unit 30. For example, the information acquisition unit 11 can acquire road traffic information Dt provided by the traffic information server 210 and various types of provided information provided by the service providing server 211. The provided information provided by the service providing server 211 includes, for example, test information (e.g., health checkup information) on the physical and cognitive functions of the driver D provided by a medical institution, advice provided by a medical institution, and information that can identify the external environment of the driver D and the vehicle 100 (weather information, etc.). The test information may be used as biological information Db.

[0023] The intellectual function evaluation unit 12 performs processing to evaluate a plurality of intellectual functions related to the physical condition of the driver D and related to the driving ability of the driver D, based on the biological information Db. More specifically, the intellectual function evaluation unit 12 has a calculation unit 12a that calculates evaluation values ​​of a plurality of intellectual functions consisting of planning ability 311, perception ability 312, attention ability 313, judgment ability 314, and operation ability 315 shown in Fig. 3, based on the biological information Db, evaluates the plurality of intellectual functions based on the calculated evaluation values, and detects whether the physical condition of the driver D is good or not based on these evaluations.

[0024] Figure 3 shows the relationship between the items to be evaluated, the multiple intellectual functions evaluated by the intellectual function evaluation unit 12 (planning ability 311, perceptual ability 312, attention ability 313, judgment ability 314, and operating ability 315), and the driving advice items (first driving advice item 321 to fifth driving advice item 325). The biological information Db obtained by the vital sensor 6 includes information that affects physiological indices consisting of sleep 304, exercise 305, and stress 306 shown in Fig. 3, so the status of sleep 304, exercise 305, and stress 306 can be estimated from the biological information Db. The driving behavior information Dv includes information on accelerator operation 301, brake operation 302, and steering operation 303 shown in Fig. 3, so the status of accelerator operation 301, brake operation 302, and steering operation 303 can be estimated from the driving behavior information Dv.

[0025] 3, planning ability 311 is particularly related to sleep 304, perception ability 312 is related to sleep 304, accelerator operation 301, brake operation 302, and steering operation 303. Attention ability 313 is related to sleep 304, exercise 305, stress 306, accelerator operation 301, brake operation 302, and steering operation 303, and judgment ability 314 and operation ability 315 are related to sleep 304, exercise 305, accelerator operation 301, brake operation 302, and steering operation 303.

[0026] Using these relationships, the intellectual function evaluation unit 12 scores the sleep 304, exercise 305, and stress 306 of the driver D based on the biological information Db of the driver D, acquires first physical condition information corresponding to the evaluation value for each intellectual function based on the relationships shown in Fig. 3, and compares each piece of first physical condition information with a predetermined threshold to determine whether each intellectual function is good or not. If the majority of the intellectual functions are good (three or more), the intellectual function evaluation unit 12 determines that the physical condition related to the intellectual function of the driver D (appropriately referred to as the first physical condition) is good, and if less than half of the intellectual functions are good (two or less), it determines that the first physical condition of the driver D is poor.

[0027] In this description, the term "physical condition" is not limited to the narrow sense, but includes information indicating at least one of physical condition and health in the broad sense. Furthermore, a known method may be applied as appropriate to the evaluation process of the first physical condition, including the calculation process of the evaluation value for each intellectual function. Furthermore, test information (e.g., health check information) of the physical functions and cognitive functions of the driver D may be obtained from the service providing server 211, and the evaluation of each intellectual function and the first physical condition may be performed based on the biological information Db including this test information. Furthermore, the intellectual functions may be changed, increased or decreased as appropriate. In addition, the intellectual function evaluation unit 12 may further evaluate the planning ability 311, perception ability 312, attention ability 313, judgment ability 314, and operation ability 315 based on the driving behavior information Dv, and evaluate the physical condition of the driver D by taking into consideration both this evaluation and the evaluation based on the biological information Db.

[0028] Next, the driving advice items shown in FIG. 3 will be described. The first driving advice item 321 is an item for which driving advice is suitable when the planning ability 311 or the attention ability 313 is low, and is, for example, the operating speed of the accelerator pedal 141 when starting off. The second driving advice item 322 is an item for which driving advice is suitable when the planning ability 311 or the judgment ability 314 is low, and is, for example, the operating speed of the brake pedal 142 when stopping.

[0029] The third driving advice item 323 is an item for which driving advice is suitable when the perceptual ability 312, attention ability 313, or judgment ability 314 is low, and is, for example, the operating speed of the steering wheel 140 when winding or overtaking a vehicle. The fourth driving advice item 324 is an item for which driving advice is suitable when the attention ability 313 is low, and is, for example, the distance between the vehicle and the vehicle ahead.

[0030] The fifth driving advice item 325 is an item for which driving advice is suitable when the perceptual ability 312, the judgment ability 314, or the operation ability 315 is low, and is, for example, swaying of the vehicle 100 from side to side. The manner in which these driving advice items are used will be described later.

[0031] Returning to FIG. 2, the entry detection unit 13 detects the timing when the driver D gets into the vehicle 100 (hereinafter referred to as the entry timing), thereby performing a process of detecting whether the driver D is before getting into the vehicle or is in the vehicle. The process of detecting the entry timing is not particularly limited, but for example, the entry detection unit 13 acquires image information from a camera inside the vehicle 100 (e.g., the driver monitor camera 131) or a camera provided in the vehicle 100 that captures images of the outside of the vehicle, and detects the entry timing of the driver D using image recognition technology. The entry detection unit 13 may also receive a signal from a weight sensor provided in the driver's seat from the vehicle 100, and detect the entry timing of the driver D based on the received signal.

[0032] The dialogue unit 14 performs dialogue processing to dialogue with the driver D. The dialogue processing is processing to use the speaker 133a to perform dialogue voice output (predetermined voice output) to prompt a voice response from the driver D. This dialogue processing includes processing to collect voice information of the driver D by the microphone 36. In this embodiment, the dialogue processing is executed at the timing when the driver D gets in, or immediately before or immediately after the timing when the driver D gets in, which is detected by the boarding detection unit 13. This makes it possible to acquire voice information of the driver D at the time of boarding, immediately before boarding, or immediately before boarding, that is, to acquire voice information of the driver D immediately before driving.

[0033] The voice evaluation unit 15 performs a process of acquiring voice physical condition information related to the physical condition of the driver D by performing a voice analysis of the voice information of the driver D acquired through the dialogue processing. The content of this processing is not particularly limited, but in this embodiment, voice features are extracted and an arithmetic process is performed to calculate information indicating the vitality and mental stability of the driver D from the extracted features, thereby acquiring the voice physical condition information indicating the vitality and mental stability. Then, based on the voice physical condition information, the voice evaluation unit 15 determines whether the physical condition (appropriately referred to as the second physical condition) affected by the vitality and mental stability is good or not. The processing performed by the voice evaluation unit 15 is not limited to the above processing, and a wide range of processing can be applied to obtain information that can be used to evaluate the physical condition of the driver D from the voice of the driver D.

[0034] The notification control unit 16 generates notification information corresponding to the physical condition of the driver D based on a combination of the evaluation results of the intellectual function evaluation unit 12 and the evaluation results of the voice evaluation unit 15, and displays the generated notification information on the display 34, or emits a voice corresponding to the notification information from the speaker 133a. The vehicle control unit 17 has a control function for the vehicle 100, and more specifically, is capable of controlling the driving assistance functions of the vehicle 100 by communicating with the vehicle 100 using the communication unit 30. The driving assistance functions include an automatic braking function, a lane keeping assist function, and adaptive cruise control (ACC).

[0035] [3. Operation of communication terminal 1] The notification method executed by communication terminal 1 will be described with reference to the flowchart shown in Fig. 4. The flowchart shown in Fig. 4 is repeatedly executed at appropriate interrupt timings when application program 21 is being executed. Furthermore, when wearable device 5 is present within the short-range wireless communication area of ​​communication terminal 1, information acquisition unit 11 executes a process of acquiring biological information Db measured by wearable device 5.

[0036] 4, the notification control unit 16 of the communication terminal 1 determines whether or not the boarding detection unit 13 has detected that the driver D has boarded the vehicle (step S1A). If the boarding detection unit 13 has not detected that the driver D has boarded the vehicle (step S1A / NO), the notification control unit 16 determines whether or not a certain time has elapsed since the previous advice was displayed on the display 34 (step S2A). If the certain time has not elapsed (step S2A / NO), the notification control unit 16 proceeds to the processing of step S1A.

[0037] If the notification control unit 16 does not detect that the driver D has boarded the vehicle (step S1A / NO), and after a certain period of time has elapsed (step S2A / YES), the intellectual function evaluation unit 12 acquires first physical condition information corresponding to the evaluation value for each of multiple intellectual functions based on the biometric information Db for a predetermined period in the biometric information DB23 stored in the memory 20, and performs a process of evaluating the first physical condition of the driver D based on the first physical condition information (step S3A). As a result, the intellectual function evaluation unit 12 detects whether or not the majority (three or more) of the intellectual functions in the planning ability 311, perception ability 312, attention ability 313, judgment ability 314, and operation ability 315 during a predetermined period are good, and detects whether or not the first physical condition is good depending on whether or not the majority of the intellectual functions are good.

[0038] Next, the notification control unit 16 performs a pre-driving notification process (step S4A) based on the evaluation results of the intellectual function evaluation unit 12. For example, if the majority of the intellectual functions (three or more) are good, the first physical condition of the driver D is detected as good, and the notification control unit 16 displays notification information consisting of advice suitable for maintaining health on the display 34, or emits a sound corresponding to the notification information from the speaker 133a. On the other hand, if less than half of the intellectual functions are good, the first physical condition of the driver D is detected as poor, and the notification control unit 16 displays notification information consisting of information encouraging exercise or going outside for the purpose of promoting health, and / or notification information related to the intellectual function detected as the poorest on the display 34, or emits a sound corresponding to the notification information from the speaker 133a.

[0039] The content of the pre-driving notification process may be changed as appropriate and need not be limited to notification information based on multiple intellectual function evaluation results. For example, since the intellectual function evaluation unit 12 can identify sleep 304, exercise 305, and stress 306 based on the biological information Db, it may notify notification information related to the worst state among sleep 304, exercise 305, and stress 306.

[0040] When the notification control unit 16 detects that the driver D has boarded the vehicle (step S1A / YES), the dialogue unit 14 performs dialogue processing (step S5A), and the intellectual function evaluation unit 12 acquires first physical condition information corresponding to the evaluation value for each of a plurality of intellectual functions based on the biometric information Db for a predetermined period in the biometric information DB23, and performs processing to evaluate the physical condition of the driver D (corresponding to the first physical condition) based on the first physical condition information (step S6A). As a result, similarly to step S3A, it is detected whether or not the majority (three or more) of the intellectual functions in the planning ability 311, perception ability 312, attention ability 313, judgment ability 314, and operation ability 315 during a predetermined period are good, and whether or not the first physical condition is good is detected depending on whether or not the majority of the intellectual functions are good.

[0041] Here, the predetermined period is set to a period during which the relatively long-term physical condition of the driver D (corresponding to the first physical condition) can be identified, for example, set to a period of several hours, several days, or several months from the present time. This makes it possible to obtain an evaluation result of the relatively long-term physical condition of the driver D. Note that the predetermined period in step S6A may be longer than the period in step S3A. At least in step S6A, the relatively long-term physical condition of the driver D can be detected.

[0042] FIG. 5 is a diagram illustrating an example of the dialogue process. The left side of Fig. 5 shows notification information that is emitted as sound by the dialogue unit 14 via the speaker 133a. The notification information may also be displayed on the display 34. The right side of Fig. 5 shows the content of the utterance made by the driver D. The voice uttered by the driver D is collected by the microphone 36.

[0043] The dialogue content of the dialogue unit 14 may be changed as appropriate within the scope of prompting a voice response from the driver D. In the example of Fig. 5, the dialogue content includes a message inquiring about the destination ("Where will you be driving today?"), and messages inquiring about whether audio playback is enabled and preferences for audio playback ("Do you listen to music while driving?", "Do you prefer relaxing music or uplifting music?", etc.). Therefore, the communication terminal 1 can use known voice recognition technology to acquire information about the destination, whether audio is played, and audio preferences spoken by the driver D. The communication terminal 1 can start a route guidance process to the destination and an audio playback process by executing a process corresponding to the acquired information.

[0044] When performing route guidance processing, it is preferable to obtain road traffic information Dt provided by the traffic information server 210 and use congestion information and the like included in the road traffic information Dt to search for a recommended route.

[0045] Next, the notification control unit 16 performs a voice analysis of the voice information of the driver D acquired by the dialogue processing using the voice evaluation unit 15, acquires second physical condition information consisting of voice physical condition information indicating the driver D's vitality level, etc., and performs a process of evaluating the physical condition (corresponding to the second physical condition) of the driver D based on the second physical condition information (step S7A). This makes it possible to obtain an evaluation result of the driver D's current physical condition from the voice, that is, to obtain an evaluation result of the driver D's physical condition over a relatively short period of time.

[0046] Next, the notification control unit 16 performs notification processing during driving based on a combination of the evaluation result of the first physical condition information and the evaluation result of the second physical condition information state (step S8A). 6 is a diagram showing the relationship between the evaluation results of the first physical condition information, the evaluation results of the second physical condition information, and notification information during driving. In FIG. 6, ○ indicates good, and × indicates bad. As shown in Fig. 6, when the majority of the five types of intellectual functions indicated by the first physical condition information are good and the physical condition is detected to be good based on the second physical condition information, the notification control unit 16 executes the first notification process. The first notification process is a notification process when the driving risk is estimated to be the lowest based on the physical condition of the driver D, and as exemplified in Fig. 6, the minimum notification information ("Have a good trip.") for users in good physical condition is notified. Because it is the minimum notification information, the driver D is unlikely to feel bothered.

[0047] Furthermore, if the majority of the five types of intellectual functions are good and the physical condition is detected to be poor based on the second physical condition information, the notification control unit 16 executes a second notification process. The second notification process is a process for notifying information different from that of the first notification process, and includes notifying advice regarding driving operation, as shown in Fig. 6. In the notification information of the second notification process illustrated in Fig. 6, the driving advice item corresponding to at least the intellectual function with the lowest rating out of the driving advice items 321 to 325 shown in Fig. 3 is input at the position indicated by the symbol AA.

[0048] To give a specific example, if the intellectual function with the lowest rating is operating ability 315, the driving advice item corresponding to operating ability 315 is the fifth driving advice item 325 (left and right swaying of vehicle 100), so notification information including "When driving today, be careful of left and right swaying of vehicle 100" is notified.

[0049] Furthermore, if the majority of the five types of intellectual functions are not good and the physical condition is detected to be good based on the second physical condition information, the notification control unit 16 executes a third notification process. The third notification process is a process for notifying information different from the first and second notification processes, and as exemplified in Fig. 6, notification information that includes advice regarding physical condition and has a relatively small number of characters is created.

[0050] In the notification information of the third notification process illustrated in Fig. 6, at least a statement corresponding to the intellectual function with the lowest evaluation is entered at the position indicated by the symbol BB. To give a specific example, if the intellectual function with the lowest evaluation is attention ability 313, notification information including "You seem to be less attentive recently. Please drive carefully" will be notified.

[0051] Furthermore, if the majority of the five types of intellectual functions are not good and the physical condition is detected as poor based on the second physical condition information, the notification control unit 16 executes a fourth notification process. The fourth notification process is a process for notifying information different from the first to third notification processes, and includes notification of advice regarding physical condition and driving operation, as exemplified in Fig. 6.

[0052] In the notification information of the fourth notification process illustrated in Figure 6, the position indicated by the symbol CC is filled with text corresponding to at least the intellectual function with the lowest evaluation, and the position indicated by the symbol DD is filled with driving advice items (see Figure 3) corresponding to at least the intellectual function with the lowest evaluation. To give a specific example, if the intellectual function with the lowest rating is operating ability 315, the driving advice item corresponding to operating ability 315 is the fifth driving advice item 325 (left and right swaying of vehicle 100), so in the fourth notification process, notification information including "Your operating ability seems to have been low recently.Your physical condition does not seem to be good at the moment, so please drive carefully, paying attention to left and right swaying of vehicle 100." is notified.

[0053] As shown in Figure 6, the first notification process is a notification process when the driving risk is estimated to be the lowest based on the physical condition of driver D, the second notification process is a notification process when the driving risk is estimated to be the next lowest when the first notification process is performed, the third notification process is a notification process when the driving risk is estimated to be the next lowest when the second notification process is performed, and the fourth notification process is a notification process when the driving risk is estimated to be the highest based on the physical condition of driver D.

[0054] As shown in FIG. 6, when the third notification process is performed, a process for changing the settings of the driving assistance function is also performed. The setting change process for the driving assistance function is a process in which the vehicle control unit 17 changes the driving assistance function of the vehicle 100 to a predetermined setting that prioritizes safety. The predetermined setting is, for example, a setting that increases the inter-vehicle distance when adaptive cruise control is in effect. Note that, after the setting change process for the driving assistance function is performed, when the first notification process and the second notification process are performed, that is, when the physical condition of the driver D has improved, the driving assistance function may be returned to the setting before the setting change (for example, a user setting value).

[0055] As shown in FIG. 6, when the fourth notification process is performed, in addition to the process of changing the settings of the driving assistance function, a process of notifying the surroundings is also performed. The notification process to the surroundings is a process in which the notification control unit 16 uses the communication unit 30 to wirelessly communicate advice to the surroundings related to the driving of the driver D to occupants of other vehicles around the vehicle 100, family members of the driver D, etc., of the advice to the surroundings. For example, by wirelessly transmitting information corresponding to the advice to the in-vehicle systems of the other vehicles or to mobile devices carried by the occupants or family members, the advice to the surroundings is notified from the in-vehicle systems or mobile devices (e.g., smartphones). In this way, if the driver D is feeling unwell, it is possible to notify those around him / her to pay attention. Note that communication with the in-vehicle systems of the other vehicles, etc., may be performed using vehicle-to-vehicle communication.

[0056] 4, after executing the notification process during driving in step S8A, the notification control unit 16 determines whether or not it has detected that the driver D has dismounted (step S9A), and if it has detected that the driver D has dismounted (step S9A / YES), it performs post-driving processing (step S10A). There are no particular limitations on the method for detecting dismounting, and it is possible to apply, for example, a method that uses the camera 130 or a weight sensor in the driver's seat.

[0057] The post-driving process of step S10A is a process of creating a hazard map describing locations where inappropriate driving behavior occurred based on the driving behavior information Dv acquired during driving. For example, the information acquisition unit 11 continuously acquires the driving behavior information Dv while the vehicle is traveling, and detects locations where inappropriate driving behavior such as sudden braking or swerving to the left or right based on the driving behavior information Dv and information on the current position detected by the GNSS sensor 33, and creates a hazard map describing the detection results. The created hazard map may be data described in the map information 22 in the memory 20, or may be data stored in the memory 20 separately from the map information 22.

[0058] When a hazard map is created, the hazard map can be used while driving, for example, by the notification control unit 16 notifying the driver D of a location where inappropriate driving behavior has occurred when the driver D is driving near the location, thereby urging the driver D to drive safely. Note that the processing after driving is not limited to the processing of creating a hazard map, and any appropriate processing may be applied.

[0059] [4. Effects of the embodiment] As described above, the communication terminal 1 includes an intellectual function evaluation unit 12 that evaluates multiple intellectual functions related to the physical condition of the user, i.e., the driver D, based on the biometric information Db of the driver D; a dialogue unit 14 that functions as a voice acquisition unit that acquires voice information of the driver D; a voice evaluation unit 15 that evaluates second physical condition information consisting of voice analysis results related to the subject of the driver D based on the voice information; and a notification control unit 16 that notifies notification information corresponding to the physical condition of the driver D based on a combination of the evaluation results of the intellectual function evaluation unit 12 and the evaluation results of the voice evaluation unit 15.

[0060] This configuration makes it possible to accurately detect the user's physical condition and provide notifications tailored to the user's physical condition, thereby realizing healthcare mobility services that maintain the safety and health of users, and ultimately contributing to the development of sustainable transportation systems by further improving traffic safety. The intellectual function evaluation unit 12 can also be called a first physical condition evaluation unit, the voice evaluation unit 15 can also be called a second physical condition evaluation unit, and the information acquisition unit 11 can also be called a biological information acquisition unit.

[0061] The notification information also includes advice related to the driver D's driving, and the voice evaluation unit 15 performs processing to acquire voice information of the driver D when it detects that the driver D is getting into the vehicle 100. With this configuration, it is possible to provide an appropriate notification that takes into account the physical condition of the driver D immediately before driving.

[0062] Furthermore, when the intellectual function evaluation unit 12 detects that multiple intellectual functions are good and the voice evaluation unit 15 detects that the physical condition is good, the notification control unit 16 performs the first notification process. This allows notifications tailored to the user's good physical condition. For example, by notifying the user of the minimum amount of notification information for a user in good physical condition, the user is less likely to feel annoyed.

[0063] Furthermore, when the intellectual function evaluation unit 12 detects that the multiple intellectual functions are good and the voice evaluation unit 15 detects that the physical condition is poor, the notification control unit 16 performs a second notification process to notify advice regarding driving operation as a process to notify information different from the first notification process. With this configuration, in the case of temporary or mild poor physical condition, advice can be given to support driving caused by the poor physical condition.

[0064] Furthermore, when the number of intellectual functions detected as poor by the intellectual function evaluation unit 12 is greater than the number of intellectual functions detected as good, and the voice evaluation unit 15 detects that the physical condition is good, the notification control unit 16 performs a third notification process to notify advice regarding physical condition, which is a process to notify information different from the first notification process and the second notification process. With this configuration, when the physical condition is not so good but the current condition is not bad, advice on how to take care of the physical condition can be given.

[0065] Furthermore, when the number of intellectual functions detected as poor by the intellectual function evaluation unit 12 is greater than the number of intellectual functions detected as good, and the voice evaluation unit 15 detects that the physical condition is poor, the notification control unit 16 performs a fourth notification process that notifies information different from the first to third notification processes, and notifies advice regarding physical condition and driving operation. With this configuration, when the physical condition is poor and the current condition is also poor, advice can be given to take care of both physical condition and driving.

[0066] The processing by the intellectual function evaluation unit 12 is an example of an intellectual function evaluation step of the present disclosure, the processing by the dialogue unit 14 is an example of a voice acquisition step of the present disclosure, the processing by the voice evaluation unit 15 is an example of a voice evaluation step of the present disclosure, and the processing by the notification control unit 16 is an example of a notification control step of the present disclosure.

[0067] Furthermore, the notification information includes advice for those around the driver D related to the driving of the driver D, and when the fourth notification process is performed, the notification control unit 16 performs a process of transmitting the advice for those around the driver D to the outside using wireless communication. With this configuration, when the driver D is driving and is not feeling well, it is possible to send a notification to those around the driver D to call their attention.

[0068] Furthermore, when at least one of the third notification process and the fourth notification process is performed, the vehicle control unit 17 is provided which changes the driving assistance function of the vehicle 100 driven by the driver D to a predetermined setting that prioritizes safety. With this configuration, when the driver D is not in good physical condition and is currently in poor health, driving assistance that prioritizes safety, in other words, driving assistance tailored to the user who is in poor physical condition, can be provided.

[0069] 5. Other Embodiments The above embodiment is merely one embodiment of the present invention, and any modifications and applications are possible without departing from the spirit of the present invention.

[0070] For example, the notification information may include information indicating the external environment (weather information, etc.) of the driver D or the vehicle 100, obtained from the service providing server 211. Furthermore, the configuration of the communication terminal 1 shown in FIG. 2 is an example, and the configuration may be changed as appropriate. The configuration of each part shown in FIG. 2 and other figures can be widely applied to a configuration realized by software, a configuration realized by hardware, or a configuration realized by a combination of software and hardware. Furthermore, the way in which the processing units of the flowchart are divided and the processing order are not limited to the example shown in the figure, and may be changed as appropriate.

[0071] Furthermore, although the notification device of the present invention has been described as being realized by the communication terminal 1, which is a portable device, the present invention is not limited to this and may also be realized by the vehicle 100. Furthermore, although the notification device has been described as acquiring the biometric information Db from the wearable device 5, the notification device itself may have a function to measure the biometric information Db. Although the program 21 for realizing the notification method of the present invention has been described as being stored in the notification device, the program 21 may be acquired from an external device via communication. The program 21 may also be recorded on a computer-readable recording medium. The recording medium may be a magnetic or optical recording medium, a semiconductor memory device, or the like.

[0072] 6. Configurations Supported by the Above Embodiments The above embodiment is a specific example of the following configuration.

[0073] (Configuration 1) A notification device comprising: an intellectual function evaluation unit that evaluates multiple intellectual functions related to the user's physical condition based on the user's biometric information; an audio acquisition unit that acquires the user's audio information; an audio evaluation unit that evaluates audio analysis results related to the user's physical condition based on the audio information; and a notification control unit that notifies the user of notification information corresponding to the user's physical condition based on a combination of the evaluation results of the intellectual function evaluation unit and the evaluation results of the audio evaluation unit. According to this configuration, the user's physical condition can be detected with high accuracy, and notifications tailored to the user's physical condition can be provided.

[0074] (Configuration 2) The notification device described in Configuration 1, wherein the notification information includes advice related to the user's driving, and the voice acquisition unit performs a process of acquiring the user's voice information when it detects that the user is getting into a target vehicle. According to this configuration, it is possible to provide an appropriate notification that takes into consideration the physical condition of the driver immediately before driving.

[0075] (Configuration 3) The notification device according to configuration 1, wherein the notification control unit performs a first notification process when the intellectual function evaluation unit detects that the plurality of intellectual functions are good and the voice evaluation unit detects that the physical condition is good. According to this configuration, notifications can be made that are tailored to the state of good physical condition.

[0076] (Configuration 4) The notification device described in Configuration 3, wherein the notification control unit performs a second notification process to notify advice regarding driving operations as a process to notify information different from the first notification process when the intellectual function evaluation unit detects that the multiple intellectual functions are good and the voice evaluation unit detects that the physical condition is poor. According to this configuration, in the case of temporary or mild poor health, advice can be given to support driving caused by the poor health.

[0077] (Configuration 5) The notification device described in Configuration 4, wherein the notification control unit performs a third notification process to notify advice regarding physical condition as a process to notify information different from the first notification process and the second notification process when the number of intellectual functions detected as poor by the intellectual function evaluation unit is greater than the number of intellectual functions detected as good and the voice evaluation unit detects the physical condition as good. According to this configuration, when the user is not in very good physical condition but is not currently in a bad state, advice on how to take care of the user's physical condition can be given.

[0078] (Configuration 6) The notification device described in Configuration 5, wherein the notification control unit performs a fourth notification process to notify advice regarding physical condition and driving operation as a process to notify information different from the first to third notification processes when the number of intellectual functions detected as poor by the intellectual function evaluation unit is greater than the number of intellectual functions detected as good and the voice evaluation unit detects the physical condition as poor. According to this configuration, if the driver is not feeling well and is currently in poor condition, advice can be given to take care of both the driver's physical condition and driving.

[0079] (Configuration 7) The notification device described in Configuration 6, wherein the notification information includes advice for surrounding areas related to the user's driving, and when the fourth notification process is performed, the notification control unit performs a process of transmitting the advice for surrounding areas to the outside using wireless communication. According to this configuration, if a user who is in poor health is driving, a notification can be sent to alert those around them.

[0080] (Configuration 8) A notification device as described in Configuration 5 or 6, which is provided with a vehicle control unit that changes the driving assistance function of the target vehicle driven by the user to a predetermined setting at least when the number of intellectual functions detected as poor by the intellectual function evaluation unit is greater than the number of intellectual functions detected as good and the voice evaluation unit detects the physical condition as good, and when the number of intellectual functions detected as poor is greater than the number of intellectual functions detected as good and the voice evaluation unit detects the physical condition as poor. According to this configuration, when a user who is in poor physical condition is driving, driving assistance tailored to the user's poor physical condition can be provided.

[0081] (Configuration 9) A notification method executed by a notification device, comprising: an intellectual function evaluation step of evaluating multiple intellectual functions related to the user's physical condition based on the user's biometric information; an audio information acquisition step of acquiring the user's audio information; an audio information evaluation step of evaluating audio analysis results related to the user's physical condition based on the audio information; and a notification step of notifying notification information corresponding to the user's physical condition based on a combination of the evaluation results of the intellectual function evaluation step and the evaluation results of the audio information evaluation step. According to this method, the user's physical condition can be detected with high accuracy, and notifications tailored to the user's physical condition can be provided.

[0082] (Configuration 10) A program that causes a notification device controlled by a computer to function as an intellectual function evaluation unit that evaluates multiple intellectual functions related to the user's physical condition based on the user's biometric information, an audio acquisition unit that acquires the user's audio information, an audio evaluation unit that evaluates audio analysis results related to the user's physical condition based on the audio information, and a notification control unit that notifies the user of notification information corresponding to the user's physical condition based on a combination of the evaluation results of the intellectual function evaluation unit and the evaluation results of the audio evaluation unit. This program makes it possible to accurately detect the user's physical condition and provide notifications tailored to the user's physical condition. [Explanation of symbols]

[0083] 1...Communication terminal, 5...Wearable device, 6...Vital sensor, 7, 30...Communication unit, 10...Processor, 11...Information acquisition unit, 12...Intellectual function evaluation unit, 13...Riding detection unit, 14...Dialogue unit (voice acquisition unit), 15...Health condition related index calculation unit, 16...Notification control unit, 17...Vehicle control unit, 20...Memory, 21...Program, 22...Map information, 23...Biometric information database, 30...Communication unit, 31...Camera, 32...Acceleration sensor, 33...GNSS sensor, 3 4...display, 35...speaker, 36...microphone, 100...vehicle (target vehicle), 110...ECU, 121...navigation device, 130...front camera, 131...driver monitor camera, 132...display, 133a, 133b...speakers, 150...terminal holder, 160...wearable device, 200...communication network, 210...traffic information server, 211...service providing server, D...driver, Db...biometric information, Dv...driving behavior information.

Claims

1. an intellectual function evaluation unit that evaluates a plurality of intellectual functions related to the user's physical condition based on the user's biological information; a voice acquisition unit that acquires voice information of the user; a voice evaluation unit that evaluates a voice analysis result related to the user's physical condition based on the voice information; a notification control unit that notifies the user of notification information corresponding to the user's physical condition based on a combination of the evaluation result of the intellectual function evaluation unit and the evaluation result of the voice evaluation unit; A notification device comprising:

2. the notification information includes advice related to the user's driving; The voice acquisition unit performs a process of acquiring voice information of the user when it is detected that the user gets into a target vehicle. The notification device of claim 1 .

3. The notification control unit performs a first notification process when the intellectual function evaluation unit detects that the plurality of intellectual functions are good and the voice evaluation unit detects that the physical condition is good. The notification device of claim 1 .

4. When the intellectual function evaluation unit detects that the plurality of intellectual functions are good and the voice evaluation unit detects that the physical condition is poor, the notification control unit performs a second notification process of notifying advice regarding driving operation as a process of notifying information different from the first notification process. The notification device of claim 3 .

5. When the number of the intellectual functions detected as poor by the intellectual function evaluation unit is greater than the number of the intellectual functions detected as good, and the voice evaluation unit detects that the physical condition is good, the notification control unit performs a third notification process of notifying advice regarding physical condition as a process of notifying information different from the first notification process and the second notification process. The notification device of claim 4.

6. When the number of the intellectual functions detected as poor by the intellectual function evaluation unit is greater than the number of the intellectual functions detected as good, and the physical condition is detected as poor by the voice evaluation unit, the notification control unit performs a fourth notification process of notifying advice regarding physical condition and driving operation as a process of notifying information different from the first to third notification processes. The notification device of claim 5.

7. The notification information includes advice for surroundings related to the user's driving, When the fourth notification process is performed, the notification control unit performs a process of transmitting the advice to an external device by using wireless communication. The notification device of claim 6.

8. and a vehicle control unit that changes a driving assistance function of the target vehicle driven by the user to a predetermined setting at least when the number of intellectual functions detected as poor by the intellectual function evaluation unit is greater than the number of intellectual functions detected as good and the physical condition is detected as good by the voice evaluation unit, or when the number of intellectual functions detected as poor is greater than the number of intellectual functions detected as good and the physical condition is detected as poor by the voice evaluation unit.

7. The notification device according to claim 5 or 6.

9. A notification method performed by a notification device, comprising: an intellectual function evaluation step of evaluating a plurality of intellectual functions related to the user's physical condition based on the user's biological information; a voice information acquiring step of acquiring voice information of the user; a voice information evaluation step of evaluating a voice analysis result related to the user's physical condition based on the voice information; a notification step of notifying the user of notification information corresponding to the user's physical condition based on a combination of the evaluation result of the intellectual function evaluation step and the evaluation result of the voice information evaluation step; A notification method comprising:

10. a computer-controlled notification device; an intellectual function evaluation unit that evaluates a plurality of intellectual functions related to the user's physical condition based on the user's biological information; a voice acquisition unit that acquires voice information of the user; a voice evaluation unit that evaluates a voice analysis result related to the user's physical condition based on the voice information; a notification control unit that notifies the user of notification information corresponding to the user's physical condition based on a combination of the evaluation result of the intellectual function evaluation unit and the evaluation result of the voice evaluation unit; A program that makes it work.

Images