Driving assistance device, driving assistance method, and recording medium

Abstract

The present invention provides a driving assistance device, a driving assistance method, and a recording medium. The present invention realizes a technology that can quickly obtain information related to the state of a driver driving a mobile body through a simple structure. Output multiple questions for obtaining multiple driver information related to the life of the driver of the mobile body; based on the driver's answers to each of the multiple questions, obtain the multiple driver information; based on the multiple driver information, classify the driver status related to at least one of the driver's psychological state and physical state; based on the classification result of the driver status, generate auxiliary information containing suggestions related to driving; output the auxiliary information.

Description

Technical Field

[0001] The present invention relates to a driving assistance device, a driving assistance method and a recording medium. Background Art

[0002] In recent years, efforts have been actively made to allow disadvantaged people among traffic participants to use sustainable transportation systems that take them into consideration. To achieve this, research and development related to driver assistance technology is being devoted to further improving the safety and convenience of traffic. For example, Patent Document 1 discloses the following technology: biological information of a vehicle driver is acquired through a biometric sensor, the driver's status is classified by analyzing the biological information, and information is provided to the driver based on the classification result.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: International Publication No. WO2021 / 014632 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, in driving assistance technology, it is preferred to estimate or classify the driver's state quickly. Therefore, compared with the conventional method that requires complicated sensors, it is desired to quickly obtain information related to the driver with a simple structure.

[0008] In order to solve the above-mentioned problems, the present application aims to realize a technology that can quickly obtain information about the state of a driver driving a mobile body with a simple structure, and further contribute to the development of a sustainable transportation system.

[0009] Means for solving problems

[0010] One embodiment of the present invention is a driving assistance device that performs the following processing: outputs a plurality of questions for obtaining a plurality of driver information related to the life of a driver of a mobile body; obtains the plurality of driver information based on the driver's answers to each of the plurality of questions; classifies the driver's state related to at least any one of the driver's psychological state and physical state based on the plurality of driver information; generates auxiliary information including suggestions related to driving based on the classification result of the driver's state; and outputs the auxiliary information.

[0011] Effects of the Invention

[0012] According to one embodiment of the present invention, information on the state of a driver driving a mobile object can be quickly obtained with a simple structure, so that the state of the driver can be quickly classified, the driver can be assisted, and further, the development of a sustainable transportation system can be contributed to. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a diagram showing a configuration example of a system including a mobile terminal in the first embodiment.

[0014] Figure 2 This is a block diagram showing a configuration example of the main parts of the system.

[0015] Figure 3 It is a diagram showing a structural example of driver information.

[0016] Figure 4 It is a diagram showing an example of display setting data.

[0017] Figure 5 It is a flowchart showing an example of the operation of a mobile terminal.

[0018] Figure 6 This is a flowchart showing an example of the operation of a mobile terminal.

[0019] Figure 7 The figure shows a display example in a mobile terminal.

[0020] Figure 8 It is a diagram showing an example of display setting data in the second embodiment.

[0021] Fig. 9 It is a diagram showing an example of display setting data in the second embodiment.

[0022] Fig.10 This is a flowchart showing an example of the operation of the mobile terminal in the second embodiment.

[0023] Description of Reference Numerals

[0024] 1…Mobile terminal (driving assistance device), 2…Wearable device (other device), 3…Communication system, 5…Server, 10…Control unit, 100…Processor, 101…Acquisition unit, 102…Reception unit, 103…State determination unit, 104…Assistance information generation unit, 105…Output control unit, 120…Memory, 121…Control program, 122…Application (program), 123…Setting data, 124…Assistance database, 124A, 124B, 124C…Display setting data, 125, 125A, 125B, 125C, 125D…Driver information, 126…Assistance information, 130…First communication unit (communication unit), 131… Second communication unit, 132...display (display unit, output unit), 133...touch sensor, 134...microphone, 135...speaker (output unit), 150, 150A~150D...display object, 151...first object, 152...second object, 153...third object, 155A~155M...display object, 170, 170A~170F...screen, 181, 181A~181F...vehicle object, 182...start indication unit, 183, 184...conversion indication unit, 185...speaking display, 186...speaking guidance, 187...answer guidance, M...vehicle (moving body), NW...communication network, U...user (driver). DETAILED DESCRIPTION

[0025] [1. First embodiment]

[0026] Reference Figure 1 to Figure 7 , a first embodiment to which the present invention is applied is described.

[0027] [1-1. Configuration of system including mobile terminal]

[0028] Figure 1 It is a diagram showing a configuration example of a system including the mobile terminal 1.

[0029] The mobile terminal 1 is a device used by a user U who drives a vehicle M. The user U is a driver of the vehicle M and can use the mobile terminal 1 before getting on the vehicle M. The mobile terminal 1 is, for example, a portable computer, specifically, a smartphone, a tablet computer, a notebook computer, or other computers.

[0030] The mobile terminal 1 includes a display 132 and a speaker 135 (described later). Figure 2 ). The mobile terminal 1 may output information to the user U by displaying an image or text on the display 132 and outputting a voice through the speaker 135 .

[0031] The mobile terminal 1 outputs a question to the user U before the user U drives the vehicle M. For example, the mobile terminal 1 outputs a question before the user U gets on the vehicle M. The content of the question is a question used to obtain information related to the life of the user U, which will be described in detail later. When the user U inputs an answer to the question, the mobile terminal 1 estimates the state of the user U based on the answer of the user U and classifies the state of the user U. The mobile terminal 1 generates auxiliary information based on the classified state of the user U. The auxiliary information is information that assists the user U in driving the vehicle M. The mobile terminal 1 assists the user U in driving the vehicle M by outputting the auxiliary information from at least one of the display 132 and the speaker 135.

[0032] The mobile terminal 1 performs wireless communication with a wearable device (hereinafter referred to as WD) 2 used by a user U. The mobile terminal 1 and the WD 2 together constitute a communication system 3 .

[0033] WD2 is a device worn on the body or clothes of user U. WD2 is, for example, an electronic device of watch type, dongle type, glasses type, or other shapes. WD2 uses various sensors described later to perform detections related to user U. For example, WD2 detects biological information of user U and body movements of user U. As biological information of user U, pulse, blood pressure, respiratory rate, blood sugar level, sweating status, and other vital data can be listed. Based on the detection results of the sensor, WD2 generates information related to the life of user U. Information related to the life of user U is, for example, the sleep time of user U, the walking amount of user U, the activity amount of user U, the nervous state of user U, and various information called so-called life logs. The information generated by WD2 can be collectively referred to as information related to the life behavior of user U. WD2 communicates with mobile terminal 1 to send information related to the life behavior of user U to mobile terminal 1.

[0034] When acquiring information related to the life behavior of the user U from the WD2 , the mobile terminal 1 can estimate and classify the state of the user U based on the information acquired from the WD2 .

[0035] like Figure 1 As shown, the mobile terminal 1 may also be configured to be able to communicate with the server 5 via the communication network NW.

[0036] The communication network NW is a communication network composed of a public line network, a dedicated line, other communication circuits, etc. The server 5 is a computer that sends and receives data with various devices via the communication network NW. The server 5 can be a single server computer, or it can be composed of multiple server computers, or it can be a cloud server. For example, in the process of generating auxiliary information, the mobile terminal 1 performs data communication with the server 5 as needed.

[0037] [1-2. Structure of mobile terminal]

[0038] Figure 2 It is a block diagram showing a configuration example of the communication system 3 , and shows the configurations of the mobile terminal 1 and the WD 2 .

[0039] The mobile terminal 1 includes a control unit 10 , and includes a first communication unit 130 , a second communication unit 131 , a display 132 , a touch sensor 133 , a microphone 134 , and a speaker 135 that are controlled by the control unit 10 .

[0040] The first communication unit 130 and the second communication unit 131 are wireless communication devices having a transmitter for sending data and a receiver for receiving data. The first communication unit 130 performs wireless data communication with a device other than the mobile terminal 1 through Bluetooth (registered trademark), Wi-Fi (registered trademark) or other short-range wireless communication methods. For example, the first communication unit 130 performs communication with WD2. In this case, WD2 is an example of another device.

[0041] The second communication unit 131 performs wireless data communication using a different communication method from the first communication unit 130. The second communication unit 131 is connected to the communication network NW by, for example, cellular communication, and performs data communication with the server 5 via the communication network NW. The first communication unit 130 is an example of a communication unit.

[0042] The display 132 has a display screen formed of a liquid crystal display panel or an organic EL (Electro Luminescence) panel. The display 132 displays various screens including images and characters according to the control of the control unit 10 .

[0043] The touch sensor 133 is a sensor that detects a contact operation by the user U. In the present embodiment, the touch sensor 133 is arranged to overlap with the display 132, and is integrated with the display 132 to form a touch panel. The touch sensor 133 detects an operation by the user U touching the surface of the display 132, and outputs operation data indicating the position of the detected operation to the control unit 10.

[0044] The microphone 134 detects voice and generates a voice signal, converts the generated voice signal into digital voice data, and outputs the digital voice data to the control unit 10. The control unit 10 can detect the voice of the user U through the microphone 134, for example.

[0045] The speaker 135 outputs voice based on a voice signal or digital voice data input from the control unit 10 .

[0046] The display 132 is an example of a display unit. The display 132 and the speaker 135 function as an output unit for the mobile terminal 1 to output. The touch sensor 133 and the microphone 134 are used as an input unit for the mobile terminal 1 to receive input from the user U.

[0047] The control unit 10 includes a processor 100 and a memory 120. The processor 100 is a computer composed of a CPU (Central Processing Unit), an MPU (Micro Processing Unit) or other integrated circuits. The memory 120 is a storage device (recording medium) for storing programs and data. The processor 100 can use a volatile RAM (Random Access Memory) as a working area. The RAM can be integrated into the processor 100 for installation, or the memory 120 can be configured to have the RAM.

[0048] The memory 120 is a rewritable non-volatile storage device that stores programs executed by the processor 100 and data processed by the processor 100. The memory 120 is composed of, for example, a semiconductor storage device such as a flash ROM (Read Only Memory) or an SSD (Solid State Disk), or a magnetic storage device. The memory 120 stores a control program 121, an application program (hereinafter referred to as an application) 122, setting data 123, an auxiliary database (hereinafter referred to as a DB) 124, driver information 125A, 125B, 125C, 125D, and auxiliary information 126.

[0049] The control program 121 and the application 122 are programs executed by the processor 100 and stored in the memory 120 in a manner readable by the processor 100. The control program 121 is a basic control program for the processor 100 to control each part of the mobile terminal 1 and is an OS (Operating System). The application 122 is an application program executed on the OS.

[0050] The setting data 123 is data that is referred to when the processor 100 executes the application 122. The setting data 123 includes setting values ​​related to the functions of the application 122 and the like.

[0051] The auxiliary DB 124 is a database that stores image data and the like used by the auxiliary information generating section 104 described later.

[0052] The driver information 125 is information acquired by the acquisition unit 101 .

[0053] The auxiliary information 126 is information generated by the auxiliary information generating unit 104 .

[0054] The processor 100 includes an acquisition unit 101, a reception unit 102, a state determination unit 103, an auxiliary information generation unit 104, and an output control unit 105 as functional units. These functional units are implemented by the processor 100 executing an application 122. The application 122 is an example of a program.

[0055] The acquisition unit 101 acquires information related to the life of the user U. The reception unit 102 receives input from the user U. Specifically, when the touch sensor 133 detects a touch operation, the reception unit 102 determines the input content based on the touch operation and generates input data representing the determined content. In addition, the reception unit 102 analyzes the voice of the user U detected by the microphone 134 and performs text conversion processing, thereby generating input data representing the content of the user U's speech.

[0056] The acquisition unit 101 outputs a question to the user U by displaying the display 132 and outputting a voice through the speaker 135. The acquisition unit 101 acquires an answer to the question based on the input data received by the receiving unit 102 after the question is output, and generates driver information 125A, 125B, 125C, and 125D based on the acquired answer. Hereinafter, the driver information 125A, 125B, 125C, and 125D are recorded as driver information 125 without distinguishing them. In the present embodiment, an example in which the mobile terminal 1 acquires four driver information 125A, 125B, 125C, and 125D through the acquisition unit 101 is described, but there is no limit to the number of driver information 125 acquired by the acquisition unit 101 and the number of driver information 125 stored in the memory 120.

[0057] Figure 3 2 is a diagram showing a configuration example of the driver information 125 .

[0058] In this embodiment, the driver information 125A is information about the sleep state of the user U, and the driver information 125B is information about the exercise frequency of the user U. The driver information 125C is information about the fatigue of the user U, and the driver information 125D is information about the tension of the user U. The driver information 125A, 125B, 125C, and 125D may be set as driver information A, B, C, and D, respectively.

[0059] There is no limitation on the value that each of the driver information 125A to 125D can take. In the present embodiment, each of the driver information 125A to 125D has any value of A, B, or C.

[0060] Specifically, the driver information 125A is information about the most recent sleep of the user U based on the date when the driver information 125A is generated. The value A of the driver information 125A indicates that the user U has not slept, and the value B indicates that the user has slightly insufficient sleep. The value C of the driver information 125A indicates that the user has had sufficient sleep, that is, is sleeping.

[0061] Specifically, the driver information 125B is information about the number of days in a week in which exercise is performed, and exercise refers to, for example, an amount of exercise equivalent to walking for more than 20 minutes, or an amount of exercise equivalent to more than 20 minutes. The value A of the driver information 125B indicates the number of days in which no exercise is performed, that is, the number of days in which exercise is performed is 0 days in a week. The value B of the driver information 125B indicates that the number of days in which exercise is performed is 1 day or 2 days in a week, and the value C indicates that the number of days in which exercise is performed is more than 3 days in a week.

[0062] Specifically, the driver information 125C is information about the fatigue level of the user U at the time when the driver information 125C is generated. The value A of the driver information 125C indicates that the user U is fatigued, and the value C indicates that the user U is alert. The value B of the driver information 125C is ordinary, that is, it indicates a state that is neither fatigued nor alert.

[0063] Specifically, the driver information 125D is information about whether the user U is tense or relaxed at the time when the driver information 125D is generated. The value A of the driver information 125D indicates that the user U is tense, and the value C indicates that the user U is relaxed. The value B of the driver information 125D is normal, that is, it indicates that the user U is neither tense nor relaxed.

[0064] The acquisition unit 101 outputs a question for specifying the value of each of the driver information 125A to 125D. For example, the acquisition unit 101 outputs a question for specifying whether the value of the driver information 125A is A, B, or C. The same applies to the driver information 125B to 125D.

[0065] After outputting the question, the acquisition unit 101 analyzes the input content received by the reception unit 102 to identify the value of each of the plurality of driver information 125A to 125D included in the driver information 125. Identifying the value of the driver information 125 is equivalent to generating the driver information 125.

[0066] The acquisition unit 101 receives and acquires information related to life behavior from WD2 when data communication with WD2 is possible through the first communication unit 130. When the information acquired from WD2 corresponds to any one of the driver information 125A to 125D, the acquisition unit 101 determines the value of the driver information 125 based on the acquired information. The acquisition unit 101 omits outputting questions for the driver information 125 whose value is determined based on the information acquired from WD2. For example, when the acquisition unit 101 determines the value of the driver information 125A based on the information acquired from WD2, the question corresponding to the driver information 125A is not output. When the acquisition unit 101 determines the values ​​of all the driver information 125 based on the information acquired from WD2, the question to the user U is not output.

[0067] The state determination unit 103 classifies the driver state related to the state of the user U based on the driver information 125. The driver state is a state related to at least one of the psychological state and physical state of the user U. For example, the driver state is an index reflecting the attention or cognitive function of the user U, the perception ability of the user U, the operation ability of the user U, the judgment of the user U, etc. The driver state can be a plurality of index values ​​reflecting the attention or cognitive function of the user U, the perception ability of the user U, the operation ability of the user U, the judgment of the user U, etc., respectively, or it can be an index value synthesized from them. The index value of the driver state can be expressed by a plurality of pre-set stages, or it can be expressed by a numerical value. In the present embodiment, the state determination unit 103 is an index value indicating whether the driver state of the user U is a positive state, a negative state, or an intermediate state between the positive state and the negative state. That is, the state determination unit 103 expresses the driver state by three-stage indexes. This is an example, for example, the state determination unit 103 can also classify the driver state into four or more stages including a positive state, a negative state, and an intermediate state. Specifically, the state determination unit 103 can be classified into five stages, namely, a positive state, a negative state, an intermediate state between the positive state and the negative state, a quasi-positive state between the positive state and the intermediate state, and a quasi-negative state between the intermediate state and the negative state, or classification of four or more other stages can be performed.

[0068] Furthermore, the state determination unit 103 can estimate the change of the driver's state. For example, the time point at which the driver information 125 stored in the memory 120 is generated, or the time point at which the answer for generating the driver information 125 is received is set as the first time point, and the time point after the prescribed time from the first time point is set as the second time point. As described above, the state determination unit 103 classifies the driver's state reflecting the state of the user U at the first time point. Furthermore, the state determination unit 103 can estimate the state of the user U at the second time point and classify the estimated driver's state. The length of the prescribed time is not limited. For example, the prescribed time can be set to within 1 hour so that the user U continues to drive the vehicle M at the second time point. In addition, the prescribed time can also be set to about 6 hours to 12 hours, in which case, the change of the physical condition of the user U during the day can be estimated. The driver's state at the first time point among the driver's states classified by the state determination unit 103 is set as the first driver's state, and the driver's state at the second time point is set as the second driver's state.

[0069] As an example, assume that the values ​​of driver information 125A and 125B are both A, the value of driver information 125C is B, and the value of driver information 125D is C. In this example, the user U is physically tired but mentally relaxed. Therefore, the state determination unit 103, for example, classifies the first driver state as an intermediate state. In this example, the user U is in an intermediate state when starting to drive the vehicle M, but it is assumed that fatigue accumulates during driving, so the state determination unit 103 classifies the second driver state as a passive state. In this case, the driver state of the user U deteriorates from the intermediate state to the passive state.

[0070] As another example, assume that the values ​​of driver information 125A and 125B are both C, the value of driver information 125C is B, and the value of driver information 125D is A. In this example, the user U is in a state of less physical fatigue, but is mentally tense. Therefore, the state determination unit 103, for example, classifies the first driver state as an intermediate state. In this example, it is assumed that the user U is in an intermediate state when starting to drive the vehicle M, but the tension is relieved and relaxed during driving, so the state determination unit 103 classifies the second driver state as an active state. In this case, the driver state of the user U is optimized from the intermediate state to the active state.

[0071] The method by which the state determination unit 103 estimates and classifies the first driver state and the second driver state is not limited. For example, a table that associates the combination of the values ​​of the driver information 125A to 125D, the classification of the first driver state, and the classification of the second driver state may be used. The table may be pre-stored in the memory 120 as the setting data 123, for example.

[0072] Furthermore, for example, the state determination unit 103 may also use a function or a program to obtain the driver state based on the driver information 125. That is, the mobile terminal 1 has a function or a program for obtaining the classification of the first driver state based on the combination of the values ​​of the driver information 125A to 125D. Similarly, the mobile terminal 1 has a function or a program for obtaining the classification of the second driver state based on the combination of the values ​​of the driver information 125A to 125D. These functions or programs can be pre-stored in the memory 120 as the setting data 123, for example.

[0073] Furthermore, for example, the state determination unit 103 may also obtain the driver state by calculating the value of the driver information 125. That is, the mobile terminal 1 replaces the value of each driver information 125 with a score that can be calculated, such as a numerical value, and sums up the scores of the plurality of driver information 125. The mobile terminal 1 converts the sum of the scores into the driver state. In this configuration, data that associates the scores with the driver states is pre-stored in the memory 120 as, for example, the setting data 123.

[0074] Furthermore, for example, the state determination unit 103 may also have a learning model that associates the value of the driver information 125 with the driver state. That is, the mobile terminal 1 may also have a learned AI (Artificial Intelligence) model that has learned the correlation between the value of the driver information 125 and the classification of the driver state. The learned model has learned the correlation between the combination of the values ​​of the driver information 125A to 125D and the classification of the first driver state, and the correlation between the combination of the values ​​of the driver information 125A to 125D and the classification of the second driver state. Furthermore, it may be configured so that when a specified time is specified by the learned model, the second driver state after the specified time has passed can be estimated. The learned model and teacher data can be pre-stored in the memory 120 as setting data 123, for example.

[0075] The various examples described above are examples in which the state determination unit 103 classifies the first driver state and the second driver state separately. This is an example, and for example, the state determination unit 103 may also classify the first driver state and the second driver state in a combined manner. Specifically, the state determination unit 103 may also estimate a combination of the classification of the first driver state and the classification of the second driver state. For example, the state determination unit 103 may also be a structure that classifies an indicator that is a combination of the first driver state and the second driver state. In this case, the state determination unit 103 does not perform the process of classifying the first driver state and the second driver state separately.

[0076] The auxiliary information generating unit 104 generates auxiliary information 126 based on the result of the classification of the driver's state by the state determining unit 103 , and stores the auxiliary information 126 in the memory 120 .

[0077] The auxiliary information 126 is information output by the mobile terminal 1 to the user U. The auxiliary information 126 can include still images, moving images, audio, or other contents. In this embodiment, the auxiliary information 126 including still images and audio is exemplified.

[0078] The auxiliary information 126 includes a message consisting of a character string. The message is output by display or voice. The message is information that assists the user U in driving the vehicle M, and can be called advice. Specifically, it is matters that the user U should pay attention to when driving the vehicle M. More specifically, the advice included in the auxiliary information 126 is advice on points of attention in cognition, judgment, and operation during driving.

[0079] The assist information 126 includes a still image or a moving image displayed on the display 132. The image included in the assist information 126 expresses at least one of the driver's state and information advising the user U to drive the vehicle M.

[0080] The output form of the auxiliary information 126 and the amount of information of the auxiliary information 126 are designed to reduce the cognitive load of the user U. Specifically, the auxiliary information 126 is configured to be suitable for the user U to recognize and remember the content of the auxiliary information 126.

[0081] In order to output the auxiliary information 126 in a form that easily remains in the memory of the user U, the information displayed on the display 132 is preferably four or less, more preferably two or less. In addition, the information output by voice from the speaker 135 is also preferably four or less, more preferably two or less.

[0082] In this embodiment, in order to reduce the amount of information of the auxiliary information 126 displayed on the display 132, the auxiliary information 126 including an image imitating the weather forecast is generated. The weather forecast is suitable for making people recognize both the state at a certain point in time and the state after a certain time, and their changes. In addition, many people have a common understanding of the weather forecast in society. Specifically, it can be considered that the user U generally recognizes that the weather forecast is a weather forecast that represents the outline of the phenomenon with an image, and the reliability of the weather forecast is limited. In addition, since the weather is something that many people care about, the weather forecast will attract the attention of many people.

[0083] In the case where the driver's state is represented by an image that imitates a weather forecast, it is expected that the user U will pay attention to the image, and it is allowed even if the user U's conscious physical state is different from the driver's state. In addition, since many people are familiar with weather forecast images, by using images that imitate weather forecasts to represent the driver's state, there is an advantage that the user U can quickly understand his or her own state. In the case where such images are displayed on the display 132 in a number of four or less or two or less, it is easy for the user U to store them while driving the vehicle M and to recall them while driving. Therefore, it is possible to give the user U appropriate advice on the driving of the vehicle M, and effectively assist the user U in driving. In addition, if the message included in the auxiliary information 126 is set as an explanation associated with the image that imitates the weather forecast, it is possible to make suggestions that are more likely to remain in the user U's consciousness.

[0084] The auxiliary DB 124 stores messages and images used to generate the auxiliary information 126. The auxiliary information generation unit 104 obtains messages and images corresponding to the first driver state and the second driver state from the auxiliary DB 124. The storage method of the information in the auxiliary DB 124 is not limited. For example, the auxiliary DB 124 stores messages and images corresponding to the combination of the first driver state and the second driver state.

[0085] Figure 4 1 is a diagram showing an example of the display setting data 124A. The display setting data 124A is data stored in the support DB 124, and includes display objects 150 corresponding to the first driver state and the second driver state.

[0086] exist Figure 4 In the example of , the display setting data 124A associates a combination of the first driver state and the second driver state with the display object 150. Figure 4 150A, 150B, 150C, and 150D are illustrated in the example, but the number of display objects 150 is not limited. For example, the display setting data 124A associates one display object 150 with a combination of a first driver state and a second driver state. Hereinafter, when display objects 150A, 150B, 150C, and 150D are not distinguished, they are described as display objects 150.

[0087] As described above, the display object 150 of the present embodiment is a metaphor for weather forecast. The display object 150 is a set of images including a first object 151 indicating a sunny day, a second object 152 indicating rain, and a third object 153 indicating cloudy days. For example, in the display setting data 124A, the display object 150A corresponds to a combination of a first driver state being a positive state and a second driver state being a positive state. The display object 150A is composed of the first object 151 indicating a sunny day, indicating that the driver state of the user U is maintained in a good state.

[0088] For example, in the display setting data 124A, the display object 150B corresponds to a combination in which the first driver state is a positive state and the second driver state is an intermediate state. The display object 150B is composed of a combination of the first object 151 and the third object 153, and indicates the weather of "sunny with occasional clouds". That is, it indicates that the driver state of the user U is initially good, but may deteriorate later.

[0089] The mobile terminal 1 of this embodiment expresses the driver status of the user U using seven weather metaphors, for example, clear, sunny, sunny to cloudy, cloudy to clear, cloudy, cloudy to rainy, rainy to cloudy, and rain. That is, the display setting data 124A associates the seven weather display objects 150 with the driver status.

[0090] like Figure 4 As shown, the display setting data 124A may be in a table form including image data of the display object 150, or may be a list or function including data specifying the image data of the display object 150. In this case, the image data of the display object 150 may be stored in the auxiliary DB 124 separately from the display setting data 124A.

[0091] The auxiliary information generating unit 104 acquires image data of the display object 150 corresponding to the driver's state based on the display setting data 124A, and generates auxiliary information 126 including the image data. In addition, the auxiliary information generating unit 104 may include display objects composed of images such as icons in the auxiliary information 126 in addition to the display object 150.

[0092] The output control unit 105 outputs the auxiliary information 126. The output control unit 105 outputs the character string included in the auxiliary information 126 from the speaker 135 by voice, and causes the display 132 to display the display object 150 included in the auxiliary information 126. The auxiliary information 126 may also include information specifying an output method. For example, the auxiliary information 126 may also include information specifying that the output method of the character string included in the auxiliary information 126 is either output based on voice or display based on the display 132. In this case, the output control unit 105 outputs the auxiliary information 126 in the output method specified by the auxiliary information 126.

[0093] The output control unit 105 preferably outputs the auxiliary information 126 before the user U drives the vehicle M. For example, the application 122 causes the display 132 to display a message requesting the user U to operate the application 122 before driving the vehicle M. In the case of a structure capable of determining whether the mobile terminal 1 is located inside the vehicle M, the control unit 10 may be configured to activate the application 122 only while the mobile terminal 1 is located outside the vehicle M. This structure can be implemented as a structure in which the mobile terminal 1 can communicate with an unillustrated vehicle-mounted device mounted on the vehicle M through short-range wireless communication, and determines whether the mobile terminal 1 is located inside the vehicle M by detecting the communication state between the vehicle-mounted device and the mobile terminal 1.

[0094] [1-3 Structure of wearable devices]

[0095] The WD (wearable device) 2 includes a control unit 21 , a biosensor 22 controlled by the control unit 21 , a sensor unit 23 , and a communication unit 24 .

[0096] The control unit 21 includes a processor and a memory (not shown). The processor constituting the control unit 21 is a computer composed of a CPU, an MPU, or other integrated circuits. The memory is a storage device that stores programs executed by the processor and data processed by the processor. The control unit 21 may be an integrated circuit (IC (integrated circuit)) that integrates a processor, a memory, and other peripheral circuits.

[0097] The biosensor 22 is a sensor that detects the biometric information of the user U. The biosensor 22 detects, for example, pulse, blood pressure, blood oxygen concentration, respiratory rate, blood sugar level, sweating status, and other vital data. Specifically, the biosensor 22 is a heart rate sensor, and may also be a blood pressure sensor, a blood oxygen concentration sensor, a sweating sensor, etc. The sensor unit 23 is a unit that integrates sensors for detecting the movement of WD2. The sensor unit 23 includes an acceleration sensor, a rotational angular acceleration sensor or a gyroscope sensor, a magnetic field sensor, etc. The sensor unit 23 may also include a sensor for measuring the external environment, specifically, an air pressure sensor, and a temperature sensor. The sensor unit 23 may also be an inertial measurement unit (IMU: Inertial Measurement Unit) that combines these sensors into one package.

[0098] The communication unit 24 is a wireless communication device having a transmitter for sending data and a receiver for receiving data. The communication unit 24 performs wireless data communication with devices other than the WD2 through Bluetooth (registered trademark), Wi-Fi (registered trademark) or other short-range wireless communication methods. For example, the communication unit 24 performs communication with the mobile terminal 1.

[0099] WD2 may also include a display unit composed of a liquid crystal display panel, an organic EL panel, or an LED (Light Emitting Diode). In addition, WD2 may also include an input unit such as a switch or a touch sensor.

[0100] As described above, the control unit 21 generates information related to the life behavior of the user U based on at least one of the detection results of the biosensor 22 and the detection results of the sensor unit 23. For example, the control unit 21 generates information such as the sleep time of the user U, the walking amount of the user U, the activity amount of the user U, the fatigue level of the user U, and the nervous state of the user U.

[0101] The control unit 21 is connected to the mobile terminal 1 via wireless communication through the communication unit 24, and when information transmission is requested from the mobile terminal 1, the control unit 21 transmits information related to the life behavior of the user U to the mobile terminal 1. In addition, the control unit 21 may also transmit the detection results of the biosensor 22 and the detection results of the sensor unit 23 in response to the request of the mobile terminal 1.

[0102] [1-4. Actions of mobile terminal]

[0103] Figure 5 and Figure 6 This is a flowchart showing an example of the operation of the mobile terminal 1. Figure 6 Detailed description Figure 5 Step S11.

[0104] Figure 5 Steps S12 to S14 of the operation are executed by the state determination unit 103, steps S15 to S17 are executed by the auxiliary information generation unit 104, and step S18 is executed by the output control unit 105. Figure 6 Steps S21 to S27 and S29 to S31 in the operation are executed by the acquisition unit 101, and step S28 is executed by the reception unit 102.

[0105] The mobile terminal 1 acquires the driver information 125 (step S11 ).

[0106] In detail, Figure 6As shown, the mobile terminal 1 determines whether there is a device that can communicate via the first communication unit 130 (step S21). If there is no device that can communicate via the first communication unit 130 (step S21: No), the mobile terminal 1 proceeds to step S26 described later.

[0107] If there is a device that can communicate via the first communication unit 130 (step S21: Yes), the mobile terminal 1 attempts to obtain information from the device (step S22). For example, in step S22, the mobile terminal 1 communicates with WD2, requests WD2 for information related to the life behavior of the user U, and obtains information sent by WD2 in response to the request.

[0108] The mobile terminal 1 determines whether all the driver information 125 can be acquired through the operation of step S22 (step S23). In step S23, if the mobile terminal 1 successfully receives the information through the first communication unit 130 and the received information is information that can be used as the driver information 125, the mobile terminal 1 determines that the driver information 125 can be acquired. If the mobile terminal 1 can acquire all the required driver information 125 (step S23: Yes), the operations of steps S24 to S31 are omitted and the process is transferred to step S24. Figure 5 Step S12.

[0109] In the case where the driver information 125 that the mobile terminal 1 can obtain in step S22 is a part of the required driver information 125, and in the case where the driver information 125 cannot be obtained, a negative determination is made in step S23 (step S23: No). In this case, the mobile terminal 1 determines whether a part of the required driver information 125 can be obtained in step S22 (step S24).

[0110] When the mobile terminal 1 cannot acquire the driver information 125 in step S22 (step S24 : No), the process proceeds to step S26 described later.

[0111] When the mobile terminal 1 can acquire part of the required driver information 125 in step S22 (step S24 : Yes), the acquired driver information 125 is excluded from the question objects (step S25 ), and then the process shifts to step S26 .

[0112] The mobile terminal 1 selects the driver information 125 to be questioned in step S26 (step S26). The mobile terminal 1 outputs a question for obtaining the selected driver information 125 (step S27). Next, the mobile terminal 1 outputs the question, for example, by at least one of display on the display 132 and voice output from the speaker 135 (step S27).

[0113] The mobile terminal 1 receives the input of the answer to the question (step S28). The mobile terminal 1 determines whether the answer received in step S28 is an appropriate answer (step S29). An appropriate answer is an answer that can determine any value of A, B, or C for the driver information 125 selected in step S26.

[0114] If the answer received in step S28 is not an appropriate answer (step S29: No), the mobile terminal 1 returns to step S27. If the answer received in step S28 is an appropriate answer (step S29: Yes), the mobile terminal 1 determines the value of the driver information 125 determined based on the answer (step S30). Thus, the mobile terminal 1 has acquired the driver information 125 selected in step S26.

[0115] The mobile terminal 1 determines whether all the driver information 125 can be obtained (step S31). All the driver information 125 in step S31 refers to the driver information 125 that should be obtained by asking questions at the time of starting step S26. If there is driver information 125 that has not been obtained (step S31: No), the mobile terminal 1 returns to step S26. If all the driver information 125 can be obtained (step S31: Yes), the mobile terminal 1 transfers to Figure 5 Step S12.

[0116] The mobile terminal 1 analyzes the driver information 125 acquired in step S11 (step S12 ), classifies the first driver status (step S13 ), and classifies the second driver status (step S14 ).

[0117] The mobile terminal 1 acquires the display objects 150 corresponding to the first driver state and the second driver state from the auxiliary DB 124 (step S15 ). In addition, the mobile terminal 1 acquires the messages corresponding to the first driver state and the second driver state from the auxiliary DB 124 (step S16 ).

[0118] The mobile terminal 1 generates auxiliary information 126 including the display object 150 and the message acquired in steps S15 and S16 (step S17). The mobile terminal 1 performs speech output from the speaker 135 and display on the display 132 based on the auxiliary information 126 (step S18).

[0119] The auxiliary DB 124 may store messages corresponding to the respective display objects 150. In this case, the state determination unit 103 acquires the messages corresponding to the display objects 150 from the auxiliary DB 124 in step S16. Alternatively, the auxiliary DB 124 may store messages corresponding to the combinations of values ​​of the driver information 125. In this case, the state determination unit 103 acquires the messages corresponding to the combinations of values ​​of the driver information 125 from the auxiliary DB 124 in step S16.

[0120] Figure 7 1 is a diagram showing a display example in the mobile terminal 1. Figure 7 1 shows transitions of screens 170A, 170B, 170C, 170D, 170E, and 170F constituting screen 170 as an example of screen 170 displayed on display 132. Hereinafter, screens 170A to 170F are referred to as screen 170 when not being distinguished.

[0121] After the application 122 is started in the mobile terminal 1, the mobile terminal 1 displays the screen 170A while waiting for the operation of the user U. The screen 170A includes a vehicle object 181A. The vehicle object 181A is an image of a character displayed in the screen 170 in a manner of communicating with the user U, for example, Figure 7 The vehicle object 181A may be an icon that does not change, or may be a still image or an animation. The vehicle object 181A of the present embodiment is displayed as vehicle objects 181B to 181F having expressions or impressions different from those of the vehicle object 181A in the screens 170B to 170F. When collectively referring to the vehicle objects 181A to 181F, they are recorded as vehicle objects 181. The vehicle object 181 is a character constituting the conversational user interface of the application 122, and has the role of being a conversation partner of the user U.

[0122] The screen 170A is a screen displayed before the user U starts using the application 122 , and therefore the vehicle object 181A is an image that simulates a sleeping state of the character.

[0123] Screen 170A includes start instruction portion 182. Start instruction portion 182 is an image displayed on display 132 and is an operation portion for user U to instruct start of use of application 122. When a touch operation on start instruction portion 182 is detected, mobile terminal 1 switches the display from screen 170A to screen 170B.

[0124] Screen 170B is displayed while processor 100 is preparing to execute the function of application 122 in response to the operation of start instruction unit 182. For example, it is displayed while processor 100 is loading data related to application 122 or initializing the working area of ​​memory 120. Vehicle object 181B arranged on screen 170B is an image imitating the appearance of a character waking up.

[0125] Screen 170B includes transition instruction unit 183 and transition instruction unit 184. Transition instruction units 183 and 184 are images displayed on display 132. Transition instruction unit 183 is an operation unit that instructs to return the display of display 132 to the previous screen, and transition instruction unit 184 is an operation unit that instructs to switch to the screen immediately after application 122 is started. For example, when transition instruction unit 183 is operated in the display of screen 170B, the display of display 132 returns to screen 170A. In addition, for example, when transition instruction unit 184 is operated in screen 170B and screens 170C to 170F described later, the display of display 132 returns to screen 170A.

[0126] When the execution preparation of the function of the application 122 is completed in the display of the screen 170B, the mobile terminal 1 switches the display of the display 132 from the screen 170B to the screen 170C, and outputs the voice of the question. This stage corresponds to Figure 6 Step S27. Screen 170C is displayed while the mobile terminal 1 is outputting the voice of the question. Screen 170C is configured with a vehicle object 181C, transition indicator units 183 and 184, and a speaking display 185. The speaking display 185 is an indicator indicating that the mobile terminal 1 is outputting a voice. The speaking display 185 may also be an animation. The vehicle object 181C may also be an image imitating a smiling face to remind the user U of a sense of intimacy.

[0127] When the output of the voice of the question is finished, the mobile terminal 1 switches the display of the display 132 from the screen 170C to the screen 170D. The screen 170D is displayed while the input of the answer based on the voice or touch operation is accepted. This stage corresponds to Figure 6 Step S28. The screen 170D includes a vehicle object 181D, transition instructions 183, 184, and a speech guide 186. The speech guide 186 is an image indicating that the mobile terminal 1 is in a state where it can receive a voice input from the user U. The speech guide 186 may be an animation that changes according to the voice detection state of the microphone 134. The vehicle object 181D may be an image that imitates an expression different from that of the vehicle object 181C.

[0128] When receiving the voice of the user U and determining that the voice of the user U is not an appropriate answer, the mobile terminal 1 switches the display of the display 132 from the screen 170D to the screen 170E. Figure 6 If the judgment in step S29 is negative, screen 170E is displayed. Screen 170E is provided with a vehicle object 181E, conversion instruction units 183, 184, and an answer guide 187. Since the vehicle object 181E is used to notify the user U that the requested answer was not given, it is, for example, an image imitating a regretful expression. The answer guide 187 is an image that prompts the user U to give an answer suitable for determining the value of the driver information 125. The answer guide 187 includes, for example, a character string that assists the user U in giving an appropriate answer. The character string of the answer guide 187 may also be a character string representing the value of the driver information 125 itself. Figure 7 As shown, the screen 170E may also include a plurality of answer guides 187 corresponding to a plurality of values ​​that can be obtained from the driver information 125. In this case, the character string of each answer guide 187 is determined according to the driver information 125 corresponding to the question. The answer guide 187 may also be an operation unit that accepts a touch operation, in which case the mobile terminal 1 can accept a touch operation on the answer guide 187 as an input of an answer.

[0129] In the display of screen 170D or screen 170E, the user U inputs an answer. If the input answer is an appropriate answer, the mobile terminal 1 performs the above-mentioned step S31 ( Figure 6 ) to determine whether all driver information 125 has been obtained.

[0130] If there is unacquired driver information 125 (step S31: No), mobile terminal 1 returns to step S26 to select driver information 125 and outputs a question in step S27. At this time, mobile terminal 1 switches the display of display 132 from screen 170D or screen 170E to screen 170C.

[0131] When all the driver information 125 is acquired (step S31 : YES), the mobile terminal 1 generates the auxiliary information 126 (step S17 ), switches the display of the display 132 to the screen 170F, and outputs the auxiliary information 126 (step S18 ).

[0132] Screen 170F is displayed as one form of output of auxiliary information 126. The audio of auxiliary information 126 output by mobile terminal 1 is output on screen 170F.

[0133] The screen 170F includes a vehicle object 181F, conversion instruction units 183 and 184, and an auxiliary information display unit 189. The auxiliary information display unit 189 displays the content of the auxiliary information 126. The auxiliary information display unit 189 displays the display object 150 and the attention object 160. As described above, the number of images and icons displayed on the auxiliary information display unit 189 is preferably four or less, and more preferably two or less. Figure 7 As a preferred example, two images of the display object 150 and the caution object 160 are displayed on the auxiliary information display unit 189. Therefore, the image displayed on the auxiliary information display unit 189 is easy to remain in the memory of the user U driving the vehicle M during subsequent driving, and is easy to recall during driving.

[0134] In the screen 170F, the vehicle object 181F may be an image imitating an expression corresponding to the weather shown by the display object 150 .

[0135] The attention object 160 is an image that functions as advice to the user U regarding driving of the vehicle M. The attention object 160 is, for example, an icon, a still image, an animation, etc. that indicates an object, a place, a person, etc. that the user U should pay attention to when driving the vehicle M. In addition, the attention object 160 is a display object that indicates driving tips or tricks (TIPS) for the user U to drive the vehicle M. The attention object 160 is an image that the auxiliary information generation unit 104 obtains from the auxiliary DB 124 together with the display object 150 and is included in the auxiliary information 126. The attention object 160 is stored in the auxiliary DB 124 in correspondence with the driver status classified by the state determination unit 103, for example. Alternatively, the attention object 160 may be stored in the auxiliary DB 124 in correspondence with a combination of values ​​of the driver information 125. The state determination unit 103 may select a attention object 160 from a plurality of attention objects 160 stored in the auxiliary DB 124 at random or in accordance with the content of the display object 150 and include it in the auxiliary information 126. Furthermore, the auxiliary DB 124 may store messages corresponding to the plurality of attention objects 160 , respectively. In this case, the state determination unit 103 acquires the message corresponding to the attention object 160 from the auxiliary DB 124 and includes it in the auxiliary information 126 .

[0136] In the display of screen 170F, the message output by the mobile terminal 1 through voice includes a message corresponding to the display object 150. The voice of the message, for example, associates the display object 150 with imitating the weather forecast, and includes a sentence such as "publishing today's driving forecast". In addition, the voice of the message may also include a sentence describing the content of the display object 150. The mobile terminal 1, for example, says "cloudy and turning to rain". In addition, the voice of the message may also include a sentence describing the content of the display object 150 in detail, for example, the mobile terminal 1 may also say "due to the appearance of a slightly deserted area, it is sometimes not possible to exert your usual driving ability". In addition, the mobile terminal 1 may also output a message related to the attention object 160 as described above. For example, the mobile terminal 1 displays the sentence "When overtaking a bicycle, if you drive slowly at a distance, you will be satisfied. Also imagine the mood of the cyclist and drive safely!" in association with the attention object 160 which is an image of a bicycle.

[0137] In this way, the mobile terminal 1 obtains the driver information 125 from the biological information of the user U, the information heard using the questions, etc., and estimates the driver status representing the status of the user U by combining a plurality of driver information 125. Then, based on the driver status, the mobile terminal 1 determines the capabilities of the user U that are insufficient or cannot function well, and generates auxiliary information 126 that is a set of suggestions to supplement these capabilities. When generating the auxiliary information 126, the mobile terminal 1 selects appropriate suggestions based on the driver status of the user U. Thus, it is possible to provide driving assistance to the user U who is driving the vehicle M. By using the information output from the mobile terminal 1 as a driving forecast, which is a metaphor for the weather forecast, the user U can understand that the driving forecast is only a prediction, and the driving forecast can be expressed so that the user U can easily and intuitively understand the driving forecast.

[0138] [2. Second Embodiment]

[0139] Reference Figure 8 to Figure 10 , a second embodiment to which the present invention is applied is described.

[0140] In the second embodiment described below, the configurations of the mobile terminal 1 , WD 2 , and the communication system 3 are the same as those of the first embodiment, and thus the same configurations as those of the first embodiment are denoted by the same reference numerals, and illustration and description thereof are omitted.

[0141] [2-1. Classification of driver status]

[0142] In the first embodiment, the mobile terminal 1 classifies the driver status of the user U into three stages, namely, the active state, the passive state, and the intermediate state. In the second embodiment, the mobile terminal 1 classifies the driver status of the user U into seven stages.

[0143] Figure 8 It is a diagram showing a configuration example of the display setting data 124B in the second embodiment.

[0144] The display setting data 124B is data stored in the assist DB 124 , and includes data corresponding to the seven-stage classification of the driver's state.

[0145] The display setting data 124A described in the first embodiment includes a display object 150 that reflects the temporal change of the driver state corresponding to the combination of the first driver state and the second driver state. In contrast, the display setting data 124B includes data related to seven stages for classifying the driver state at the first time point and the driver state at the second time point, respectively.

[0146] Specifically, in the display setting data 124B, seven stages are associated with the classification of the driver's status. The highest stage 1 among the seven stages indicates that the driver is in a good state, and the symbol of the driver's status is "sunny". The symbol of the driver's status is a metaphor that imitates the weather forecast to express the driver's status, that is, the status of the user U.

[0147] The lowest stage 7 among the seven stages indicates a poor driver state, and the symbol of the driver state is "rain". Stages 1 to 7 are divided into stages 2 to 6. In the symbols of the driver state of each stage, stage 2 is "sunny with occasional clouds", stage 3 is "cloudy with occasional sunny", stage 4 is "cloudy", stage 5 is "cloudy with occasional rain", and stage 6 is "rainy with occasional clouds".

[0148] Attributes are set for the driver status at each stage. Attributes are a combination of a main attribute and a sub-attribute. The main attribute of stage 1 to stage 2 is "sunny", the main attribute of stage 3 to stage 5 is "cloudy", and the main attribute of stage 6 to stage 7 is "rain". In other words, the display setting data 124B includes seven stages corresponding to the combination of the three main attributes of sunny, cloudy, and rainy and the sub-attribute.

[0149] The display setting data 124B includes display objects 155 corresponding to stages 1 to 7, respectively. The display object 155A corresponding to stage 1 corresponds to "sunny" as the driver's state in stage 1, and is composed of a first object 151 meaning sunny. The display object 155B corresponding to stage 2 corresponds to "sunny with occasional clouds" as the driver's state in stage 2, and is composed of a first object 151 and a third object 153. For each stage from stage 3 to stage 7, display objects 155C to 155G are also corresponding. In this way, display objects 155A to 155G are images that use images used in weather forecasts as metaphors to express the driver's state, that is, the state of the user U.

[0150] The display object 155 includes a combination of an object representing a primary attribute and an object representing a secondary attribute. For example, the display object 155F representing stage 5 is a combination of a second object 152 representing a primary attribute and a third object 153 representing a secondary attribute.

[0151] The combination of the main attribute and the sub-attribute in the display setting data 124B does not represent the change of the driver's state over time. For example, "Sunny with some clouds" as the symbol of stage 2 expresses the driver's state by adding the subtle difference of "cloudy" as the sub-attribute to "Sunny" as the main attribute, and does not mean that the driver's state changes from "Sunny" to "Cloudy". The same is true for the display object 155. For example, the display object 155F representing stage 5 does not mean that the driver's state changes from "rain" shown by the second object 152 to "cloudy" shown by the third object 153.

[0152] Fig. 9 It is a diagram showing a configuration example of the display setting data 124C in the second embodiment.

[0153] The display setting data 124C is data stored in the assist DB 124 , and includes data corresponding to six time-series changes in the driver's state.

[0154] In detail, the display setting data 124C includes six display objects 155 corresponding to six types of changes over time of the main attributes of the driver's state. For example, in the display setting data 124C, a display object 155I is associated with a situation where the main attribute of the driver's state changes from "sunny" to "rainy". The display object 155I includes a first object 151 and a second object 152. The display object 155I is an image that uses the first object 151 and the second object 152 and uses the metaphor of the weather forecast to express the change of the driver's state from a good state to a bad state. The display objects 155H, 155J, 155K, 155L, and 155M are also images that express the time series changes of the driver's state using the metaphor of the weather forecast.

[0155] [2-2. Actions of mobile terminal]

[0156] The mobile terminal 1 uses the display setting data 124B through the function of the state determination unit 103. Figure 5 In step S13 and step S14, the driver status of the user U is classified into seven stages.

[0157] Specifically, in step S13, the mobile terminal 1 classifies the first driver status into seven stages based on the result of analyzing the driver information 125 in step S12. Furthermore, in step S14, the mobile terminal 1 classifies the second driver status into seven stages based on the result of analyzing the driver information 125 in step S12. Therefore, the first driver status and the second driver status are respectively classified into any one of the seven stages shown in the display setting data 124B.

[0158] The method for the state determination unit 103 to classify the first driver state and the second driver state based on the driver information 125 can use the method illustrated in the first embodiment. For example, the state determination unit 103 may also use a table that corresponds to a combination of the respective values ​​of the driver information 125A to 125D, the classification of the first driver state, and the classification of the second driver state. And, for example, the state determination unit 103 may also use a function or a program to determine the classification of the first driver state and the classification of the second driver state respectively according to the driver information 125. And, for example, the state determination unit 103 may also determine the classification of the first driver state and the classification of the second driver state respectively by calculating the value of the driver information 125. For example, the state determination unit 103 may also determine the classification of the first driver state and the classification of the second driver state by using a learning model that corresponds the value of the driver information 125 to the driver state.

[0159] Mobile terminal 1 Figure 5 In step S15, the display object 155 is acquired from the display setting data 124B or the display setting data 124C based on the first driver state and the second driver state. In step S17, the mobile terminal 1 generates the auxiliary information 126 including the display object 155 acquired in step S15.

[0160] Fig.10 is a flowchart showing an example of the operation of the mobile terminal 1 in the second embodiment. Figure 5 The action of obtaining the display object 155 in step S15. Fig.10 The action is performed by the auxiliary information generating unit 104.

[0161] The mobile terminal 1 obtains the stage of the first driver state and the stage of the second driver state (step S41). The stage obtained in step S41 is the numerical value of each stage set in the display setting data 124B. The mobile terminal 1 calculates the difference between the stage of the first driver state and the stage of the second driver state (step S42), and determines whether the calculated difference is greater than a threshold value (step S43). The threshold value is, for example, 3.

[0162] When the difference between the first driver state stage and the second driver state stage is less than the threshold value (step S43: No), the mobile terminal 1 acquires the display object 155 corresponding to the first driver state stage from the display setting data 124B (step S44). That is, the mobile terminal 1 uses any one of the display objects 155A to 155G as the display object included in the auxiliary information 126.

[0163] When the difference between the stage of the first driver state and the stage of the second driver state is greater than the threshold value (step S43: Yes), the mobile terminal 1 determines the time series change of the main attribute based on the main attribute of the first driver state and the main attribute of the second driver state (step S45). The mobile terminal 1 obtains the display object 155 corresponding to the change of the main attribute of the driver state determined in step S45 (step S46). That is, the mobile terminal 1 uses any one of the display objects 155H to 155M as the display object included in the auxiliary information 126.

[0164] After executing step S44 or step S46, the mobile terminal 1 transfers to Figure 5 Step S16.

[0165] As described above, in the second embodiment, the mobile terminal 1 classifies the driver status of the user U into seven stages. Thus, the mobile terminal 1 can classify the status of the user U in more detail using the driver information 125 obtained based on the answer to the question to the user U.

[0166] The mobile terminal 1 classifies at least one of the first driver state at the first time point and the second driver state at the second time point into seven stages. Therefore, the mobile terminal 1 can obtain more detailed changes in the state of the user U.

[0167] The mobile terminal 1 classifies the driver status at the first time point and the driver status at the second time point into seven stages respectively. The mobile terminal 1 determines the change of the driver status according to the driver status at the first time point and the driver status at the second time point. The auxiliary information 126 displayed on the screen 170 by the mobile terminal 1 includes the driver status at the first time point or a display object corresponding to the change of the driver status. In this way, the status of the user U can be classified in detail, and detailed suggestions can be given to the user U.

[0168] The mobile terminal 1 can determine whether the change in the state of the user U is large based on the difference between the first driver state and the second driver state using a threshold value.

[0169] Furthermore, when the change in the state of the user U is small, that is, when the change in the stage of the driver state is less than the threshold, the mobile terminal 1 displays the display object 150 including any one of the display objects 155A to 155G indicating the first driver state on the screen 170. Thus, the user U can be informed of the current state of the user U, and the driving of the vehicle M can be assisted by prompting attention to driving.

[0170] Furthermore, when the change in the state of the user U is large, that is, when the change in the stage of the driver's state is greater than the threshold, the mobile terminal 1 displays the display object 150 including any one of the display objects 155H to 155M indicating the time series change in the driver's state on the screen 170. Thus, the user U can be notified of the change in the state of the user U, and the driving of the vehicle M can be effectively assisted by prompting further attention to driving.

[0171] In any of the above cases, the mobile terminal 1 uses the image used for the weather forecast as a metaphor for the state of the user U. Thus, the state of the user U can be notified in a method that is easy to understand and easy to leave an impression. In addition, by using the metaphor of the weather forecast, information related to the state of the user U can be provided without making the user U feel nervous or uneasy, thereby assisting the driving of the vehicle M.

[0172] [3. Other embodiments]

[0173] The above-mentioned embodiments merely show one mode and can be arbitrarily modified and applied.

[0174] In the above-mentioned first embodiment, an example of displaying a display object 150 corresponding to a combination of a first driver state and a second driver state on a screen 170 is described. In addition, in a second embodiment, an example of displaying a display object 155 corresponding to the first driver state, or a display object 155 corresponding to a change in a main attribute of the driver state, on a screen 170 based on a difference between the first driver state and the second driver state is described. These are examples, and for example, the mobile terminal 1 may also use only one of the first driver state and the second driver state for display. Specifically, the mobile terminal 1 may also obtain a display object 150 or a display object 155 corresponding to either the first driver state or the second driver state, and display it on the screen 170.

[0175] In the above-mentioned embodiments, an example is described in which the auxiliary information generating unit 104 generates the auxiliary information 126 based on both the classification of the first driver state at the first time point and the classification of the second driver state at the second time point. This is an example. For example, the auxiliary information generating unit 104 may generate the auxiliary information 126 including the display object 155 corresponding to the first driver state based on the result of the classification of the first driver state at the first time point by the state determining unit 103. In this case, as the auxiliary information 126, the display object 155 representing the first driver state is displayed on the screen 170. In addition, the auxiliary information generating unit 104 may include advice corresponding to the first driver state in the auxiliary information 126. In these cases, the state of the user U at the first time point is notified to the user U through an image or the like, and the advice corresponding to the state is notified. In this example, the auxiliary information 126 is generated and output based on the first driver state, so there is an advantage that the information notified to the user U is simple, easier to understand, and easy to store. In addition, the processing of the auxiliary information generating unit 104 is simple, so there is also an advantage of a light processing load.

[0176] In each of the above-mentioned embodiments, an example is described in which, when the mobile terminal 1 is able to obtain the driver information 125 from WD2, a question corresponding to the obtained driver information 125 is not output. This is an example, for example, the mobile terminal 1 may also output a question for the driver information 125 whose value is determined based on the information obtained from WD2. In this case, the mobile terminal 1 may also assign additional information indicating a high reliability value to the value of the driver information 125 when the value based on the information obtained from WD2 is consistent with the value indicated by the answer of the user U to the question. In addition, the mobile terminal 1 may also assign additional information indicating a low reliability value to the value of the driver information 125 when the value based on the information obtained from WD2 is inconsistent with the value indicated by the answer of the user U to the question. Here, the mobile terminal 1 gives priority to either the information obtained from WD2 or the answer of the user U to determine the value of the driver information 125.

[0177] In this way, the mobile terminal 1 may also add additional information indicating the reliability to the value of the driver information 125. In this case, the mobile terminal 1 may change the message to be included in the auxiliary information 126 according to the additional information. For example, when the mobile terminal 1 obtains the driver information 125 with high reliability, the mobile terminal 1 includes a message including the determined performance in the auxiliary information 126. In addition, for example, when the reliability of the driver information 125 is low, the mobile terminal 1 may generate the auxiliary information 126 including a message with weakened performance.

[0178] Figure 4 , Figure 8 as well as Fig. 9This is a diagram schematically showing an example of the structure of data stored in the auxiliary DB 124. The structure of each data can be changed arbitrarily. For example, the composition, shape, and number of images of the display objects 150 and 155 included in the display setting data 124A, 124B, and 124C can be changed arbitrarily and may include text.

[0179] The vehicle M described in each of the above embodiments is an example of a mobile body. The vehicle M is not limited to a vehicle with four wheels, and may be a vehicle with five or more wheels, or may be a vehicle with three or fewer wheels. For example, the vehicle M may also be a two-wheeled motor vehicle. The mobile body may also be a large vehicle such as a bus, a commercial vehicle, or a work vehicle. The present disclosure can be applied to a mobile body that moves according to the manipulation or driving of the user U. As a mobile body, in addition to the land mobile bodies such as the above-mentioned automobiles, there may be marine mobile bodies such as ships and submarines, air mobile bodies such as eVTOL (Electric Vertical Take-Off and Landing aircraft) aircraft and airships, or space mobile bodies such as spacecraft and artificial satellites. In addition, the driving of the user U refers to the user U operating the mobile body to make it move, including manipulation and other operations.

[0180] In the above-mentioned embodiments, the portable terminal 1 used by the user U is described as executing Figure 5 to Figure 7 and Fig.10 The examples of the actions shown in the figure, however, these actions may also be performed by a device installed in the vehicle M. Specifically, the present disclosure may also be implemented by a display audio (DA), a car navigation device, an IVI (In-Vehicle Information Entertainment) system, or other vehicle-mounted devices installed in the vehicle M. In addition, the server 5 may also execute Figure 5 to Figure 7 as well as Fig.10 For example, the mobile terminal 1 may also perform communication with the WD2, output questions to the user U, accept input of the user U's answer, and output auxiliary information, and the server 5 may perform other Figure 5 In this case, the mobile terminal 1, the server 5 and the WD2 constitute the communication system 3.

[0181] The processor 100 may be composed of a single processor or a plurality of processors. The processor 100 may also be hardware programmed to implement corresponding functional units. That is, the processor 100 may include, for example, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0182] in addition, Figure 2 The structure of each part shown is an example, and the specific installation method is not particularly limited. That is, it is not necessary to install hardware corresponding to each part separately, and of course it can also be configured so that the functions of each part are realized by executing a program through a processor. In addition, in the above-mentioned embodiment, part of the function realized by software can also be set as hardware, or part of the function realized by hardware can also be realized by software. In addition, Figure 1 The specific detailed structures of the parts shown can also be changed arbitrarily.

[0183] in addition, Figure 5 , Figure 6 as well as Fig.10 The step units of the actions shown are units divided according to the main processing contents for easy understanding of the actions, and the present invention is not limited to the method and name of the division of the processing units. Depending on the processing contents, it can also be divided into more step units. In addition, it can also be divided into a step unit containing more processing. In addition, the order of the steps can also be appropriately replaced within the scope that does not hinder the main purpose of the present invention.

[0184] In addition, when the processor 100 is used to implement the driving assistance method of the mobile terminal 1, the program executed by the processor 100 can be stored in a non-temporary recording medium, or it can be configured in the form of a transmission medium that transmits the program. For example, at least any one of the control program 121 and the application 122 can also be implemented in a state recorded in a portable information recording medium. Information recording media can include magnetic recording media such as hard disks, optical recording media such as CDs, USB (Universal Serial Bus) memories, SSD (Solid State Drive) and other semiconductor storage devices, but other recording media can also be used.

[0185] [4. Structure supported by the above-mentioned embodiment]

[0186] The above-mentioned embodiment supports the following structure.

[0187] (Structure 1) A driving assistance device that performs the following processing: outputs a plurality of questions for obtaining a plurality of driver information related to the life of a driver of a mobile body; obtains the plurality of driver information based on the driver's answers to each of the plurality of questions; classifies the driver's state related to at least any one of the driver's psychological state and physical state based on the plurality of driver information; generates auxiliary information including suggestions related to driving based on the classification result of the driver's state; and outputs the auxiliary information.

[0188] According to the driving assistance device of structure 1, by outputting questions and allowing the driver to input answers, information related to the driver's state can be easily obtained, and suggestions corresponding to the driver's state can be made. As a result, the driver's state can be quickly classified, the driver can be assisted, and further, it can contribute to the development of a sustainable transportation system.

[0189] (Structure 2) A driving assistance device as described in Structure 1, wherein the driving assistance device includes a communication unit, which communicates with other devices and attempts to obtain the driver information through the communication unit, and when any one or more of the driver information is obtained through the communication unit, the question related to the driver information obtained through the communication unit is not output.

[0190] According to the driving assistance device of the second configuration, it is possible to obtain information on the driver's state by a method different from the driver inputting an answer to a question.

[0191] (Structure 3) The driving assistance device according to Structure 1 or Structure 2, wherein the driver state is classified into any one of a positive state, a negative state, and an intermediate state between the positive state and the negative state.

[0192] According to the driving assistance device of Configuration 3, by classifying the driver's state into three stages, it is possible to reduce the processing load required to generate assistance information in accordance with the driver's state, and to assist the driver more quickly.

[0193] (Configuration 4) The driving assistance device according to Configuration 1, wherein the driver state is classified into any one of four or more stages including a positive state, a negative state, and an intermediate state.

[0194] According to the driving assistance device of the fourth configuration, by classifying the driver's state into four or more stages, it is possible to output assistance information that more finely corresponds to the driver's state, thereby assisting the driver.

[0195] (Structure 5) The driving assistance device described in Structure 1, wherein, in the process of classifying the driver state, the driver state at a first time point is classified, and the assistance information including the classification result of the driver state at the first time point is generated.

[0196] According to the driving assistance device of configuration 5, the auxiliary information including the classification result of the driver's state at the first time point is output. Therefore, the content of the information output to the driver can be simplified, the driver's understanding and storage load can be reduced, and the processing load for generating the auxiliary information can be reduced.

[0197] (Structure 6) The driving assistance device described in Structure 5, wherein the auxiliary information including a display object corresponding to the classification result of the driver state at the first time point is generated as the auxiliary information, and the auxiliary information is output by displaying the display object on a display unit.

[0198] According to the driving assistance device of configuration 6, by displaying a display object representing the classification result of the driver's state at the first time point, information corresponding to the driver's state can be output to the driver through display. Therefore, the load on the driver to understand and store the output assistance information can be further reduced.

[0199] (Structure 7) A driving assistance device as described in any one of Structures 1 to 4, wherein, in the process of classifying the driver state, the driver state at a first time point is classified, the driver state at a second time point after a specified time has passed from the first time point is predicted and classified, and the auxiliary information including the classification result of the driver state at the first time point and the classification result of the driver state at the second time point is generated.

[0200] According to the driving assistance device of Configuration 7, by classifying the driver's state and the change in the driver's state over time, it is possible to provide advice corresponding to the change in the driver's state.

[0201] (Structure 8) A driving assistance device as described in Structure 7, wherein the auxiliary information is generated as the auxiliary information, including a display object corresponding to the classification result of the driver status at the first time point and the classification result of the driver status at the second time point, and the auxiliary information is output by displaying the display object on a display unit.

[0202] According to the driving assistance device of the eighth configuration, since the display objects corresponding to the driver's state and the temporal change in the driver's state are displayed, it is possible to provide advice to the driver in an easily understandable manner.

[0203] (Structure 9) A driving assistance device as described in Structure 8, wherein, in the process of classifying the driver state, the driver state at the first time point is classified into any one of a positive state, a negative state, and an intermediate state between the positive state and the negative state, and the driver state at the second time point is classified into any one of the positive state, the negative state, and the intermediate state, and the display object included in the auxiliary information uses at least any one of a first object representing the positive state, a second object representing the negative state, and a third object representing the intermediate state to reflect the passage of time and express the classification result of the driver state at the first time point and the classification result of the driver state at the second time point.

[0204] According to the driving assistance device of Configuration 9, by expressing the driver's state and its change as a display object reflecting the passage of time, it is possible to provide advice that matches the driver's state in a manner that the driver can easily visually recognize.

[0205] (Structure 10) A driving assistance device as described in Structure 9, wherein the display object has a structure that imitates a weather forecast, the first object is an image that imitates the performance of a sunny day, the second object is an image that imitates the performance of a rainy day, and the third object is an image that imitates the performance of a cloudy day.

[0206] According to the driving assistance device of configuration 10, advice reflecting the driver's condition is given using an image imitating a weather forecast. Therefore, advice that is easy to remember and understand is given, and the driver can be assisted more appropriately.

[0207] (Structure 11) The driving assistance device according to Structure 8, wherein, in the process of classifying the driver state, at least any one of the driver state at the first time point and the driver state at the second time point is classified into seven stages.

[0208] According to the driving assistance device of configuration 11, by finely classifying the driver's state, it is possible to provide assistance in accordance with the driver's state in a more fine manner.

[0209] (Structure 12) A driving assistance device as described in Structure 11, wherein, in the process of classifying the driver state, the driver state at the first time point and the driver state at the second time point are respectively classified into seven stages, and based on the driver state at the first time point and the driver state at the second time point, a change in the driver state is determined, and the assistance information includes the display object corresponding to the driver state at the first time point or the change in the driver state.

[0210] According to the driving assistance device of configuration 12, it is possible to finely classify the driver's state and assist driving in accordance with the driver's state or a change in the driver's state.

[0211] (Configuration 13) The driving assistance device according to any one of Configurations 1 to 12, wherein the assistance information includes information indicating driving tips or tricks that the driver should pay attention to when driving the mobile body.

[0212] According to the driving assistance device of configuration 13, it is possible to convey driving tips or tricks to the driver.

[0213] (Structure 14) The driving assistance device according to Structure 13, wherein the assistance information includes an auxiliary display object that expresses the secret or trick of driving.

[0214] According to the driving assistance device of configuration 14, it is possible to convey driving tips or tricks to the driver in a manner that is easy to retain in the driver's memory and easy to understand.

[0215] (Structure 15) A driving assistance method, which performs the following processing by a driving assistance device having a processor: outputting a plurality of questions for obtaining a plurality of driver information related to the life of a driver of a mobile body; obtaining the plurality of driver information based on the driver's answers to each of the plurality of questions; classifying the driver status related to at least any one of the driver's psychological state and physical state based on the plurality of driver information; generating auxiliary information including suggestions related to driving based on the classification result of the driver status; and outputting the auxiliary information.

[0216] According to the driving assistance method of structure 15, by outputting questions and allowing the driver to input answers, information related to the driver's state can be easily obtained, and suggestions corresponding to the driver's state can be made. As a result, the driver's state can be quickly classified, the driver can be assisted, and further, it can contribute to the development of a sustainable transportation system.

[0217] (Structure 16) A recording medium having a program executable by a computer, wherein the recording medium has a program for causing the computer to perform the following processing: outputting a plurality of questions for obtaining a plurality of driver information related to the life of a driver of a mobile body; obtaining a plurality of driver information based on the driver's answers to each of the plurality of questions; classifying a driver state related to at least one of a psychological state and a physical state of the driver based on the plurality of driver information; generating auxiliary information including suggestions related to driving based on the classification result of the driver state; and outputting the auxiliary information.

[0218] According to the recording medium having the program recorded thereon of structure 16, by outputting questions and allowing the driver to input answers, information related to the driver's state can be easily obtained, and suggestions corresponding to the driver's state can be made. As a result, the driver's state can be quickly classified, the driver can be assisted, and further, the development of a sustainable transportation system can be contributed to.

Claims

1. A driving assistance device, which performs the following processing: outputting a plurality of questions for acquiring a plurality of driver information related to the life of a driver of a mobile body; acquiring a plurality of driver information based on the driver's answer to each of the plurality of questions; classifying a driver state related to at least any one of a psychological state and a physical state of the driver based on the plurality of driver information; generating auxiliary information including driving-related suggestions based on the classification result of the driver status; The auxiliary information is output.

2. The driving assistance device according to claim 1, wherein: The driving assistance device includes a communication unit that communicates with another device. Attempting to obtain the driver information through the communication unit, When at least one of the plurality of pieces of driver information is acquired through the communication unit, the question related to the driver information acquired through the communication unit is not output.

3. The driving assistance device according to claim 1, wherein: The driver state is classified into any one of a positive state, a negative state, and an intermediate state between the positive state and the negative state.

4. The driving assistance device according to claim 1, wherein: The driver state is classified into any one of four or more stages including a positive state, a negative state, and an intermediate state.

5. The driving assistance device according to claim 1, wherein: In the process of classifying the driver status, the driver status at a first time point is classified. The auxiliary information including the classification result of the driver's state at the first time point is generated.

6. The driving assistance device according to claim 5, wherein: generating, as the auxiliary information, the auxiliary information including a display object corresponding to the classification result of the driver state at the first time point, The auxiliary information is outputted by displaying the display object on a display unit.

7. The driving assistance device according to claim 1, wherein: In the process of classifying the driver state, the driver state at a first time point is classified, and the driver state at a second time point after a predetermined time has passed from the first time point is predicted and classified, The auxiliary information including the classification result of the driver's state at the first time point and the classification result of the driver's state at the second time point is generated.

8. The driving assistance device according to claim 7, wherein: generating, as the auxiliary information, the auxiliary information including a display object corresponding to the classification result of the driver status at the first time point and the classification result of the driver status at the second time point, The auxiliary information is outputted by displaying the display object on a display unit.

9. The driving assistance device according to claim 8, wherein: In the process of classifying the driver state, the driver state at the first time point is classified as any one of a positive state, a negative state, and an intermediate state between the positive state and the negative state, and the driver state at the second time point is classified as any one of the positive state, the negative state, and the intermediate state, The display object included in the auxiliary information uses at least any one of a first object representing the positive state, a second object representing the negative state, and a third object representing the intermediate state to reflect the passage of time and express the classification result of the driver state at the first time point and the classification result of the driver state at the second time point.

10. The driving assistance device according to claim 9, wherein: The display object has a structure imitating a weather forecast, the first object is an image imitating the performance of a sunny day, the second object is an image imitating the performance of a rainy day, and the third object is an image imitating the performance of a cloudy day.

11. The driving assistance device according to claim 8, wherein: In the process of classifying the driver state, at least any one of the driver state at the first time point and the driver state at the second time point is classified into seven stages.

12. The driving assistance device according to claim 11, wherein: In the process of classifying the driver state, the driver state at the first time point and the driver state at the second time point are respectively classified into seven stages, and based on the driver state at the first time point and the driver state at the second time point, a change in the driver state is determined, The auxiliary information includes the display object corresponding to the driver state or a change in the driver state at the first time point.

13. The driving assistance device according to any one of claims 1 to 12, wherein: The assist information includes information indicating driving tips or tricks that the driver should pay attention to when driving the mobile body.

14. The driving assistance device according to claim 13, wherein: The auxiliary information includes an auxiliary display object representing the driving tips or tricks.

15. A driving assistance method, comprising: performing the following processing by a driving assistance device having a processor: outputting a plurality of questions for acquiring a plurality of driver information related to the life of a driver of a mobile body; acquiring a plurality of driver information based on the driver's answer to each of the plurality of questions; classifying a driver state related to at least any one of a psychological state and a physical state of the driver based on the plurality of driver information; generating auxiliary information including driving-related suggestions based on the classification result of the driver status; The auxiliary information is output.

16. A recording medium having a computer-executable program recorded thereon, wherein: The recording medium records a program for causing the computer to perform the following processing: outputting a plurality of questions for acquiring a plurality of driver information related to the life of a driver of a mobile body; acquiring a plurality of driver information based on the driver's answer to each of the plurality of questions; classifying a driver state related to at least one of a psychological state and a physical state of the driver based on the plurality of driver information; generating auxiliary information including driving-related suggestions based on the classification result of the driver status; The auxiliary information is output.

Images