program

Abstract

While being easy to implement, it reduces the processing load on the CPU and other components, and enables the suggestion of breaks in accordance with the driver's preferences. [Solution] The program causes the computer to perform the following actions: acquire utterances from the vehicle driver; determine whether a first phrase directly indicating the driver's fatigue is included in the utterances; if the first phrase is not included in the utterances, perform an estimation process to estimate whether the driver is fatigued based on the utterances; if the first phrase is included in the utterances, determine that the driver is fatigued without performing the estimation process; and if the driver is estimated or determined to be fatigued, output suggestion information to the driver suggesting a break.

Description

Technical Field

[0001] The present disclosure relates to a program that estimates the fatigue state of a driver and makes a rest proposal to the driver according to the estimation result.

Background Art

[0002] Conventionally, there has been known a technique of making a rest proposal to a driver according to the result of estimating the fatigue state of the driver. For example, Patent Document 1 discloses a technique of making a rest proposal when the driver is in a drowsy / fatigued state within a predetermined distance from SA / PA. Also, a technique of using the degree of eye opening for determining the "drowsy / fatigued state" at that time is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, as described in the above Patent Document 1, when the drowsy state is always measured by the same method, the processing load on the CPU or the like is large. Therefore, a technique for reducing such a processing load is desired. In addition, in the existing technologies as described above, it is necessary to implement dedicated processing for calculating the fatigue degree from signal information such as voice or camera, and there is a problem that the implementation difficulty is high.

[0005] More simply, it is also possible to judge only by the continuous driving time. In that case, however, conditions such as the amount of sleep of the driver are often not taken into account, and since it deviates from the driver's intention to rest, the driver may feel uncomfortable.

[0006] This disclosure provides a program that can appropriately resolve such problems, reduce the processing load on the CPU and other components, and make rest suggestions in accordance with the driver's intentions. [Means for solving the problem]

[0007] A program according to one embodiment of the present disclosure is a program that causes a computer to perform the following operations: acquire utterances by a vehicle driver; determine whether a first phrase directly indicating the driver's fatigue is included in the utterances; if the first phrase is not included in the utterances, perform an estimation process to estimate whether the driver is fatigued based on the utterances; if the first phrase is included in the utterances, determine that the driver is fatigued without performing the estimation process; and, if the driver is estimated or determined to be fatigued, output suggestion information to the driver suggesting a break. [Effects of the Invention]

[0008] According to one embodiment of this disclosure, it is possible to reduce the processing load on the CPU and other components while providing break suggestions that align with the driver's intentions, all while maintaining a simple implementation. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram illustrating this disclosure. [Figure 2] This is a flowchart illustrating an example of how the program works. [Modes for carrying out the invention]

[0010] The embodiments will be described below.

[0011] As shown in Figure 1, the system according to this embodiment comprises a voice assistant device 10 and a vehicle device 20 to be operated in a vehicle 1. The voice assistant device 10 is, for example, a voice assistant device in a navigation system installed in a vehicle such as an automobile, a smartphone, or a tablet terminal, but is not limited to these, and may be any device used by the user as long as it has a voice assistant function. The vehicle device 20 to be operated is generally a vehicle-mounted device, such as a navigation system (for POI search), air conditioner, doors, sound system, etc.

[0012] First, the program in one embodiment of the present disclosure acquires utterances made by the vehicle driver. Next, it determines whether or not a first phrase that directly indicates driver fatigue is included in the utterances. If the first phrase is present during such utterance, the system directly determines that the driver is fatigued without estimating the driver's fatigue level. In this disclosure, the first phrase can be, for example, phrases such as "I'm tired," "My shoulders are stiff," "I want to rest a little," or "I'm sleepy." Furthermore, such phrases can be modified by having the voice assistant device 10 learn them, and by adding or deleting them as appropriate.

[0013] On the other hand, if the first phrase is not included in such utterance, an estimation process is performed to estimate whether the driver is fatigued based on the utterance. This estimation process may include estimating whether the driver is fatigued based on the utterance and the vehicle's remaining range. Furthermore, this estimation may include the step of estimating that the driver is fatigued if the utterance contains a second phrase that indirectly indicates the driver's fatigue and the vehicle's remaining range is greater than or equal to a predetermined threshold.

[0014] In one embodiment of the present disclosure, if the program is estimated or determined to be fatigued, it outputs suggestion information to the driver suggesting a break and then terminates.

[0015] Next, the configuration of each element included in the system of the present disclosure will be described in detail.

[0016] As shown in FIG. 1, the voice assistant device 10 includes a visual device 11, an input unit 12, a voice playback unit 13, a communication unit 14, a storage unit 15, and a control unit 16.

[0017] The visual device 11 may be a display for displaying information. The display may include, for example, a panel display that displays information on a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. In the present embodiment, the visual device 11 mainly shows a map (image) among the proposals output by the control unit 16, but may also display the character string of the proposal output by the control unit 16.

[0018] The input unit 12 includes one or more input interfaces, that is, a microphone that receives voice input by a driver. The input unit 12 may also include a keyboard that receives character input and a touch panel provided integrally with the visual device 11.

[0019] The voice playback unit 13 includes one or more speakers and plays back the voice of the proposal output by the control unit 16.

[0020] The communication unit 14 includes one or more communication interfaces corresponding to an arbitrary communication standard. For example, the communication unit 14 may include communication interfaces corresponding to mobile communication standards such as 4G (4th Generation), wired LAN (Local Area Network) standards, wireless LAN standards, CAN (Controller Area Network), and USB (Universal Serial Bus) standards, respectively.

[0021] The storage unit 15 includes one or more memories. In the present embodiment, the memories may include, for example, semiconductor memories, magnetic memories, optical memories, and the like. Each memory included in the storage unit 15 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 15 stores any data used for the operation of the voice assistant device 10 and data obtained by the operation of the voice assistant device 10. Also, the first phrase and the second phrase are stored as speech text.

[0022] The control unit 16 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process, but is not limited thereto. The programmable circuit is, for example, an FPGA (Field-Programmable Gate Array), but is not limited thereto. The dedicated circuit is, for example, an ASIC (Application Specific Integrated Circuit), but is not limited thereto. The control unit 16 controls the operation of the entire voice assistant device 10.

[0023] In addition, the vehicle equipment 20 to be operated generally includes equipment installed in a vehicle, that is, a navigation (for POI search), an air conditioner, a door, an audio system, and the like.

[0024] Next, referring to FIG. 2, the operation of the program according to the present embodiment will be described.

[0025] FIG. 2 is a flowchart for explaining an operation example of the program in the present embodiment. Each of the following steps is executed by the control unit 16 of the voice assistant device 10 upon receiving an instruction from the voice assistant device 10 operated by, for example, a driver.

[0026] In S100, the voice assistant device 10 detects the voice assistant activation phrase and activates the control unit 16 and other parts of the voice assistant device 10.

[0027] In S101, the input unit 12 of the voice assistant device 10 acquires the driver's speech. In this disclosure, in order to distinguish which speech is the driver's speech, for example, the microphone (input unit 12) of the voice assistant device 10 can be attached to each seat, and it can be determined which seat's occupant made the speech.

[0028] In S102, the control unit 16 converts the driver's utterance acquired in S101 into speech text, compares it with the speech text of a pre-set first phrase, and determines whether or not the first phrase is included. If it is included (S102-Yes), the process proceeds to S103. On the other hand, if it is not included (S102-No), the process proceeds to S201. Examples of the first phrase include "I'm tired," "My shoulders are stiff," "I want to rest a little," "I'm sleepy," and "I want to rest right away," but it is not limited to these; any phrase that directly indicates that the driver is fatigued can be set as the first phrase.

[0029] In S103, the control unit 16 can decide which locations to search and in what order of priority. That is, it may search using different search conditions depending on the first phrase. An example of setting different search conditions depending on the first phrase is to classify "tired," "stiff shoulders," and "want to rest a little" as low-priority fatigue, and "sleepy" and "want to rest immediately" as high-priority fatigue, and then set search conditions according to these classifications.

[0030] To give an example of such search conditions, if you are driving on a highway, the nearest service area (SA) or parking area (PA) will be prioritized. If you are driving on a regular road and experiencing fatigue of low urgency, the search will prioritize a specific category along the route (for example, prioritizing coffee shops over convenience stores, and then roadside rest areas over coffee shops). On the other hand, if you are driving on a regular road and experiencing fatigue of high urgency, the search will be conducted for locations with parking, regardless of category. Note that these categories can be changed as appropriate. For example, the category "tired" can be changed to "high-urgency fatigue," or the categories can be further subdivided into four categories: low-urgency fatigue / high-urgency fatigue / low-urgency drowsiness / high-urgency drowsiness. If four categories are used in this way, for low-urgency drowsiness, for example, the specific category could be set to locations with a large number of parking spaces.

[0031] In S104, the control unit 16 searches for a POI (Point of Interest) for a rest area using the navigation system or the like of the vehicle equipment 20 to be operated, according to the conditions determined in S103.

[0032] In S105, the control unit 16 uses the search results from S104 and appropriately uses the visual device 11 and the voice playback unit 13 to suggest a break (response and / or display). For example, it might say, "Thank you for your hard work. I've found a place for a break. There's a convenience store nearby. I'll set it as a waypoint," using both voice and a map of the convenience store. After making the suggestion, the voice assistant terminates.

[0033] In S201, it is preferable that the control unit 16 performs a first step of estimation processing to estimate whether or not the driver is fatigued based on the utterance. For example, if a second phrase indirectly indicating driver fatigue is included in the utterance (S201-Yes), the process proceeds to S202. On the other hand, if a second phrase indirectly indicating driver fatigue is not included in the utterance (S201-No), the process proceeds to S301. In such cases, this estimation can be performed by checking whether or not a second phrase indirectly indicating driver fatigue is included in the utterance. Examples of such second phrases include idle utterances ("I'm bored," "Tell me a funny story," "Let's play rock-paper-scissors"), utterances related to changing comfort ("Lower the air conditioner temperature," "Ventilate the seat"), utterances related to ventilation ("Open the window a little," "Open the moonroof"), and utterances related to the sound system (Turn on the audio, change the source ("Play music," "Change the music")). In addition, although not intentionally indicating fatigue, utterances that might be made when tired can be appropriately selected, pre-transcribed into text, and used as the second phrase mentioned above. Furthermore, conditions such as no utterances related to audio operation for a certain period of time can also be used as check conditions in the first step of the estimation process instead of the second phrase.

[0034] In S202, the control unit 16 preferably checks whether the remaining flight time is above a predetermined threshold as the second step of the estimation process to estimate whether the driver is fatigued based on the utterance. If the remaining flight time is above the threshold (S202-Yes), it is estimated that the driver is fatigued, and the process proceeds to S203. On the other hand, if the remaining flight time is below the threshold (S202-No), it is not estimated that the driver is fatigued, and the process proceeds to S301.

[0035] In S203, the control unit 16 may search under the same conditions as for low-priority fatigue in S103. That is, if driving on an expressway, the nearest service area (SA) or parking area (PA) may be prioritized. Alternatively, if driving on an ordinary road, a specific category along the route may be prioritized in the search. If the categories are further subdivided into four categories, the search may be performed by appropriately referring to the same conditions as for low-priority fatigue and low-priority drowsiness in S103.

[0036] In S204, the control unit 16 searches for a POI (Point of Interest) for a rest area using the navigation system or other means of the vehicle equipment 20 to be operated, according to the conditions determined in S203.

[0037] In S205, the control unit 16 acquires normal response information using the dialogue model.

[0038] "Normal response information" refers to information output by the dialogue model as a response to the utterance acquired in S101. For example, if the utterance acquired in S101 includes the phrase "lower the air conditioner temperature," the normal response information "I will lower the air conditioner temperature by 1 degree" may be output in response to that utterance. In this disclosure, in S206, the control unit 16 may merge the normal response information acquired in S205 with the result of the rest location searched in S204.

[0039] In S207, the control unit 16 merges the results of the rest location search in S204 with the normal response information obtained in S205, and makes a suggestion using the visual device 11 and the voice playback unit 13 as appropriate. An example of such merging is, "I'll lower the air conditioner temperature by 1 degree (air conditioner operation). By the way, would you like to take a short break? There's a convenience store nearby." This suggests merging the search results for rest locations with the normal response information and making a suggestion using voice and a map of the convenience store. After making the suggestion, the voice assistant is terminated.

[0040] In S301, the control unit 16 acquires the normal response information.

[0041] In S302, the control unit 16 appropriately uses the visual device 11 and the audio playback unit 13 to respond and / or display the normal response information. After making the suggestion, the audio assistant is terminated.

[0042] As described above, embodiments of this disclosure have been explained based on the drawings and examples, but the configuration of the voice assistant device 10 and the vehicle equipment 20 to be operated used in this disclosure, as shown in Figure 1, can be used with known configurations to date without any particular problems. In other words, a person skilled in the art can easily make various modifications and alterations, including such configurations, based on the entirety of this disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of this disclosure. For example, the functions, etc., included in each means, each step, etc., can be rearranged in a way that is not logically contradictory, and multiple means, steps, etc., can be combined into one or divided as appropriate. As an example of modifications in this disclosure, steps S205 and S206 can be skipped as needed, or the number of specific genres to search in S103 and S203 can be increased if the cruising time is too long. Also, steps S201 and S202 can be executed using the AI ​​or image processing technology that has already been implemented, if available. [Explanation of Symbols]

[0043] 1 Vehicle; 10 Voice assistant device; 11 Visual device; 12 Input unit; 13 Voice playback unit; 14 Communication unit; 15 Memory unit; 16 Control unit; 20 Vehicle equipment to be operated

Claims

1. To acquire speech from the vehicle driver, To determine whether the first phrase directly indicating the driver's fatigue is included in the utterance, If the first phrase is not included in the utterance, an estimation process is performed to estimate whether the driver is fatigued based on the utterance. If the first phrase is included in the utterance, determine that the driver is fatigued without performing the estimation process, and If the driver is estimated or determined to be fatigued, the system will output suggestion information to the driver to suggest taking a break. A program that causes a computer to perform an action that includes [a specific action].

2. The program according to claim 1, The computer, when performing the estimation process, has a program that estimates whether the driver is fatigued or not based on the utterance and the vehicle's remaining operating time.

3. The program according to claim 2, A program that, when performing the estimation process, estimates that the driver is fatigued if the utterance contains a second phrase that indirectly indicates the driver's fatigue and the vehicle's remaining range is greater than or equal to a predetermined threshold.

4. The program according to claim 1, The aforementioned operation is, If the first phrase is not included in the utterance, a dialogue model is used to obtain response information indicating a response to the utterance, and Output the aforementioned response information. A program that further includes the following.

5. The program according to claim 1, The operation further includes, if it is determined that the driver is fatigued, searching for a rest area where the vehicle can park using different search conditions according to the first phrase. The suggested information, which is output when the driver is determined to be fatigued, is a program that includes information indicating one or more rest locations that have been searched.

Images