Driver state estimation system

Abstract

A driver state estimation system transmits screen information of an in-vehicle terminal device to a call center when it is estimated that the driver is difficult on the basis of the screen operation status of the in-vehicle terminal device mounted in a vehicle by the driver.

Description

Technical Field

[0001] This invention relates to the technical field of a driver state inference system for inferring the state of a vehicle driver. Background Technology

[0002] For example, when a driver encounters difficulties operating the in-vehicle terminal device, they may sometimes contact a call center. Japanese Patent Application Publication No. 2008-101979 discloses a system in which operational data instructing the in-vehicle terminal device is generated at the call center. This operational data is then sent to the in-vehicle terminal device. Consequently, the system causes the in-vehicle terminal device to perform actions corresponding to the operational data. Summary of the Invention

[0003] The technology described in Japanese Patent Application Publication No. 2008-101979 has room for improvement. Furthermore, in call centers, learning models built using machine learning can be used to answer drivers' inquiries.

[0004] The present invention was made in view of the above circumstances, and its objective is to provide a driver state inference system that can reduce the burden on the driver.

[0005] In a driver state inference system according to one aspect of the present invention, if it is inferred that the driver is encountering difficulties based on the screen operation status of the vehicle-mounted terminal device performed by the driver, the screen information of the vehicle-mounted terminal device is sent to the call center. Attached Figure Description

[0006] Hereinafter, with reference to the accompanying drawings, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, in which the same reference numerals denote the same elements, and wherein:

[0007] Figure 1 This is a diagram illustrating an example of the structure of the driver state inference system according to the implementation method.

[0008] Figure 2 This is a flowchart illustrating the learning process of the driver state inference system according to the implementation method.

[0009] Figure 3 This is a diagram illustrating another example of the structure of the driver state inference system according to the implementation method.

[0010] Figure 4 This is a flowchart illustrating the actions of the driver state inference system according to the implementation method during inference. Detailed Implementation

[0011] refer to Figures 1 to 4The implementation methods involved in the driver state inference system are described. Figure 1 In this system, the driver state inference system 1 includes an in-vehicle terminal device 11 (hereinafter appropriately referred to as "in-vehicle unit 11") mounted on the vehicle 10 and an operator terminal 21 located in the call center 20. For example, the in-vehicle unit 11 may have a microphone, a speaker, and a communication device. For example, the in-vehicle unit 11 may be a navigation device.

[0012] When studying

[0013] First, the learning of the driver state inference model M in the driver state inference system 1 will be explained. Furthermore, prior to the learning of the driver state inference model M, the on-board unit 11 does not have a driver state inference machine 111.

[0014] If the driver of vehicle 10 encounters difficulties operating the onboard unit 11, the driver can use the onboard unit 11 to inquire with the call center 20. At this time, the driver can use the microphone, speaker, and communication device of the onboard unit 11 to communicate with the operator of the call center 20. Upon inquiring with the call center 20, the onboard unit 11 sends its screen operation history and onboard unit setting information to the call center 20. The screen operation history and onboard unit setting information are stored in database DB1. Furthermore, the operator of the call center 20 can use the operator terminal 21 to input information related to the driver's inquiry. This information is stored as a call center call history in database DB2. Additionally, databases DB1 and DB2 can be implemented by a server. In this case, databases DB1 and DB2 can be implemented by a single server or by multiple servers.

[0015] The screen operation history and vehicle-mounted unit setting information are fully accumulated in database DB1. The call center call history is fully accumulated in database DB2. Then, the call center 20 (e.g., a server set up in the call center 20) uses the screen operation history, vehicle-mounted unit setting information, and call center call history as learning data to learn the driver state inference model M.

[0016] Furthermore, the driver state inference model M can be a rule-based model or a model built through machine learning. Alternatively, the driver state inference model M can be a model built by fine-tuning the basic model using screen operation history, vehicle-mounted unit setting information, and call center call history as learning data.

[0017] refer to Figure 2 The flowchart illustrates the learning of the driver state inference model M. Figure 2 The actions shown can, for example, be performed by a server located in call center 20. Figure 2 In the process, the server uses screen operation history, vehicle-mounted unit setting information, and call center call history as learning data to learn the driver state inference model M (S101). Next, the server determines whether the constructed driver state inference model M can infer the driver's state (S102).

[0018] In the S102 process, the evaluation index involved in the constructed driver state inference model M can be used to determine whether the driver's state can be inferred. In the S102 process, if it is determined that the driver's state cannot be inferred (S102: "No"), the S101 process is performed.

[0019] In the processing of S102, if it is determined that the driver's state can be inferred (S102: "Yes"), the process ends. Figure 2 The actions shown are then described. The constructed driver state inference model M can then be distributed to multiple vehicles, including vehicle 10. As a result, for example, the onboard unit 11 of vehicle 10 has a driver state inference machine 111 using the constructed driver state inference model M.

[0020] Inference

[0021] Next, refer to Figure 3 and Figure 4 The operation of the vehicle-mounted unit 11, which has a driver state inference machine 111, will be described. When the driver of the vehicle 10 operates the screen of the vehicle-mounted unit 11 (i.e., touch panel), the driver state inference machine 111 infers the driver state (S201). At this time, the driver state inference machine 111 can output the confidence level involved in the inferred driver state.

[0022] For example, if the driver repeatedly presses the connect button on the Bluetooth (registered trademark) settings screen, the driver state inference unit 111 can infer that the driver is having difficulty connecting via Bluetooth. Similarly, if the driver repeatedly adjusts the volume, the driver state inference unit 111 can infer that the driver is having difficulty adjusting the volume. Likewise, if the driver repeatedly operates the air conditioner's on / off button, the driver state inference unit 111 can infer that the driver is having difficulty operating the air conditioner.

[0023] After the processing in S201, the on-board unit 11 determines whether the confidence level involved in the inferred driver state is above a predetermined threshold (S202). In the processing in S202, if it is determined that the confidence level is less than the predetermined threshold (S202: "No"), the processing in S201 is performed.

[0024] In the process of S202, if the confidence level is determined to be above the prescribed threshold (S202: "Yes"), the vehicle-mounted unit 11 uploads at least one of the images from the vehicle-mounted unit 11 and the warning light images to the operator terminal 21 of the call center 20 (S203). Here, the process of S203 is performed before the driver of the vehicle 10 inquires with the call center 20.

[0025] For example, if the driver state inference unit 111 infers that the driver is having difficulty connecting via Bluetooth, in step S203, the vehicle-mounted unit 11 can upload the Bluetooth settings screen to the operator terminal 21. Similarly, if the driver state inference unit 111 infers that the driver is having difficulty adjusting the volume, in step S203, the vehicle-mounted unit 11 can upload the volume settings screen to the operator terminal 21. Likewise, if the driver state inference unit 111 infers that the driver is having difficulty operating the air conditioning, in step S203, the vehicle-mounted unit 11 can upload the air conditioning options screen to the operator terminal 21.

[0026] Technical effect

[0027] Automobile companies typically operate call centers (e.g., call center 20) capable of inquiring with drivers of vehicles (e.g., vehicle 10) about at least one of their difficulties with the onboard unit (e.g., onboard unit 11) and vehicle operation. In these cases, call center operators often spend considerable time discussing the driver's difficulties. To reduce the time it takes for operators to understand the driver's difficulties, a method could be considered whereby the vehicle's onboard unit uploads the currently displayed screen to the call center. However, the screen currently displayed on the onboard unit is typically the screen used to make inquiries to the call center. Therefore, even if the screen currently displayed on the onboard unit is uploaded to the call center, it is difficult for the operator to understand the driver's difficulties from the uploaded screen.

[0028] In contrast, in this embodiment, the driver state inference machine 111 infers that the driver is experiencing difficulties. Furthermore, before the driver of vehicle 10 inquires with the call center 20, at least one of the images from the onboard unit 11 and the warning light images related to the driver's difficulties is uploaded to the call center 20 (operator terminal 21). As a result, when the driver of vehicle 10 inquires with the call center 20, the operator of the call center 20 can quickly grasp the driver's difficulties from at least one of the uploaded images from the onboard unit 11 and the warning light images. Moreover, even if the images from the onboard unit 11 at the time of inquiry differ from those at the time the driver is experiencing difficulties, the images from the onboard unit 11 at the time the driver is experiencing difficulties can still be uploaded to the call center 20. Therefore, according to the driver state inference system 1, the burden on the driver can be reduced.

[0029] The invention described below is an embodiment derived from the above-described implementation.

[0030] In a driver state inference system according to one aspect of the invention, if it is inferred that the driver is encountering difficulties based on the screen operation status of the in-vehicle terminal device mounted on the vehicle performed by the driver, the screen information of the in-vehicle terminal device is sent to the call center.

[0031] In one example of this driver status inference system, the system can send the screen information before the driver makes an inquiry to the call center.

[0032] This invention is not limited to the embodiments described above, and appropriate modifications can be made without departing from the spirit or concept of the invention as read from the claims and the entire specification. Driver state inference systems accompanying such modifications are also included within the scope of this invention.

Claims

1. A driver state inference system, characterized in that, Based on the driver's operation of the in-vehicle terminal device, if it is inferred that the driver is encountering difficulties, the screen information of the in-vehicle terminal device is sent to the call center.

2. The driver state inference system according to claim 1, characterized in that, The video information is sent before the driver makes an inquiry to the call center.

Images