System and System Operation Method

Abstract

To improve the accuracy of drivers' perception and understanding of diagnostic results regarding vehicle operation. [Solution] The system comprises an in-vehicle device mounted in a vehicle, a terminal device used by the driver of the vehicle, and a server device that communicates with the in-vehicle device and the terminal device, wherein the in-vehicle device outputs a notification of diagnostic results corresponding to the driver's driving to the driver, sends first information including the content of the notification to the server device, and the server device sends second information to the terminal device for the terminal device to output information supplementing the notification.

Description

Technical Field

[0001] The present disclosure relates to a system and a method for operating the system.

Background Art

[0002] Techniques for diagnosing the state of a moving body based on the behavior of the moving body including a vehicle are known. For example, Patent Documents 1 to 3 disclose systems that detect information obtained by detecting the behavior of a vehicle or the like and perform diagnosis of the vehicle or the like by processing the information on a server on the cloud.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] When notifying a driver of various diagnostic results regarding the driving of a vehicle by AI (Artificial Intelligence) or the like, there is room for improving the driver's accuracy of recognizing and understanding the diagnostic results.

[0005] Hereinafter, a system or the like that enables improvement in the driver's accuracy of recognizing and understanding diagnostic results regarding the driving of a vehicle will be disclosed.

Means for Solving the Problems

[0006] The system in this disclosure is a system comprising an in-vehicle device mounted in a vehicle, a terminal device used by the driver of the vehicle, and a server device that communicates with the in-vehicle device and the terminal device, wherein the in-vehicle device outputs a notification of diagnostic results corresponding to the driver's driving to the driver, sends first information including the content of the notification to the server device, and the server device sends second information to the terminal device for the terminal device to output information supplementing the notification.

[0007] A method for operating a system in another aspect of the present disclosure is a method for operating a system having an in-vehicle device mounted in a vehicle, a terminal device used by the driver of the vehicle, and a server device that communicates with the in-vehicle device and the terminal device, the method comprising: the in-vehicle device outputting a notification of diagnostic results corresponding to the driver's driving to the driver and sending first information including the content of the notification to the server device; and the server device sending second information to the terminal device for the terminal device to output information supplementing the notification. [Effects of the Invention]

[0008] The system described in this disclosure will enable improved accuracy in the driver's recognition and understanding of diagnostic results regarding vehicle operation. [Brief explanation of the drawing]

[0009] [Figure 1] This is a diagram showing an example of the configuration of an information processing system. [Figure 2] This is a sequence diagram illustrating an example of the operation of an information processing system. [Figure 3] This is a flowchart illustrating an example of server device operation. [Modes for carrying out the invention]

[0010] The embodiments will be described below.

[0011] Figure 1 shows an example of the configuration of an information processing system in one embodiment. The information processing system 1 has one or more server devices 10, an in-vehicle device 13, and a user terminal 14, each connected to each other via a network 11 in a manner that enables information communication. The server device 10 is, for example, a server computer belonging to a cloud computing system or other computing system, and functions as a server that implements various functions. The in-vehicle device 13 is, for example, a navigation system, which has communication functions and information processing functions, and is mounted on a vehicle 12. The vehicle 12 is a passenger car, commercial vehicle, etc., in which part or all of the driving is done manually by the driver. The vehicle 12 is, for example, a gasoline car, an electric vehicle (BEV, Battery Electric Vehicle), a hybrid vehicle (HEV, Hybrid Electric Vehicle), a plug-in hybrid vehicle (PHEV, Plug-in Hybrid Electric Vehicle), a fuel cell vehicle (FCEV, Fuel Cell Electric Vehicle), etc. The user terminal 14 is an information processing terminal used by the driver of the vehicle 12, and is, for example, a smartphone, tablet, PC (Personal Computer), etc. The user terminal 14 corresponds to the "terminal device" in this embodiment. The network 11 is, for example, the internet, but may be an ad-hoc network, LAN, MAN (Metropolitan Area Network), or other network, or any combination thereof.

[0012] In this embodiment, the information processing system 1 supports the diagnosis of the vehicle 12 and the notification of the diagnosis results to the driver. The information processing system 1 includes an on-board device 13 mounted on the vehicle 12, a user terminal 14 used by the driver of the vehicle 12, and a server device 10 that communicates with the on-board device 13 and the user terminal 14. The on-board device 13 outputs a notification of the diagnosis results (hereinafter referred to as the diagnosis notification) corresponding to the driver's driving to the driver, and sends first information (hereinafter referred to as history information) including the content of the diagnosis notification to the server device 10. The server device 10 sends second information to the user terminal 14 for outputting supplementary information (hereinafter referred to as supplementary information) that supplements the diagnosis notification on the user terminal 14. Since the information that can be transmitted to the driver during driving is somewhat limited, the server device 10 causes the supplementary information to be output to the user terminal 14, so that the driver can check the supplementary information on the user terminal 14, for example, at the end of driving, and deepen their understanding of the diagnosis results by reviewing them. Furthermore, even if there is a high probability that the diagnostic notification is not adequately conveyed to the driver due to an event that interferes with the diagnostic notification during driving, the driver can still be sure to recognize the diagnostic results through supplementary information. Therefore, it becomes possible to improve the accuracy of the driver's recognition and understanding of the diagnostic results regarding the operation of vehicle 12.

[0013] Next, we will describe an example configuration of the server device 10.

[0014] The server device 10 includes a communication unit 101, a storage unit 102, and a control unit 103. The server device 10 may be a single computer, or it may consist of two or more computers that are connected and operate in coordination with each other. When the server device 10 consists of two or more computers, the configuration shown in Figure 1 is appropriately arranged across the two or more computers.

[0015] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the control unit 103 and transmits information obtained through the operation of the control unit 103. The server device 10 is connected to the network 11 by the communication unit 101 and communicates information with the in-vehicle device 13 and the user terminal 14 via the network 11.

[0016] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these, which function as main memory, auxiliary memory, or cache memory. The semiconductor memory is, for example, RAM (Random Access Memory) or ROM (Read Only Memory). The RAM is, for example, SRAM (Static RAM) or DRAM (Dynamic RAM). The ROM is, for example, EEPROM (Electrically Erasable Programmable ROM). The storage unit 102 stores information used in the operation of the control unit 103 and information obtained by the operation of the control unit 103.

[0017] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processors are, for example, general-purpose processors such as CPUs (Central Processing Units) or dedicated processors such as GPUs (Graphics Processing Units) specialized for specific processing. The dedicated circuits are, for example, FPGAs (Field-Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits). The control unit 103 controls each part of the server device 10 and performs information processing related to the operation of the server device 10.

[0018] The functions of the server device 10 are realized by executing a control program with a processor included in the control unit 103. The control program is a program for causing a computer to execute the processing of the steps included in the operation of the server device 10, so as to cause the computer to realize the functions corresponding to the processing of those steps. That is, the control program is a program for causing a computer to function as the server device 10. Also, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Further, the control program may be stored in a non-transitory recording and storage medium readable by the server device 10, and the server device 10 may read it from the medium.

[0019] In this embodiment, the storage unit 102 stores a server diagnosis model 108 and a server agent 109. The server diagnosis model 108 is an AI model that generates supplementary information suitable for the diagnosis result. The server diagnosis model 108 may be a model that machine-learns the correspondence between the diagnosis result and the supplementary information, or may be a rule-based model that derives the supplementary information from the diagnosis result. The diagnosis result is the result of determining whether the feature amount of the operation by the driver corresponds to dangerous driving or not. The supplementary information is the past history, assumed cause, avoidance proposal, etc. for the driving operation diagnosed as dangerous driving. The information for constituting the supplementary information may be obtained from, for example, another server that provides traffic safety information. The server agent 109 is an interactive AI module for generating a notification (hereinafter referred to as a supplementary notification) for transmitting the supplementary information by the server diagnosis model 108 to the driver, and has a natural language processing function, a knowledge base about the diagnosis result and the driver's preferences, etc.

[0020] Next, a configuration example of the in-vehicle device 13 will be described.

[0021] The in-vehicle device 13 includes a communication unit 131, a memory unit 132, a control unit 133, a positioning unit 134, an input unit 135, an output unit 136, and a detection unit 137. These may be configured as one control device, or may be composed of two or more control devices, or may be composed of a control device and other devices such as a communication device. The control device includes, for example, an ECU (Electronic Control Unit), etc. The communication device includes, for example, a DCM (Data Communication Module), etc. Each unit is connected to each other or to the equipment of the vehicle 12 so as to be able to communicate information via an in-vehicle network compliant with a standard such as CAN (Controller Area Network). Also, the in-vehicle device 13 may be configured to partly include a device equivalent to the user terminal 14.

[0022] The communication unit 131 has a communication module corresponding to a wired or wireless LAN standard, a module corresponding to a mobile communication standard such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation), etc. The in-vehicle device 13 is connected to the network 11 via the communication unit 131 through a nearby router device or a base station for mobile communication, and performs information communication with other devices via the network 11.

[0023] The memory unit 132 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, a SRAM or a DRAM. The ROM is, for example, an EEPROM. The memory unit 132 functions as, for example, a main memory device, an auxiliary memory device, or a cache memory. The memory unit 132 stores information used for the operation of the control unit 133 and information obtained by the operation of the control unit 133.

[0024] The control unit 133 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processors are general-purpose processors such as CPUs, or dedicated processors such as GPUs specialized for specific processing. The dedicated circuits are, for example, FPGAs or ASICs. The control unit 133 controls each part of the in-vehicle device 13 and performs information processing related to the operation of the in-vehicle device 13.

[0025] The functions of the control unit 133, such as navigation and multimedia playback, are realized by executing a control / processing program on the processor included in the control unit 133. The control / processing program is a program that causes the computer to execute the processing steps included in the operation of the control unit 133, thereby realizing the functions corresponding to the processing of those steps. In other words, the control / processing program is a program that causes the computer to function as the control unit 133. Furthermore, some or all of the functions of the control unit 133 may be realized by dedicated circuits included in the control unit 133.

[0026] The positioning unit 134 includes one or more GNSS (Global Navigation Satellite System) receivers. GNSS includes, for example, GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System), BeiDou, GLONASS (Global Navigation Satellite System), and Galileo. The positioning unit 134 sends the positioning result to the control unit 133, which then obtains the location information of the in-vehicle device 13, i.e., the vehicle 12.

[0027] The input unit 135 includes one or more input interfaces. The input interfaces are, for example, a microphone that accepts voice input, a physical key, a capacitive key, a pointing device, or a touchscreen integrated with a display. The input interfaces include an interface with a camera provided in the vehicle 12 that captures images of the inside or outside of the vehicle 12. The camera may be built into the in-vehicle device 13 or it may be a separate unit. The input unit 135 receives information used for the operation of the control unit 133, such as operations, voice input, or images of the driver, etc., from the driver, etc., or from the camera, and sends the received information to the control unit 133.

[0028] The output unit 136 includes one or more output interfaces. These output interfaces include, for example, a speaker for outputting sound, a display for outputting images, etc. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 136 outputs information obtained through the operation of the control unit 133.

[0029] The detection unit 137 has sensors that detect various events occurring in the vehicle 12, or an interface with sensors. The sensors include, for example, sensors that detect the vehicle 12's speed, longitudinal acceleration, lateral acceleration, deceleration, accelerator operation amount, brake operation amount, steering angle, turn signal illumination time, fuel consumption per unit time, eco mode selection status, odometer value, safety equipment operation information, remaining engine oil, brake pad wear, battery degradation, etc. The sensors also include millimeter wave, infrared, and other radars that detect targets around the vehicle 12. The detection unit 137 sends vehicle information indicating various states of the vehicle 12 detected by the sensors to the control unit 133.

[0030] The control unit 133 controls the communication unit 131, storage unit 132, positioning unit 134, input unit 135, output unit 136, and detection unit 137 while exchanging various information with these units, and also controls the operation of the vehicle 12. When the vehicle 12 is running, the control unit 133 provides navigation functions by presenting various information necessary for driving, such as route information, to the driver via the output unit 136, and also provides multimedia content such as videos and music. In addition, the control unit 133 controls the partial automatic driving of the vehicle 12.

[0031] In this embodiment, the memory unit 132 stores the in-vehicle diagnostic model 138 and the in-vehicle agent 139. The in-vehicle diagnostic model 138 is an AI model that has learned diagnostic results corresponding to diagnostic target information such as driving operation characteristics or the driver's state. Based on the diagnostic target information acquired by the detection unit 137, the in-vehicle diagnostic model 138 diagnoses whether the driving operation or state constitutes dangerous driving. The driving operation characteristics include information indicating the control quantities in the driving operation or the motion state of the vehicle 12. The control quantities in the driving operation are control quantities such as brakes, accelerators, steering, and turn signals. The control quantities include the amount of change in the control quantities per unit time. The information indicating the motion state of the vehicle 12 includes the vehicle's speed, acceleration in the direction of travel and lateral direction, and the distance to other vehicles detected by radar or images captured from outside the vehicle 12. The information indicating the driver's state while driving includes the driver's captured images, voice, etc. The in-vehicle agent 139 is an interactive AI module for generating diagnostic notifications that convey the diagnostic results from the in-vehicle diagnostic model 138 to the driver, and includes natural language processing capabilities, a knowledge base on diagnostic results and driver preferences, and the like.

[0032] Next, we will describe an example configuration of user terminal 14.

[0033] The user terminal 14 includes a communication unit 141, a storage unit 142, a control unit 143, a positioning unit 144, an input unit 145, and an output unit 146.

[0034] The communication unit 141 includes a communication module compatible with wired or wireless LAN standards, a module compatible with mobile communication standards such as LTE, 4G, or 5G, etc. The user terminal 14 is connected to the network 11 via the communication unit 141 through a nearby router device or mobile communication base station, and communicates information with other devices via the network 11.

[0035] The storage unit 142 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memory is, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The storage unit 142 functions, for example, as a main memory, auxiliary memory, or cache memory. The storage unit 142 stores information used in the operation of the control unit 143 and information obtained by the operation of the control unit 143.

[0036] The control unit 143 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU, or a dedicated processor specialized for a specific process such as a GPU. The dedicated circuit is, for example, an FPGA or ASIC. The control unit 143 controls each part of the user terminal 14 and performs information processing related to the operation of the user terminal 14.

[0037] The positioning unit 144 includes one or more GNSS receivers. GNSS includes, for example, GPS, QZSS, BeiDou, GLONASS, and Galileo. The positioning unit 144 sends the positioning result to the control unit 143, which then obtains the location information of the user terminal 14.

[0038] The input unit 145 includes one or more input interfaces. These input interfaces include, for example, a microphone for receiving voice input, physical keys, capacitive keys, a pointing device, a touchscreen integrated with a display, and a camera for capturing images. The input unit 145 receives an operation to input information used in the operation of the control unit 143 and sends the input information to the control unit 143.

[0039] The output unit 146 includes one or more output interfaces. The output interfaces are, for example, speakers, displays, etc. The displays are, for example, LCDs or organic EL displays. The output unit 146 outputs information obtained by the operation of the control unit 143.

[0040] The functions of the control unit 143 are realized by executing a control / processing program on the processor included in the control unit 143. The control / processing program is a program that causes the computer to execute the processing steps included in the operation of the control unit 143, thereby realizing the functions corresponding to the processing of those steps. In other words, the control / processing program is a program that causes the computer to function as the control unit 143. Furthermore, some or all of the functions of the control unit 143 may be realized by dedicated circuits included in the control unit 143.

[0041] Next, the operation of the information processing system 1 will be explained using Figures 2 and 3.

[0042] Figure 2 is a sequence diagram illustrating the operation procedure of the information processing system 1 in this embodiment. Figure 2 shows the procedure for the coordinated operation of the server device 10, the in-vehicle device 13, and the user terminal 14. The steps related to various information processing of the server device 10, the in-vehicle device 13, and the user terminal 14 in Figure 2 are executed by the respective control units 103, 133, and 143. Furthermore, the steps related to the sending and receiving of various information of the server device 10, the in-vehicle device 13, and the user terminal 14 are executed by the respective control units 103, 133, and 143 sending and receiving information to each other via the communication units 101, 131, and 141, respectively. In the server device 10, the in-vehicle device 13, and the user terminal 14, the control units 103, 133, and 143 respectively appropriately store the information to be sent, received, and processed in the storage units 102, 132, and 142. Furthermore, in the in-vehicle device 13 and the user terminal 14, the control units 133 and 143 receive various types of information via input units 135 and 145, respectively, and output various types of information via output units 136 and 146, respectively.

[0043] Steps S201 to S208 of the in-vehicle device 13 in Figure 2 are executed at arbitrary intervals of several tens of milliseconds to several seconds, for example, when the vehicle 12 is in motion.

[0044] In S201, the in-vehicle device 13 acquires diagnostic information. The diagnostic information includes characteristic quantities of driving operations and information indicating the driver's state. For example, the control unit 133 acquires diagnostic information including control quantities such as brakes, accelerators, steering wheel, and turn signals, the vehicle's speed, acceleration in the direction of travel and lateral direction, distance to other vehicles, captured images of the driver, and voice, using various sensors and input interfaces provided on the vehicle 12. Each piece of information may be accompanied by a timestamp at the time of acquisition.

[0045] In S202, the in-vehicle device 13 performs a diagnosis on the information to be diagnosed. The in-vehicle device 13 executes the in-vehicle diagnostic model 138 via the control unit 133 to perform a diagnosis on the information to be diagnosed. The in-vehicle diagnostic model 138 diagnoses whether or not dangerous driving is occurring based on, for example, characteristics of driving operations such as control quantities for brakes, accelerators, steering, and turn signals, the vehicle's speed, acceleration in the direction of travel and lateral direction, distance from other vehicles, or driver images and voice recordings.

[0046] In S203, the in-vehicle device 13 generates a diagnostic notification based on the diagnostic results. The control unit 133 generates a diagnostic notification to the driver according to the diagnostic results using the in-vehicle agent 139. For example, if the in-vehicle agent 139 diagnoses sudden braking, sudden acceleration, sudden steering, etc., based on the amount of brake control, acceleration, etc., it generates a diagnostic notification with phrases such as "Sudden braking is dangerous!", "Sudden acceleration is dangerous!", and "Caution: sudden steering!" to warn of danger. If it diagnoses that the driver's condition includes drowsiness, fatigue, decreased attention, etc., it generates a diagnostic notification with phrases such as "Wake up!" and "Look ahead!" to encourage wakefulness and attention. Alternatively, the diagnostic notification may include, in addition to or instead of phrases, flashing warning lights or outputting warning sounds.

[0047] In S204, the in-vehicle device 13 generates interference information. The interference information indicates events that may occur in the vehicle 12 and interfere with the diagnostic notification to the driver. An interference event is, for example, the display or audio output of navigation information, road traffic information, etc., which takes precedence over the display or audio output of the diagnostic notification. Alternatively, the interference event may be the output of music audio within the vehicle 12 that interferes with the audio output of the diagnostic notification. The control unit 133 generates information indicating the presence of an interference event if the driver is using the navigation system, outputting traffic information, playing multimedia, etc., and otherwise indicating the absence of an interference event. If an interference event has occurred, the control unit 133 may include information indicating the type of the interference event in the interference information.

[0048] In S205, the in-vehicle device 13 outputs a diagnostic notification to the driver. However, if information that takes precedence over the diagnostic notification is displayed by the navigation system or traffic information output, the control unit 133 will stop displaying the diagnostic notification. Also, if information that takes precedence over the diagnostic notification is being output as audio, the control unit 133 will stop outputting the diagnostic notification as audio.

[0049] In S206, the in-vehicle device 13 generates response information indicating the driver's reaction to the diagnostic notification. The response information indicates whether or not the driver recognized the diagnostic notification. The control unit 133 acquires captured images and audio information from the input interface and determines the driver's reaction based on the captured images and audio information. For example, the control unit 133 performs arbitrary image processing on the captured images to detect the driver's facial movements. Then, for example, if the driver nods, the control unit 133 determines that the driver recognized the diagnostic notification, and otherwise determines that they did not. The control unit 133 analyzes the driver's speech based on the audio information and determines whether it contains words indicating recognition, such as "yes" or "I understand." Then, the control unit 133 determines that the driver recognized the diagnostic notification if it contains words indicating recognition, and otherwise determines that they did not. Depending on the determination result, the control unit 133 generates response information indicating whether or not the driver recognized the result notification.

[0050] In S207, the in-vehicle device 13 generates history information. The history information includes the diagnostic result, the diagnostic notification, whether or not the diagnostic notification was output, and, if the diagnostic notification was output, interference information and response information at the time of output. The control unit 133 can process the output of the diagnostic notification as if it had occurred when the time difference between the timing of the output of the diagnostic notification and the timing of the occurrence of the interference event is within an arbitrary reference range (for example, 1 to several seconds).

[0051] In S208, the in-vehicle device 13 sends history information to the server device 10.

[0052] Steps S209 to S210 of the server device 10 in Figure 2 are executed at any interval, for example, every day to every few days. That is, after the in-vehicle device 13 has executed the processing cycle of steps S201 to S208 multiple times, the server device 10 may execute the processing cycle of steps S209 to S210 once.

[0053] In step S209, the server device 10 processes the history information. A detailed example of the procedure in step S209 is shown in Figure 3.

[0054] Figure 3 shows an example of the operation procedure of the server device 10 related to the processing of historical information. Each step in Figure 3 is executed by the control unit 103.

[0055] In S31, the control unit 103 determines whether or not an interference event occurs. The control unit 103 determines whether or not an interference event occurs by referring to the interference information included in the history information.

[0056] In step S32, if the control unit 103 determines that an interference event exists (Yes in S32), it proceeds to step S33. If it determines that there is no interference event (No in S32), it terminates the procedure shown in Figure 3.

[0057] In S33, the control unit 103 confirms a positive reaction. The control unit 103 determines whether or not there is a positive reaction by referring to the reaction information included in the history information.

[0058] In S34, if the control unit 103 determines that there is no positive response (Yes in S33), it proceeds to step S35. If it determines that there is a positive response (No in S32), it terminates the procedure shown in Figure 3.

[0059] In S35, the control unit 103 generates supplementary information. The control unit 103 generates notification information from the diagnostic results included in the history information using the server diagnostic model 108. For example, from a diagnostic result that diagnoses dangerous driving, the server diagnostic model 108 generates supplementary information such as the history of similar past driving operations, the suspected causes of the dangerous driving operation, and methods to avoid the dangerous driving operation.

[0060] In S36, the control unit 103 generates a supplementary notification. The control unit 103 generates notification information from the diagnostic results included in the history information, with the help of the server agent 109. The server agent 109 generates supplementary notifications such as, "There were X sudden braking incidents this month," "The cause of the sudden braking is likely due to a short following distance," and "Let's increase the following distance."

[0061] In S37, the control unit 103 sends a supplementary notification. In response to this, in step S210 of Figure 2, the supplementary notification is sent from the server device 10 to the user terminal 14. On the other hand, if steps S35 to S37 in Figure 3 are not executed and the procedure in Figure 3 is completed, that is, if no interference event has occurred (No in S32) and there is a high probability that the diagnostic notification has been transmitted to the driver, or if an interference event has occurred (Yes in S32) but there is an affirmative response (No in S34), for example, if the driver recognizes the display or sound of the diagnostic notification even if music is playing, there is a high probability that the diagnostic notification has been transmitted to the driver, so steps S210 and S211 in Figure 2 are omitted.

[0062] Returning to Figure 2, at S211, the user terminal 14 outputs a supplementary notification to the driver. The wording of the supplementary notification is displayed on the display of the output unit 146 or output as audio through the speaker. Outputting such supplementary notifications can help to raise awareness in the driver about their tendency towards dangerous driving. Furthermore, while there is a limit to the amount of information that the in-vehicle device 13 can notify the driver while they are driving, the driver can later review their own driving tendencies.

[0063] By using the above-described procedure, the server device 10 outputs supplementary information to the user terminal 14, allowing the driver to check the supplementary information on the user terminal 14, for example, at the end of driving, and deepen their understanding of the diagnostic results. Furthermore, even if an interference event occurs during driving and there is a high probability that the diagnostic notification has not been adequately conveyed to the driver, the driver can be sure to recognize the diagnostic results through the supplementary information. Therefore, it becomes possible to improve the accuracy of the driver's recognition and understanding of the diagnostic results regarding the operation of the vehicle 12.

[0064] This embodiment also includes cases where some of the steps in Figure 3 are omitted. For example, the server device 10 can generate supplementary information and supplementary notifications and send them to the user terminal 14 regardless of whether or not an interference event occurs. Even in that case, the driver can check the supplementary information on the user terminal 14 at the end of the drive, etc., and review the diagnostic results to deepen their understanding. Furthermore, the server device 10 can generate supplementary information and supplementary notifications and send them to the user terminal 14 regardless of whether or not an affirmative response occurs when an interference event occurs. In that case, the probability that the diagnostic results have not been conveyed to the driver increases due to the interference event, but the driver can check the supplementary information on the user terminal 14 at the end of the drive, etc., and review the diagnostic results to deepen their understanding.

[0065] In the above-described embodiment, the processing and control program that defines the operation of the in-vehicle device 13 and the user terminal 14 may be stored in the storage unit 102 of the server device 10 or in the storage unit of another server device and downloaded to each device via the network 11, or it may be stored in a non-transient recording and storage medium that can be read by each device and read from the medium by each device.

[0066] As described above, embodiments have been explained based on various drawings and examples, but it should be noted that those skilled in the art will find it easy to make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each means, each step, etc., can be rearranged in a logically consistent manner, and multiple means, steps, etc., can be combined into one or divided.

[0067] Some embodiments of the present disclosure are described below. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Note 1] A system comprising an in-vehicle device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device that communicates with the in-vehicle device and the terminal device, The in-vehicle device outputs a notification of the diagnostic results corresponding to the driver's driving to the driver, and sends first information including the content of the notification to the server device. The server device sends to the terminal device second information for outputting information supplementing the notification. system. [Note 2] In Appendix 1, The in-vehicle device further includes in the first information information information indicating whether or not an event that interferes with the notification has occurred when the notification is output. The server device transmits the second information on the condition that the event has occurred. system. [Note 3] In Appendix 2, The aforementioned event is the output of an image or sound that interferes with the notification. system. [Note 4] In Appendix 3, The first information further includes information indicating the driver's response to the notification, The server device further transmits the second information, provided that the response conforms to a predetermined pattern. system. [Note 5] In Appendix 3, The information indicating the aforementioned response is either or both of the driver's captured image and / or the driver's spoken voice. system. [Note 6] In Appendix 1, Information supplementing the diagnostic results includes information showing an analysis of the operation, system. [Note 7] In Appendix 1, The in-vehicle device has a diagnostic model that derives the diagnostic result using information corresponding to the driver's driving, obtained from the vehicle. system. [Note 8] A method for operating a system having an in-vehicle device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device that communicates with the in-vehicle device and the terminal device, The in-vehicle device outputs a notification of the diagnostic results corresponding to the driver's driving to the driver, and sends first information including the content of the notification to the server device. The server device sends to the terminal device second information for the terminal device to output information supplementing the notification, How the system works. [Note 9] In Appendix 8, The in-vehicle device further includes in the first information information information indicating whether or not an event that interferes with the notification has occurred when the notification is output. The server device transmits the second information on the condition that the event has occurred. How the system works. [Note 10] In Appendix 9, The aforementioned event is the output of an image or sound that interferes with the notification. How the system works. [Note 11] In Appendix 10, The first information further includes information indicating the driver's response to the notification, The server device further transmits the second information, provided that the response conforms to a predetermined pattern. How the system works. [Note 12] In Appendix 10, The information indicating the aforementioned response is either or both of the driver's captured image and / or the driver's spoken voice. How the system works. [Note 13] In Appendix 8, Information supplementing the diagnostic results includes information showing an analysis of the operation, How the system works. [Note 14] In Appendix 8, The in-vehicle device has a diagnostic model that derives the diagnostic result using information corresponding to the driver's driving, obtained from the vehicle. How the system works. [Explanation of Symbols]

[0068] 1. Information Processing System 10 Server devices 11 Network 12 vehicles 13 Terminal devices 100 Server Agents 101, 131, 141 Communications Department 102, 132, 142 storage section 103, 133, 143 Control Unit 108 Server Diagnostic Models 139 In-vehicle agent 134, 144 Positioning Unit 135, 145 Input section 136, 146 Output section 138 In-vehicle diagnostic models 139 Automotive information

Claims

1. A system comprising an in-vehicle device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device that communicates with the in-vehicle device and the terminal device, The in-vehicle device acquires the driver's driving operations or driving conditions using a detection unit, outputs a notification of the diagnostic result corresponding to the acquired information to the driver, and sends first information including the content of the notification and information on whether or not an event interfering with the notification has occurred to the server device. The server device sends second information to the terminal device for outputting information supplementing the notification, causing the terminal device to output the second information. system.

2. A system comprising an in-vehicle device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device that communicates with the in-vehicle device and the terminal device, The in-vehicle device outputs a notification of the diagnostic results corresponding to the driver's driving to the driver, and sends first information including the content of the notification to the server device. The server device sends to the terminal device second information for outputting information supplementing the notification. system.

3. In claim 2, The in-vehicle device further includes in the first information information information indicating whether or not an event that interferes with the notification has occurred when the notification is output. The server device transmits the second information on the condition that the event has occurred. system.

4. In claim 3, The aforementioned event is the output of an image or sound that interferes with the notification. system.

5. In claim 4, The first information further includes information indicating the driver's response to the notification, The server device further transmits the second information, provided that the response conforms to a predetermined pattern. system.

6. In claim 5, The information indicating the aforementioned response is either or both of the driver's captured image and / or the driver's spoken voice. system.

7. In claim 2, Information supplementing the diagnostic results includes information showing an analysis of the operation, system.

8. In claim 2, The in-vehicle device has a diagnostic model that derives the diagnostic result using information corresponding to the driver's driving, obtained from the vehicle. system.

9. A method for operating a system having an in-vehicle device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device that communicates with the in-vehicle device and the terminal device, The in-vehicle device outputs a notification of the diagnostic results corresponding to the driver's driving to the driver, and sends first information including the content of the notification to the server device. The server device sends to the terminal device second information for the terminal device to output information supplementing the notification, How the system works.

10. In claim 9, The in-vehicle device further includes in the first information information information indicating whether or not an event that interferes with the notification has occurred when the notification is output. The server device transmits the second information on the condition that the event has occurred. How the system works.

11. In claim 10, The aforementioned event is the output of an image or sound that interferes with the notification. How the system works.

12. In claim 11, The first information further includes information indicating the driver's response to the notification, The server device further transmits the second information, provided that the response conforms to a predetermined pattern. How the system works.

13. In claim 12, The information indicating the aforementioned response is either or both of the driver's captured image and / or the driver's spoken voice. How the system works.

14. In claim 9, Information supplementing the diagnostic results includes information showing an analysis of the operation, How the system works.

15. In claim 9, The in-vehicle device has a diagnostic model that derives the diagnostic result using information corresponding to the driver's driving, obtained from the vehicle. How the system works.

Images