System and method of operation of a system

Abstract

This invention provides a system and a method for operating the system. The objective is to improve the accuracy of a driver's perception and understanding of diagnostic results related to vehicle driving. The system includes an on-board device mounted in a vehicle, a terminal device used by the driver of the vehicle, and a server device communicating with the on-board device and the terminal device. The on-board device outputs a notification of diagnostic results corresponding to the driver's driving to the driver and sends first information containing the content of the notification to the server device. The server device sends second information to the terminal device, the second information being information used to output supplementary information to the notification on the terminal device.

Description

Technical Field

[0001] This disclosure relates to a system and a method of operating the system. Background Technology

[0002] Techniques for diagnosing the state of a moving body based on its movements, including those of vehicles, are known. For example, Patent Documents 1 to 3 disclose a system that processes information obtained by detecting the movements of vehicles, etc., on a cloud server to perform diagnosis of vehicles, etc.

[0003] Prior art literature Patent documents Patent Document 1: Japanese Patent Application Publication No. 2017-13742 Patent Document 2: Japanese Patent Application Publication No. 2019-131187 Patent document 3: Japanese Patent Application Publication No. 2021-196678. Summary of the Invention

[0004] The problem that the invention aims to solve When various diagnostic results regarding vehicle driving are communicated to the driver through AI (Artificial Intelligence), there is room to improve the accuracy of the driver's perception and understanding of these diagnostic results.

[0005] The following section discloses a system that can improve the accuracy of a driver's perception and understanding of diagnostic results regarding vehicle driving.

[0006] Methods for solving problems The system disclosed herein includes an in-vehicle device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device communicating with the in-vehicle device and the terminal device. The in-vehicle device outputs a notification of diagnostic results corresponding to the driver's driving to the driver and sends first information containing the content of the notification to the server device. The server device sends second information to the terminal device, the second information being information used to output supplementary information to the notification via the terminal device.

[0007] Another aspect of the system disclosed herein describes a system operation method comprising: an on-board device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device communicating with the on-board device and the terminal device; wherein the system operation method includes: the on-board device outputting a notification of diagnostic results corresponding to the driver's driving to the driver, and sending first information containing the content of the notification to the server device; the server device sending second information to the terminal device, the second information being information used to output supplementary information to the notification via the terminal device.

[0008] Invention Effects The system and other technologies disclosed herein can improve the accuracy of a driver's perception and understanding of diagnostic results regarding vehicle driving. Attached Figure Description

[0009] Figure 1 A diagram illustrating the structure of an information processing system.

[0010] Figure 2 A sequence diagram to represent an example of the actions of an information processing system.

[0011] Figure 3 A flowchart illustrating an example of the operation of a server device. Detailed Implementation

[0012] The implementation method will be described below.

[0013] Figure 1This diagram illustrates an example of the structure of an information processing system in one embodiment. The information processing system 1 includes one or more server devices 10, vehicle-mounted devices 13, and user terminals 14, all connected via a network 11 to each other in a manner enabling information communication. The server devices 10 are, for example, cloud computing systems or other computing systems, and function as server computers with various installed functions. The vehicle-mounted devices 13 are systems with communication and information processing functions, such as navigation systems, and are mounted on a vehicle 12. The vehicle 12 is a passenger car, commercial vehicle, or a vehicle in which driving is partially or entirely performed manually by the driver. The vehicle 12 is, for example, a gasoline car, a battery electric vehicle (BEV), a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), a fuel cell electric vehicle (FCEV), etc. The user terminal 14 is an information processing terminal used by the driver of the vehicle 12, such as a smartphone, tablet, or PC (personal computer). User terminal 14 corresponds to the "terminal device" in this embodiment. Network 11 is, for example, the Internet, but may also be an ad hoc network, a LAN (local area network), a MAN (metropolitan area network), or other networks or any combination thereof.

[0014] In this embodiment, the information processing system 1 assists in diagnosing the vehicle 12 and notifying the driver of the diagnostic results. The information processing system 1 includes an on-board unit 13 mounted on the vehicle 12, a user terminal 14 used by the driver of the vehicle 12, and a server unit 10 that communicates with the on-board unit 13 and the user terminal 14. The on-board unit 13 outputs a notification of diagnostic results (hereinafter referred to as diagnostic notification) corresponding to the driver's driving to the driver, and sends first information (hereinafter referred to as historical information) containing the content of the diagnostic notification to the server unit 10. The server unit 10 sends second information to the user terminal 14, which is information used to output supplementary information (hereinafter referred to as supplementary information) to the diagnostic notification via the user terminal 14. While the information that can be transmitted to the driver during driving is somewhat limited, by having the server unit 10 output supplementary information to the user terminal 14, the driver can, for example, confirm the supplementary information on the user terminal 14 at the end of the driving session, thereby reviewing the diagnostic results for a deeper understanding. Furthermore, even in situations where there is a high probability that diagnostic notifications may not be fully communicated to the driver due to an event that interferes with the diagnostic notification during driving, the driver can still accurately understand the diagnostic results by obtaining supplementary information. Therefore, this improves the accuracy of the driver's understanding and comprehension of diagnostic results related to the driving of vehicle 12.

[0015] Next, an example of the structure of the server device 10 will be described.

[0016] Server device 10 includes a communication unit 101, a storage unit 102, and a control unit 103. Server device 10 can be a single computer, or it can be composed of two or more computers connected in a manner capable of information communication and operating collaboratively. When server device 10 is composed of two or more computers... Figure 1 The structure shown is appropriately configured in two or more computers.

[0017] The communication unit 101 includes one or more communication interfaces. These communication interfaces may be, for example, LAN interfaces. The communication unit 101 receives information used in the operation of the control unit 103 and transmits information obtained through the operation of the control unit 103. The server device 10 is connected to the network 11 via the communication unit 101 and communicates with the vehicle-mounted device 13 and the user terminal 14 via the network 11.

[0018] Storage unit 102 includes, for example, a main storage device, an auxiliary storage device, or one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or combinations of at least two of them, functioning as a cache memory. Semiconductor memories are, for example, random access memory (RAM) or read-only memory (ROM). RAM is, for example, static random access memory (SRAM) or dynamic random access memory (DRAM). ROM is, for example, electrically erasable programmable read-only memory (EEPROM). Storage unit 102 stores information used in the operation of control unit 103 and information obtained through the operation of control unit 103.

[0019] The control unit 103 includes one or more processors, one or more dedicated circuits, or combinations thereof. The processor may be a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) for specific processing. The dedicated circuit may be, for example, a FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 103 notifies various parts of the server device 10 while performing information processing related to the operation of the server device 10.

[0020] The functions of server device 10 are implemented by executing a control program through a processor included in control unit 103. The control program is a program for enabling the computer to perform the functions corresponding to the steps involved in the operation of server device 10. In other words, the control program is a program for enabling the computer to function as server device 10. Furthermore, some or all of the functions of server device 10 can also be implemented through dedicated circuitry included in control unit 103. Additionally, the control program can be stored in a non-transitory recording / storage medium readable by server device 10 and read from the medium by server device 10.

[0021] In this embodiment, the storage unit 102 stores the server diagnostic model 108 and the server agent 109. The server diagnostic model 108 is an AI model that generates supplementary information suitable for the diagnostic results. The server diagnostic model 108 can be a model that performs machine learning on the correspondence between the diagnostic results and the supplementary information, or it can be a rule-based model that derives the supplementary information from the diagnostic results. The diagnostic results are the results of determining whether the characteristic quantity of the operation performed by the driver belongs to or does not belong to dangerous driving. The supplementary information includes the past history, the presumed reasons, and avoidance suggestions related to the driving operation diagnosed as dangerous driving. The information used to constitute the supplementary information can be obtained, for example, from other servers that provide traffic safety information. The server agent 109 is a module for generating a notification (hereinafter referred to as a supplementary notification) for conveying the supplementary information obtained by the server diagnostic model 108 to the driver, and it has natural language processing capabilities, diagnostic results, and a knowledge base related to the driver's preferences.

[0022] Next, an example of the structure of the vehicle-mounted device 13 will be described.

[0023] The vehicle-mounted device 13 includes a communication unit 131, a storage unit 132, a control unit 133, a positioning unit 134, an input unit 135, an output unit 136, and a detection unit 137. These can be configured as a single control device, or as two or more control devices, or as a combination of a control device and other devices such as communication equipment. Control devices may include, for example, an ECU (Electronic Control Unit). Communication equipment may include, for example, a DCM (Data Communication Module). Each component is connected to the other via an in-vehicle network conforming to standards such as CAN (Controller Area Network) in a manner capable of mutual information communication or communication with equipment in the vehicle 12. Furthermore, the vehicle-mounted device 13 may also be configured to include a component equivalent to the user terminal 14.

[0024] The communication unit 131 includes a communication module corresponding to wired or wireless LAN standards, and a module corresponding to mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The vehicle-mounted device 13 connects to the network 11 through the communication unit 131 and via a nearby router or mobile communication base station, and communicates with other devices through the network 11.

[0025] Storage unit 132 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them. Semiconductor memories are, for example, RAM or ROM. RAM is, for example, SRAM or DRAM. ROM is, for example, EEPROM. Storage unit 132 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. Storage unit 132 stores information used in the operation of control unit 133 and information obtained through the operation of control unit 133.

[0026] The control unit 133 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU, or a dedicated processor such as a GPU for specific processing. The dedicated circuit is, for example, an FPGA or an ASIC. While controlling the various parts of the vehicle-mounted device 13, the control unit 133 performs information processing related to the operation of the vehicle-mounted device 13.

[0027] The functions of the control unit 133, such as navigation and multimedia playback, are implemented by executing a control / processing program through a processor included in the control unit 133. The control / processing program is a program that enables the computer to perform a function corresponding to a step in the operation of the control unit 133 by causing the computer to execute that step. In other words, the control / processing program is a program that enables the computer to function as the control unit 133. Furthermore, some or all of the functions of the control unit 133 can also be implemented through dedicated circuitry included in the control unit 133.

[0028] The positioning unit 134 includes one or more GNSS (Global Navigation Satellite System) receivers. GNSS receivers include, for example, any of GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System), BeiDou, GLONASS (Global Navigation Satellite System), and Galileo. The positioning unit 134 transmits the positioning results to the control unit 133, which then determines the position information of the on-board device 13, i.e., the vehicle 12.

[0029] The input unit 135 includes one or more input interfaces. Input interfaces may include, for example, a microphone for accepting voice input, physical keys, electrostatic capacitive keys, indicator devices, or a touchscreen integrated with a display. The input interface includes an interface with a camera installed on the vehicle 12 to capture images of the interior or exterior of the vehicle 12. The camera may be built into the vehicle-mounted device 13 or be a separate unit. The input unit 135 accepts input from users such as the driver for information used in the operation of the control unit 133, including operations, voice input, or images of the driver captured by the camera, and sends the received information to the control unit 133.

[0030] The output unit 136 includes one or more output interfaces. These output interfaces may be, for example, a speaker for outputting voice or a display for outputting images. The display may be, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 136 outputs information obtained through the operation of the control unit 133.

[0031] The detection unit 137 includes sensor components or interfaces with sensor components that detect various events occurring in the vehicle 12. These sensor components include, for example, sensors for the vehicle 12's speed, longitudinal acceleration, lateral acceleration, deceleration, accelerator operation amount, brake operation amount, steering wheel angle, turn signal illumination time, fuel consumption per unit time, energy-saving mode selection status, odometer reading, safety equipment operating information, engine oil level, brake pad wear, and battery degradation. Furthermore, sensor components include millimeter-wave and infrared radars that detect objects around the vehicle 12. The detection unit 137 sends vehicle information indicating various states of the vehicle 12 detected by the sensor components to the control unit 133.

[0032] The control unit 133 receives and transmits various information to the communication unit 131, storage unit 132, positioning unit 134, input unit 135, output unit 136, and detection unit 137, while controlling these units and the actions of the vehicle 12. While the vehicle 12 is in motion, the control unit 133 provides navigation by providing the driver with route information and other necessary information via the output unit 136, or plays multimedia such as images and music. Furthermore, the control unit 133 controls a portion of the vehicle 12's automatic driving.

[0033] In this embodiment, the storage unit 132 stores the vehicle diagnostic model 138 and the vehicle intelligent agent 139. The vehicle diagnostic model 138 is an AI model that has learned diagnostic results corresponding to diagnostic object information such as characteristic quantities of driving operations or the driver's state. The vehicle diagnostic model 138 diagnoses whether a driving operation or state constitutes dangerous driving based on the diagnostic object information obtained by the detection unit 137. The characteristic quantities of driving operations include information representing control quantities or the motion state of the vehicle 12 during driving operations. The control quantities during driving operations include control quantities of the brakes, accelerator, steering wheel, turn signals, etc. The control quantities include the change in control quantities per unit time. In addition, the information representing the motion state of the vehicle 12 includes the vehicle 12's speed, direction of travel, lateral acceleration, and distance to other vehicles detected by radar or images taken from outside the vehicle 12. The information representing the driver's state during driving includes images taken of the driver, voice, etc. The vehicle-mounted intelligent agent 139 is a module of conversational AI used to generate diagnostic notifications that transmit diagnostic results obtained by the vehicle-mounted diagnostic model 138 to the driver, and has natural language processing capabilities, diagnostic results, and a knowledge base related to the driver's preferences.

[0034] Next, the structure of user terminal 14 will be described.

[0035] User terminal 14 includes a communication unit 141, a storage unit 142, a control unit 143, a positioning unit 144, an input unit 145, and an output unit 146.

[0036] The communication unit 141 includes a communication module corresponding to wired or wireless LAN standards, and a module corresponding to mobile communication standards such as LTE, 4G, or 5G. The user terminal 14 connects to the network 11 through the communication unit 141 and via a nearby router device or a mobile communication base station, and communicates with other devices through the network 11.

[0037] The storage unit 142 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them. Semiconductor memories are, for example, RAM or ROM. RAM is, for example, SRAM or DRAM. ROM is, for example, EEPROM. The storage unit 142 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 142 stores information used in the operation of the control unit 143 and information obtained through the operation of the control unit 143.

[0038] The control unit 143 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU or a dedicated processor such as a GPU, which is dedicated to specific processing. The dedicated circuit is, for example, an FPGA or an ASIC. While controlling various parts of the user terminal 14, the control unit 143 performs information processing related to the actions of the user terminal 14.

[0039] The positioning unit 144 includes one or more GNSS receivers. GNSS receivers include, for example, any of GPS, QZSS, BeiDou, GLONASS, and Galileo. The positioning unit 144 sends the positioning results to the control unit 143, which then determines the location information of the user terminal 14.

[0040] The input unit 145 includes one or more input interfaces. Input interfaces may include, for example, a microphone for accepting voice input, physical keys, electrostatic capacitive keys, indicator devices, a touchscreen integrated with a display, or a camera for capturing images. The input unit 145 accepts input operations for information used in the operation of the control unit 143 and sends the input information to the control unit 143.

[0041] The output unit 146 includes one or more output interfaces. These output interfaces may be, for example, speakers or displays. The displays may be, for example, LCD or OLED displays. The output unit 146 outputs information obtained through the operation of the control unit 143.

[0042] The functions of the control unit 143 are implemented by executing control / processing programs through a processor included in the control unit 143. A control / processing program is a program that enables the computer to perform functions corresponding to the steps involved in the actions of the control unit 143. In other words, the control / processing program is a program that enables the computer to function as the control unit 143. Furthermore, some or all of the functions of the control unit 143 can also be implemented through dedicated circuitry included in the control unit 143.

[0043] Next, use Figure 2 , Figure 3 The operation of information processing system 1 will be explained.

[0044] Figure 2 This is a sequence diagram used to explain the working order of the information processing system 1 in this embodiment. Figure 2 This indicates the sequence of operations related to the association of server device 10, vehicle-mounted device 13, and user terminal 14. Figure 2The steps related to various information processing of the server device 10, vehicle-mounted device 13, and user terminal 14 are executed by their respective control units 103, 133, and 143. Furthermore, the steps related to the transmission and reception of various information of the server device 10, vehicle-mounted device 13, and user terminal 14 are executed by the respective control units 103, 133, and 143 through communication units 101, 131, and 141, respectively. In the server device 10, vehicle-mounted device 13, and user terminal 14, the control units 103, 133, and 143 appropriately store the information they transmit, receive, and process in the storage units 102, 132, and 142, respectively. Moreover, in the vehicle-mounted device 13 and user terminal 14, the control units 133 and 143 accept various information inputs through input units 135 and 145, respectively, and output various information through output units 136 and 146, respectively.

[0045] Figure 2 Steps S201 to S208 of the vehicle-mounted device 13 are executed, for example, at any period of tens of milliseconds to several seconds when the vehicle 12 is in motion.

[0046] In S201, the on-board unit 13 acquires diagnostic target information. This diagnostic target information includes characteristic quantities representing driving operations and information about the driver's state. For example, the control unit 133 acquires diagnostic target information, including control quantities of the brakes, accelerator, steering wheel, and turn signals, the vehicle 12's speed, direction of travel, lateral acceleration, distance to other vehicles, images captured by the driver, and voice input, through various sensor components installed on the vehicle 12 and input interfaces. A timestamp indicating when the information was acquired may also be appended to each piece of information.

[0047] In S202, the vehicle-mounted device 13 performs a diagnosis based on the information of the diagnostic target. Within the vehicle-mounted device 13, the control unit 133 executes the vehicle diagnostic model 138 to perform the diagnosis based on the information of the diagnostic target. For example, as characteristic quantities of driving operations, the vehicle diagnostic model 138 diagnoses whether dangerous driving is occurring based on information representing the motion state of the vehicle 12, such as control quantities of the brakes, accelerator, steering wheel, and turn signals, the vehicle 12's speed, direction of travel, lateral acceleration, and distance from other vehicles, or images and voice recordings taken by the driver.

[0048] In S203, the vehicle-mounted device 13 generates a diagnostic notification based on the diagnostic results. The control unit 133 generates a diagnostic notification to the driver corresponding to the diagnostic results via the vehicle-mounted intelligent agent 139. For example, if the vehicle-mounted intelligent agent 139 diagnoses sudden braking, sudden acceleration, or sudden steering based on the control amount of the brakes, acceleration, etc., it generates a diagnostic notification with statements such as "Danger of sudden braking!", "Danger of sudden acceleration!", or "Please be aware of sudden steering!" to warn of danger. Furthermore, if the driver's state is diagnosed as including drowsiness, fatigue, or decreased attention, it generates a diagnostic notification with statements such as "Please stay alert!" or "Please look ahead!" to encourage alertness and attention. Alternatively, the diagnostic notification may be, in addition to statements, or may replace statements, the flashing of warning lights or the output of warning sounds.

[0049] In S204, the vehicle-mounted device 13 generates interference information. The interference information indicates an event that may occur in the vehicle 12 and interfere with the diagnostic notification to the driver. The interference event may be, for example, a display or voice output that takes precedence over the display or voice output of the diagnostic notification, such as navigation information or road traffic information. Alternatively, the interference event may be the output of music within the vehicle 12 that interferes with the voice output of the diagnostic notification. The control unit 133 generates information indicating the presence of an interference event when the driver is using the navigation system, outputting traffic information, or playing multimedia; otherwise, it generates information indicating the absence of an interference event. In the event of an interference event, the control unit 133 may also include information indicating its category in the interference information.

[0050] In S205, the vehicle-mounted device 13 outputs a diagnostic notification to the driver. However, if the control unit 133 is displaying information that takes priority over the diagnostic notification due to the use of the navigation system or the output of traffic information, it stops displaying the diagnostic notification. Furthermore, if voice output is being performed for information that takes priority over the diagnostic notification, the control unit 133 stops the voice output of the diagnostic notification.

[0051] In S206, the vehicle-mounted device 13 generates reaction information indicating the driver's response to receiving the diagnostic notification. The reaction information indicates whether the driver recognizes the diagnostic notification. The control unit 133 acquires captured image and voice information from the input interface and determines the driver's response based on the captured image and voice information. For example, the control unit 133 performs arbitrary image processing on the captured image to detect facial movements of the driver. Furthermore, the control unit 133 determines that the driver recognizes the diagnostic notification if, for example, a nodding motion is detected; otherwise, it determines that the driver does not recognize it. The control unit 133 analyzes the driver's speech based on the voice information and determines whether it contains statements indicating recognition, such as "yes" or "I understand." Furthermore, the control unit 133 determines that the driver recognizes the diagnostic notification if statements indicating recognition are included; otherwise, it determines that the driver does not recognize it. The control unit 133 generates reaction information indicating whether the driver recognizes the result notification based on the determination result.

[0052] In S207, the vehicle-mounted device 13 generates historical information. The historical information includes diagnostic results, diagnostic notifications, the presence or absence of diagnostic notification output, interference information when a diagnostic notification is output, and reaction information. When the time difference between the timing of the diagnostic notification output and the timing of the interference event is within any reference range (e.g., 1 to several seconds), the control unit 133 can perform processing that treats the output of the diagnostic notification as an interference event.

[0053] In S208, the vehicle-mounted device 13 sends historical information to the server device 10.

[0054] Figure 2 Steps S209 to S210 of the server device 10 are executed, for example, at any cycle of one to several days. That is, after the processing cycle of steps S201 to S208 has been executed multiple times by the vehicle device 13, the processing cycle of steps S209 to S210 performed by the server device 10 can be executed once.

[0055] In S209, server device 10 processes historical information. Figure 3 The following is a detailed sequential example of step S209.

[0056] Figure 3 Example of the operation sequence of server device 10 related to the processing of historical information. Figure 3 Each step is executed by the control unit 103.

[0057] In step S31, the control unit 103 determines whether an interference event has occurred. The control unit 103 refers to the interference information contained in the historical information to determine whether an interference event has occurred.

[0058] In step S32, if the control unit 103 determines that an interference event has occurred (Yes in S32), it proceeds to step S33; if it determines that no interference event has occurred (No in S32), it ends. Figure 3 The order.

[0059] In S33, the control unit 103 confirms the positive reaction. The control unit 103 refers to the reaction information contained in the historical information to determine whether there is a positive reaction.

[0060] In step S34, if the control unit 103 determines that there is no positive response (Yes in S33), it proceeds to step S35; if it determines that there is a positive response (No in S32), it ends. Figure 3 The order.

[0061] In S35, the control unit 103 generates supplementary information. The control unit 103 generates notification information based on the diagnostic results contained in the historical information via the server diagnostic model 108. For example, the server diagnostic model 108 generates supplementary information such as the history of similar past driving operations, the presumed causes of the dangerous driving operations, and methods to avoid the dangerous driving operations, based on the diagnostic results indicating dangerous driving.

[0062] In S36, the control unit 103 generates supplementary notifications. The control unit 103 generates notification information based on the diagnostic results contained in the historical information via the server agent 109. The server agent 109 generates supplementary notifications such as "There have been ○ emergency brakings this month," "The cause of the emergency braking can be attributed to a short workshop distance," and "Please maintain a wider workshop distance."

[0063] In S37, the control unit 103 sends a supplementary notification. Correspondingly, in Figure 2 In step S210, a supplementary notification is sent from the server device 10 to the user terminal 14. On the other hand, in the absence of execution... Figure 3 The process ends with steps S35 to S37. Figure 3 In the case of the following sequence, that is, when there is a high probability that the diagnostic notification will be delivered to the driver even if no interference event occurs (No in S32), or when there is a positive response even if an interference event occurs (Yes in S32) (No in S34), for example, when the driver recognizes the display or voice of the diagnostic notification even while music is playing, the diagnostic notification is more likely to be delivered to the driver, therefore omitted. Figure 2 Steps S210 and S211.

[0064] return Figure 2In S211, the user terminal 14 outputs supplementary notifications to the driver. The supplementary notification statements are displayed on the display of the output unit 146 or output via a speaker in voice form. Such supplementary notification outputs can help prompt the driver to recognize any tendency towards dangerous driving. Furthermore, while the amount of information that can be communicated to the driver in the vehicle via the in-vehicle device 13 is limited, the driver can later review their driving tendencies.

[0065] Through the aforementioned sequence of actions, the server device 10 outputs supplementary information to the user terminal 14, allowing the driver to review the supplementary information on the user terminal 14, for example, at the end of the driving session, thereby reviewing the diagnostic results for a deeper understanding. Furthermore, even in situations where there is a high probability of an interference event occurring during driving and the diagnostic notification not being adequately delivered to the driver, the driver can still gain a clear understanding of the diagnostic results through the supplementary information. Therefore, the accuracy of the driver's understanding and perception of diagnostic results related to the driving of the vehicle 12 can be improved.

[0066] Will Figure 3 In this embodiment, cases where a portion of the sequence is omitted are also included. For example, the server device 10 can generate supplementary information and supplementary notifications and send them to the user terminal 14 regardless of whether an interference event occurs. Even in this case, the driver can check the supplementary information on the user terminal 14 at the end of the drive, thereby reviewing the diagnostic results to deepen understanding. Furthermore, in the event of an interference event, the server device 10 can generate supplementary information and supplementary notifications and send them to the user terminal 14 regardless of whether there is a positive response. In this case, although the possibility of the diagnostic results not being transmitted to the driver due to the interference event is increased, the driver can check the supplementary information on the user terminal 14 at the end of the drive, thereby reviewing the diagnostic results to deepen understanding.

[0067] In the above embodiments, the prescribed processing and control program for the operation of the vehicle-mounted device 13 and the user terminal 14 can be stored in the storage unit 102 of the server device 10 or the storage unit of other server devices, and can be downloaded to each device via the network 11. Alternatively, it can be stored in a non-temporary recording / storage medium that can be read by each device, and can be read from the medium by each device.

[0068] In the foregoing, although the present disclosure has been described with reference to the accompanying drawings and embodiments, it should be noted that various modifications and alterations can be made based on the present disclosure if one is skilled in the art. Therefore, it is important to note that these modifications and alterations are all included within the scope of the present disclosure. For example, the functions included in each unit, step, etc., can be reconfigured in a theoretically consistent manner, and multiple units, steps, etc., can be combined into one or divided.

[0069] In the following description, some embodiments of the present disclosure are illustrated. However, it should be noted that the embodiments of the present disclosure are not limited thereto.

[0070] [Postscript 1] A system comprising an in-vehicle device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device communicating with the in-vehicle device and the terminal device, wherein the in-vehicle device outputs a notification of diagnostic results corresponding to the driver's driving to the driver and sends first information containing the content of the notification to the server device, and the server device sends second information to the terminal device, the second information being information for outputting supplementary information to the notification using the terminal device.

[0071] [Postscript 2] In the system described in Appendix 1, the vehicle-mounted device also includes information in the first information indicating whether an event has occurred that interfered with the notification when it was output, and the server device sends the second information conditioned on the occurrence of the event.

[0072] [Postscript 3] In the system described in Appendix 2, the event is the output of an image or voice that interferes with the notification.

[0073] [Postscript 4] In the system described in Appendix 3, the first information further includes information indicating the driver's response to the notification, and the server device also sends the second information in a predetermined manner based on the response.

[0074] [Postscript 5] In the system described in Appendix 3, the information representing the reaction is either or both of the driver's photographic image and the driver's spoken voice.

[0075] [Postscript 6] In the system described in Appendix 1, the supplementary information to the diagnostic results includes information representing an analysis of the driving.

[0076] [Postscript 7] In the system described in Appendix 1, the on-board device has a diagnostic model that uses information obtained from the vehicle corresponding to the driver's driving to derive the diagnostic results.

[0077] [Postscript 8] A method of operating a system comprising an on-board device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device communicating with the on-board device and the terminal device, the method of operating the system comprising: the on-board device outputting a notification of diagnostic results corresponding to the driver's driving to the driver, and sending first information containing the content of the notification to the server device; the server device sending second information to the terminal device, the second information being information for outputting supplementary information to the notification using the terminal device.

[0078] [Postscript 9] In the system operation method described in Appendix 8, the vehicle-mounted device also includes information indicating the occurrence of an event that interferes with the notification when the notification was output in the first information, and the server device sends the second information conditioned on the occurrence of the event.

[0079] [Postscript 10] In the system operation method described in Appendix 9, the event is the output of an image or voice that interferes with the notification.

[0080] [Postscript 11] In the system operation method described in Appendix 10, the first information further includes information indicating the driver's reaction to the notification, and the server device further sends the second information in a predetermined manner based on the reaction.

[0081] [Postscript 12] In the system operation method described in Appendix 10, the information representing the reaction is either or both of the driver's captured image and the driver's spoken voice.

[0082] [Postscript 13] In the system operation method described in Appendix 8, the information supplementing the diagnostic results includes information representing an analysis of the driving.

[0083] [Postscript 14] In the system operation method described in Appendix 8, the on-board device has a diagnostic model that uses information obtained from the vehicle corresponding to the driver's driving to derive the diagnostic results.

[0084] Symbol Explanation 1…Information processing system; 10… server devices; 11…Network; 12… vehicles; 14…Terminal device; 109…Server Intelligent Agent; 101, 131, 141… Ministry of Communications; Storage units 102, 132, 142… 103, 133, 143… Control Department; 108…Server diagnostic model; 139…In-vehicle intelligent agent; Positioning parts 134, 144… Input sections 135, 145… Output sections 136, 146… 138…On-board diagnostic model.

Claims

1. A system comprising an on-board device mounted on a vehicle, a terminal device used by a driver of the vehicle, and a server device communicating with the on-board device and the terminal device, wherein, The vehicle-mounted device acquires the driver's driving operation or driving status through a detection unit, outputs a diagnostic result notification corresponding to the acquired information to the driver, and sends first information to the server device. The first information includes the content of the notification and information on whether any event has occurred that interferes with the notification. The server device sends second information to the terminal device for outputting supplementary information to the notification using the terminal device, and causes the terminal device to output the second information.

2. A system comprising an on-board device mounted on a vehicle, a terminal device used by a driver of the vehicle, and a server device communicating with the on-board device and the terminal device, wherein, The on-board device outputs a notification of diagnostic results corresponding to the driver's driving to the driver, and sends first information containing the content of the notification to the server device. The server device sends a second message to the terminal device, the second message being information output by the terminal device to supplement the notification.

3. The system as described in claim 2, wherein, The vehicle-mounted device will also include information in the first information indicating whether any events that interfered with the notification occurred when it was output. The server device sends the second information in response to the occurrence of the event.

4. The system as described in claim 3, wherein, The event is the output of an image or voice that interferes with the notification.

5. The system as described in claim 4, wherein, The first information also includes information indicating the driver's response to the notification. The server device also sends the second information in a predetermined manner based on the response.

6. The system of claim 5, wherein, The information representing the reaction is either or both of the driver's photographic image and the driver's spoken voice.

7. The system as claimed in claim 2, wherein, The information supplementing the diagnostic results includes information representing an analysis of the driving.

8. The system as claimed in claim 2, wherein, The on-board device has a diagnostic model that uses information obtained from the vehicle corresponding to the driver's driving to derive the diagnostic results.

9. A method of operating a system, the system comprising an on-board device mounted on a vehicle, a terminal device used by the driver of the vehicle, and a server device communicating with the on-board device and the terminal device, wherein, The system's operating method includes: The vehicle-mounted device outputs a notification of diagnostic results corresponding to the driver's driving to the driver, and sends first information containing the content of the notification to the server device; The server device sends a second message to the terminal device, the second message being information used by the terminal device to output supplementary information to the notification.

10. The method of operating the system as described in claim 9, wherein, The vehicle-mounted device will also include information in the first information indicating whether an event has occurred that interfered with the notification when it was output, and the server device will send the second information if the event has occurred.

11. The method of operating the system as described in claim 10, wherein, The event is the output of an image or voice that interferes with the notification.

12. The method of operating the system as described in claim 11, wherein, The first information also includes information indicating the driver's response to the notification. The server device also sends the second information in a predetermined manner based on the response.

13. The method of operating the system as described in claim 12, wherein, The information representing the reaction is either or both of the driver's photographic image and the driver's spoken voice.

14. The method of operating the system as described in claim 9, wherein, The information supplementing the diagnostic results includes information representing the analysis of the driver.

15. The method of operating the system as described in claim 9, wherein, The on-board device has a diagnostic model that uses information obtained from the vehicle corresponding to the driver's driving to derive the diagnostic results.

Images