System and System Operation Method

Abstract

This improves driver convenience when notifying drivers of diagnostic results regarding vehicle operation. [Solution] The system comprises a terminal device used by the driver of a vehicle and a server device that communicates with the terminal device, wherein the terminal device has a first diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving in a first time, and the server device has a second diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving in a second time longer than the first time, and the terminal device holds first information from the information acquired when the driver is driving that is used for diagnosis by the first diagnostic model, and sends second information other than the first information to the server device for use in diagnosis by the second diagnostic model.

Description

Technical Field

[0001] The present disclosure relates to a system and a method for operating the system.

Background Art

[0002] Techniques for diagnosing the state of a moving body based on the behavior of the moving body including a vehicle are known. For example, Patent Documents 1 to 3 disclose systems that detect information obtained by detecting the behavior of a vehicle or the like and perform diagnosis of the vehicle or the like by processing the information on a server on the cloud.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] When various diagnostic results regarding the driving of a vehicle are notified to a driver by AI (Artificial Intelligence) or the like, there is room for improvement in convenience for the driver.

[0005] Hereinafter, a system or the like that enables improvement in the convenience of a driver when notifying the driver of diagnostic results regarding the driving of a vehicle will be disclosed.

Means for Solving the Problems

[0006] The system in this disclosure is a system comprising a terminal device used by a vehicle driver and a server device that communicates with the terminal device, wherein the terminal device has a first diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving in a first time, and the server device has a second diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving in a second time longer than the first time, and the terminal device holds first information from information acquired during the driver's driving that is used for diagnosis by the first diagnostic model, and sends second information other than the first information to the server device for use in diagnosis by the second diagnostic model.

[0007] A method for operating a system in another aspect of the present disclosure is a method for operating a system having a terminal device used by a vehicle driver and a server device that communicates with the terminal device, wherein the terminal device has a first diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving in a first time, and the server device has a second diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving in a second time that is longer than the first time. The terminal device includes storing first information used for diagnosis by the first diagnostic model from the information acquired when the driver is driving, and sending second information other than the first information to the server device for use in diagnosis by the second diagnostic model. [Effects of the Invention]

[0008] The system described in this disclosure makes it possible to improve driver convenience when notifying the driver of diagnostic results regarding vehicle operation. [Brief explanation of the drawing]

[0009] [Figure 1] This is a diagram showing an example of the configuration of an information processing system. [Figure 2] This is a sequence diagram illustrating an example of the operation of an information processing system. [Figure 3] This is a flowchart illustrating an example of the operation of an in-vehicle device. [Figure 4] This is a sequence diagram showing an example of the operation of an information processing system in a modified case. [Figure 5] This flowchart shows an example of the operation of an in-vehicle device in a modified case. [Modes for carrying out the invention]

[0010] The embodiments will be described below.

[0011] Figure 1 shows an example of the configuration of an information processing system in one embodiment. The information processing system 1 has one or more server devices 10, an in-vehicle device 13, and a user terminal 14, each connected to each other via a network 11 in a manner that enables information communication. The server device 10 is, for example, a server computer belonging to a cloud computing system or other computing system, and functions as a server that implements various functions. The in-vehicle device 13 is, for example, a navigation system, which has communication functions and information processing functions, and is mounted on a vehicle 12. The vehicle 12 is a passenger car, a commercial vehicle, etc., in which part or all of the driving is done manually by the driver. The vehicle 12 is, for example, a gasoline car, an electric vehicle (BEV; Battery Electric Vehicle), a hybrid vehicle (HEV; Hybrid Electric Vehicle), a plug-in hybrid vehicle (PHEV; Plug-in Hybrid Electric Vehicle), a fuel cell vehicle (FCEV; Fuel Cell Electric Vehicle), etc. The user terminal 14 is an information processing terminal used by the driver of the vehicle 12, and is, for example, a smartphone, tablet, or PC (Personal Computer). The network 11 is, for example, the internet, but may also be an ad-hoc network, LAN, MAN (Metropolitan Area Network), or other network, or any combination thereof.

[0012] In this embodiment, the information processing system 1 supports the diagnosis of the vehicle 12 and the notification of the diagnosis results to the driver. The information processing system 1 includes an in-vehicle device 13 or a user terminal 14 as a "terminal device" used by the driver of the vehicle 12, and a server device 10 that communicates with the in-vehicle device 13 and the user terminal 14. The in-vehicle device 13 has a first diagnostic model (hereinafter referred to as the in-vehicle diagnostic model) 138 for notifying the driver of the diagnosis results corresponding to the driver's driving in a first time. The server device 10 also has a second diagnostic model (hereinafter referred to as the server diagnostic model) 108 for notifying the driver of the diagnosis results corresponding to the driver's driving in a second time that is longer than the first time. The in-vehicle device 13 holds information (hereinafter referred to as in-vehicle information) 139 used for diagnosis by the in-vehicle diagnostic model 138 from the information acquired when the driver is driving (hereinafter referred to as target information), and sends information other than the in-vehicle information 139 to the server device 10 for use in diagnosis by the server diagnostic model 108. Therefore, the in-vehicle information 139 is stored in the in-vehicle device 13, and information other than the in-vehicle information 139 sent from the in-vehicle device 13 to the server device 10 is stored in the server device 10 as server information 109. The in-vehicle device 13 performs a diagnosis using the in-vehicle information 139 with the in-vehicle diagnostic model 138 and notifies the driver of the diagnosis results, and can also train or enhance the in-vehicle diagnostic model 138 with the in-vehicle information 139. On the other hand, the server device 10 performs a diagnosis using the server information 109 with the server diagnostic model 108 and notifies the driver of the diagnosis results via the user terminal 14, and can also train or enhance the server diagnostic model 108 with the server information 109.

[0013] When attempting to derive diagnostic results based on the driver's driving using the server diagnostic model 108 and notify the driver from the server device 10, the process of sending information acquired according to the driving from the in-vehicle device 13 to the server device 10 and then sending the diagnostic results from the server device 10 to the in-vehicle device 13 may result in a long delay before the driver receives notification of the diagnostic results due to communication delays, etc. In this embodiment, however, diagnostic results that require notification at a rapid time, such as nearly real-time, can be obtained in the in-vehicle device 13 using the in-vehicle diagnostic model 138 and notified to the driver. On the other hand, diagnostic results that do not require a certain amount of time to be notified can be obtained in the server device 10 using the server diagnostic model 108 and communicated to the driver. Therefore, the in-vehicle device 13 can obtain the necessary notifications from the driver in a shorter time compared to when communicating with the server device 10. In other words, it is possible to improve driver convenience when notifying the driver of diagnostic results regarding the driving of the vehicle 12.

[0014] Next, an example configuration of the server device 10 will be described.

[0015] The server device 10 includes a communication unit 101, a storage unit 102, and a control unit 103. The server device 10 may be a single computer, or it may consist of two or more computers that are connected and operate in coordination with each other. When the server device 10 consists of two or more computers, the configuration shown in Figure 1 is appropriately arranged across the two or more computers.

[0016] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the control unit 103 and transmits information obtained through the operation of the control unit 103. The server device 10 is connected to the network 11 by the communication unit 101 and communicates information with the in-vehicle device 13 and the user terminal 14 via the network 11.

[0017] The storage unit 102 includes, for example, one or more semiconductor memories that function as a main memory device, an auxiliary memory device, or a cache memory, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memory is, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The RAM is, for example, a SRAM (Static RAM) or a DRAM (Dynamic RAM). The ROM is, for example, an EEPROM (Electrically Erasable Programmable ROM). The storage unit 102 stores information used for the operation of the control unit 103 and information obtained by the operation of the control unit 103.

[0018] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for specific processing. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the server device 10 while controlling each part of the server device 10.

[0019] The functions of the server device 10 are realized by executing a control program on the processor included in the control unit 103. The control program is a program for causing a computer to execute the processing of the steps included in the operation of the server device 10, so as to cause the computer to realize the functions corresponding to the processing of those steps. That is, the control program is a program for causing a computer to function as the server device 10. Also, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Further, the control program may be stored in a non-transitory recording and storage medium readable by the server device 10, and the server device 10 may read it from the medium.

[0020] In this embodiment, the storage unit 102 stores the server diagnosis model 108, the server information 109, and the server agent 100. The server diagnosis model 108 is an AI model that has learned the diagnosis results corresponding to the past server information 109 in order to diagnose the driving tendency of the driver corresponding to the server information 109. The server information 109 is information for diagnosing the driving tendency of the driver, and includes, for example, information indicating the number of dangerous driving occurrences, the degree of vehicle wear, and the like. Dangerous driving is diagnosed, for example, in the in-vehicle device 13 as will be described later. Also, information indicating the degree of wear of the vehicle 12 is acquired in the in-vehicle device 13. The information indicating the degree of wear of the vehicle 12 is information such as the remaining oil amount, the degree of deterioration of the brake pads, and the degree of deterioration of the in-vehicle battery. The server device 10 acquires the server information 109 including the diagnosis history from the in-vehicle device 13. The server agent 100 is an interactive AI module for generating a notification for conveying the diagnosis result by the server diagnosis model 1088 to the driver, and has a natural language processing function, a knowledge base regarding the diagnosis result and the driver's preferences, and the like. The server diagnosis model 108 executed by the control unit 103 diagnoses, for example, a tendency of dangerous driving when the number of dangerous driving occurrences is above a certain level. Also, the server diagnosis model 108 diagnoses, for example, a tendency of vehicle wear when the degree of wear of the vehicle is above a certain level. The criteria for diagnosis with respect to the number of dangerous driving occurrences and the degree of wear of the vehicle may be set appropriately or may be generated in the server diagnosis model 108 during the learning process. Then, the server agent 100 executed by the control unit 133 generates a notification to the driver according to the diagnosis result.

[0021] Next, a configuration example of the in-vehicle device 13 will be described.

[0022] The in-vehicle device 13 includes a communication unit 131, a storage unit 132, a control unit 133, a positioning unit 134, an input unit 135, an output unit 136, and a detection unit 137. These may be configured as a single control device, or as two or more control devices, or as a control device and other devices such as a communication device. The control device includes, for example, an ECU (Electronic Control Unit). The communication device includes, for example, a DCM (Data Communication Module). Each unit is connected to each other or to the equipment of the vehicle 12 via an in-vehicle network compliant with standards such as CAN (Controller Area Network) to enable information communication. The in-vehicle device 13 may also include equipment equivalent to a user terminal 14.

[0023] The communication unit 131 includes a communication module compatible with wired or wireless LAN standards, a module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The in-vehicle device 13 is connected to the network 11 via the communication unit 131 through a nearby router device or mobile communication base station, and communicates information with other devices via the network 11.

[0024] The storage unit 132 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memory is, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The storage unit 132 functions, for example, as a main memory, auxiliary memory, or cache memory. The storage unit 132 stores information used in the operation of the control unit 133 and information obtained by the operation of the control unit 133.

[0025] The control unit 133 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processors are general-purpose processors such as CPUs, or dedicated processors such as GPUs specialized for specific processing. The dedicated circuits are, for example, FPGAs or ASICs. The control unit 133 controls each part of the in-vehicle device 13 and performs information processing related to the operation of the in-vehicle device 13.

[0026] The functions of the control unit 133 are realized by executing a control / processing program on the processor included in the control unit 133. The control / processing program is a program that causes the computer to execute the processing steps included in the operation of the control unit 133, thereby realizing the functions corresponding to the processing of those steps. In other words, the control / processing program is a program that causes the computer to function as the control unit 133. Furthermore, some or all of the functions of the control unit 133 may be realized by dedicated circuits included in the control unit 133.

[0027] The positioning unit 134 includes one or more GNSS (Global Navigation Satellite System) receivers. GNSS includes, for example, GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System), BeiDou, GLONASS (Global Navigation Satellite System), and Galileo. The positioning unit 134 sends the positioning result to the control unit 133, which then obtains the location information of the in-vehicle device 13, i.e., the vehicle 12.

[0028] The input unit 135 includes one or more input interfaces. These input interfaces include, for example, a microphone for receiving voice input, physical keys, capacitive keys, a pointing device, or a touchscreen integrated with a display. The input interface also includes an interface with a camera provided in the vehicle 12 that captures images of the interior or exterior of the vehicle 12. The camera may be built into the in-vehicle device 13 or it may be a separate unit. The input unit 135 receives information used for the operation of the control unit 133, such as user input (operations, voice, etc.) or images of the driver, etc., captured by the camera, and sends the received information to the control unit 133.

[0029] The output unit 136 includes one or more output interfaces. These output interfaces include, for example, a speaker for outputting sound, a display for outputting images, etc. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 136 outputs information obtained through the operation of the control unit 133.

[0030] The detection unit 137 has sensors that detect various events occurring in the vehicle 12, or an interface with sensors. The sensors include, for example, sensors that detect the vehicle 12's speed, longitudinal acceleration, lateral acceleration, deceleration, accelerator operation amount, brake operation amount, steering angle, turn signal illumination time, fuel consumption per unit time, eco mode selection status, odometer value, safety equipment operation information, remaining engine oil, brake pad wear, battery degradation, etc. The sensors also include millimeter wave, infrared, and other radars that detect targets around the vehicle 12. The detection unit 137 sends vehicle information indicating various states of the vehicle 12 detected by the sensors to the control unit 133.

[0031] The control unit 133 controls the communication unit 131, storage unit 132, positioning unit 134, input unit 135, output unit 136, and detection unit 137 while exchanging various information with these units, and also controls the operation of the vehicle 12. When the vehicle 12 is running, the control unit 133 provides navigation functions by presenting various information such as route information necessary for driving to the driver via the output unit 136, and also controls the partial autonomous driving of the vehicle 12.

[0032] In this embodiment, the memory unit 132 stores the in-vehicle diagnostic model 138, in-vehicle information 139, and the in-vehicle agent 130. The in-vehicle diagnostic model 138 is an AI model that has learned past diagnostic results corresponding to the in-vehicle information 139 in order to diagnose the characteristics of the driver's operations or the driver's state corresponding to the in-vehicle information 139. The in-vehicle information 139 is information indicating the driver's operation characteristics or the state during driving. The operation characteristics include information indicating the control quantities in driving operations or the motion state of the vehicle 12. The control quantities in driving operations are control quantities such as brakes, accelerators, steering, and turn signals. The control quantities include the amount of change in the control quantities per unit time. The information indicating the motion state of the vehicle 12 includes the vehicle's speed, acceleration in the direction of travel and lateral direction, the distance to other vehicles detected by radar or images captured from outside the vehicle 12, etc. The information indicating the driver's state during driving includes the driver's captured images, voice, etc. The in-vehicle agent 130 is an interactive AI module for generating notifications to the driver that convey the diagnostic results of the in-vehicle diagnostic model 138, and has natural language processing capabilities, a knowledge base on diagnostic results and driver preferences, etc. The in-vehicle diagnostic model 138, executed by the control unit 133, for example, diagnoses dangerous driving if the control amount of the driving operation or the motion state of the vehicle 12 indicates a value corresponding to dangerous driving. The in-vehicle diagnostic model 138 also diagnoses fatigue accumulation if, for example, captured images, audio, etc., indicating the driver's state while driving indicate the driver's drowsiness, fatigue, etc. The criteria for diagnosing dangerous driving and fatigue accumulation may be set as appropriate, or may be generated in the in-vehicle diagnostic model 138 during the learning process. The in-vehicle agent 130 then generates a notification to the driver according to the diagnostic results.

[0033] Next, we will describe an example configuration of user terminal 14.

[0034] The user terminal 14 includes a communication unit 141, a storage unit 142, a control unit 143, a positioning unit 144, an input unit 145, and an output unit 146.

[0035] The communication unit 141 includes a communication module compatible with wired or wireless LAN standards, a module compatible with mobile communication standards such as LTE, 4G, or 5G, etc. The user terminal 14 is connected to the network 11 via the communication unit 141 through a nearby router device or mobile communication base station, and communicates information with other devices via the network 11.

[0036] The storage unit 142 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memory is, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The storage unit 142 functions, for example, as a main memory, auxiliary memory, or cache memory. The storage unit 142 stores information used in the operation of the control unit 143 and information obtained by the operation of the control unit 143.

[0037] The control unit 143 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU, or a dedicated processor specialized for a specific process such as a GPU. The dedicated circuit is, for example, an FPGA or ASIC. The control unit 143 controls each part of the user terminal 14 and performs information processing related to the operation of the user terminal 14.

[0038] The positioning unit 144 includes one or more GNSS receivers. GNSS includes, for example, GPS, QZSS, BeiDou, GLONASS, and Galileo. The positioning unit 144 sends the positioning result to the control unit 143, which then obtains the location information of the user terminal 14.

[0039] The input unit 145 includes one or more input interfaces. These input interfaces include, for example, a microphone for receiving voice input, physical keys, capacitive keys, a pointing device, a touchscreen integrated with a display, and a camera for capturing images. The input unit 145 receives an operation to input information used in the operation of the control unit 143 and sends the input information to the control unit 143.

[0040] The output unit 146 includes one or more output interfaces. The output interfaces are, for example, speakers, displays, etc. The displays are, for example, LCDs or organic EL displays. The output unit 146 outputs information obtained by the operation of the control unit 143.

[0041] The functions of the control unit 143 are realized by executing a control / processing program on the processor included in the control unit 143. The control / processing program is a program that causes the computer to execute the processing steps included in the operation of the control unit 143, thereby realizing the functions corresponding to the processing of those steps. In other words, the control / processing program is a program that causes the computer to function as the control unit 143. Furthermore, some or all of the functions of the control unit 143 may be realized by dedicated circuits included in the control unit 143.

[0042] Next, the operation of the information processing system 1 will be explained using Figures 2 to 5.

[0043] Figure 2 is a sequence diagram illustrating the operation procedure of the information processing system 1 in this embodiment. Figure 2 shows the procedure for the coordinated operation of the server device 10, the in-vehicle device 13, and the user terminal 14. The steps related to various information processing of the server device 10, the in-vehicle device 13, and the user terminal 14 in Figure 2 are executed by the respective control units 103, 133, and 143. Furthermore, the steps related to the sending and receiving of various information of the server device 10, the in-vehicle device 13, and the user terminal 14 are executed by the respective control units 103, 133, and 143 sending and receiving information to each other via the communication units 101, 131, and 141, respectively. In the server device 10, the in-vehicle device 13, and the user terminal 14, the control units 103, 133, and 143 respectively appropriately store the information to be sent, received, and processed in the storage units 102, 132, and 142. Furthermore, in the in-vehicle device 13 and the user terminal 14, the control units 133 and 143 receive various types of information via input units 135 and 145, respectively, and output various types of information via output units 136 and 146, respectively.

[0044] Steps S201 to S209 of the in-vehicle device 13 in Figure 2 are executed at arbitrary intervals of several tens of milliseconds to several seconds, for example, when the vehicle 12 is in motion.

[0045] In S201, the in-vehicle device 13 acquires target information. The control unit 133 acquires target information, including control values ​​such as brakes, accelerators, steering wheel, and turn signals, the vehicle's speed, acceleration in the direction of travel and lateral direction, distance to other vehicles, captured images of the driver, and voice, using various sensors and input interfaces provided on the vehicle 12. Each piece of information may be accompanied by a timestamp at the time of acquisition.

[0046] In S202, the in-vehicle device 13 sorts the target information. The control unit 133 sorts the target information into in-vehicle information 139 and other server information 109. For example, the control unit 133 sorts the target information by labeling each type of target information to indicate whether it is for in-vehicle use or server use. The storage unit 132 stores, for example, reference information that has been pre-sorted into in-vehicle information 139 and server information 109 for each type of target information, and the control unit 133 refers to the reference information and labels the acquired target information according to its type.

[0047] In S203, the in-vehicle device 13 sends the server information 109 to the server device 10. After sending the server information 109 to the server device 10, the in-vehicle device 13 may erase the server information 109 from the storage unit 102 to free up storage capacity.

[0048] In S204, the in-vehicle device 13 stores in-vehicle information 139.

[0049] In S205, the in-vehicle device 13 performs a diagnosis using the in-vehicle information 139. The in-vehicle device 13 executes the in-vehicle diagnostic model 138 via the control unit 133, and performs a diagnosis using the sorted in-vehicle information 139.

[0050] In S206, the in-vehicle device 13 generates a notification based on the diagnostic results. If the in-vehicle agent 130 diagnoses, for example, sudden braking, sudden acceleration, or sudden steering based on the amount of brake control, acceleration, etc., the control unit 133 generates a notification with phrases such as "Sudden braking is dangerous!", "Sudden acceleration is dangerous!", or "Caution: Sudden steering!" to warn of danger. If the control unit 133 diagnoses that the driver's condition includes drowsiness, fatigue, decreased attention, etc., it generates a notification with phrases such as "Wake up!" or "Look ahead!" to encourage wakefulness and attention. Alternatively, the notification may include, in addition to or instead of phrases, flashing warning lights or outputting warning sounds.

[0051] In S207, the in-vehicle device 13 outputs a notification to the driver. The notification text may be displayed on the display of the output unit 136 or output as audio through the speaker. A warning light may also flash or a warning sound may be emitted. Outputting such a notification can help deter dangerous driving or drowsiness by the driver.

[0052] In step S208, the in-vehicle device 13 evaluates the driver's response to the notification. A detailed example of step S208 is shown in Figure 3.

[0053] Figure 3 is a flowchart illustrating an example of the procedure for evaluating the driver's reaction in the in-vehicle device 13. Each step in Figure 3 is an information processing step performed by the control unit 133.

[0054] In S31, the control unit 133 identifies the in-vehicle information 139. The control unit 133 reads the in-vehicle information 139 based on the diagnosis from the storage unit 102 and identifies it.

[0055] In S32, the control unit 133 acquires response information. Response information is information indicating the driver's response to a notification, and includes the driver's captured image, audio information, etc. The control unit 133 acquires the captured image and audio information from the input interface.

[0056] In S33, the control unit 133 determines whether the reaction is positive or negative. The control unit 133 determines a positive or negative reaction based, for example, on the driver's facial expression in the captured image. For example, the control unit 133 performs arbitrary image processing on the captured image to extract the driver's face image and detect landmarks in the face image, and determines whether the driver's facial expression belongs to a pattern of facial expression indicating discomfort based on the shape of the arrangement of the landmarks. Alternatively, the control unit 133 analyzes the driver's speech based on audio information and determines whether it contains words or phrases indicating discomfort. Information such as patterns of facial expressions indicating discomfort and words or phrases is set in advance as appropriate. The control unit 133 then determines that the reaction is negative if the driver's facial expression or speech indicates discomfort, and positive otherwise. If the control unit 133 determines that the reaction is negative (Yes in step S34), it proceeds to step S35. If it determines that the reaction is positive, that is, if it does not determine that the reaction is negative (Yes in step S34), it bypasses step S35 and terminates the procedure shown in Figure 3.

[0057] In S35, the control unit 133 separates the in-vehicle information 139 identified in step S31 as server information 109.

[0058] Returning to Figure 2, in step S209, the in-vehicle device 13 updates the criteria for classifying the target information. For example, the control unit 133 of the in-vehicle device 13 updates the reference information that serves as the criterion for classifying the target information into in-vehicle information 139 or server information 109, based on the results of the classification in step S35 of Figure 3. The control unit 133 updates the criteria, for example, by changing the labels for each type of target information. In this way, the target information that has been classified from in-vehicle information 139 to server information 109 will be processed as server information 109 in subsequent processing.

[0059] Steps S211 to S213 of the server device 10 in Figure 2 are executed at any interval, for example, every few days to every few weeks.

[0060] In S211, the server device 10 performs a diagnosis using the server information 109. The server device 10 executes the server diagnostic model 108 via the control unit 103, and a diagnosis is performed using the server information 109.

[0061] In S212, the server device 10 generates notifications based on the diagnostic results. If the server agent 100 diagnoses a tendency towards dangerous driving, the control unit 103 generates a notification containing cautionary phrases such as "Speed ​​more carefully," "Apply the brakes earlier," "Take curves slowly," "Use your turn signals earlier," and "Maintain a safe distance between vehicles." Alternatively, if the control unit 103 diagnoses a tendency towards vehicle wear and tear, it generates a notification containing cautionary phrases such as "It's time for vehicle maintenance" and "Check your brakes."

[0062] In S213, the in-vehicle device 13 sends notification information to the user terminal 14. The notification information is information for outputting a notification.

[0063] In S214, the user terminal 14 outputs a notification to the driver. The notification text is displayed on the display of the output unit 146 or output as audio through the speaker. Outputting such notifications can help to raise awareness of the driver's tendency towards dangerous driving and to encourage maintenance and inspection of the vehicle 12. Furthermore, while there is a limit to the amount of information that the in-vehicle device 13 can notify the driver while driving, the driver can later review their own driving tendencies.

[0064] When server information 109 is sent from the in-vehicle device 13 in step S203, the server device 10 updates the server information 109 in step S215 as needed. For example, the control unit 103 updates information indicating the degree of wear and tear of the vehicle 12, such as the oil level, the degree of deterioration of the brake pads, and the degree of deterioration of the in-vehicle battery, to the latest information and stores it in the storage unit 102. In this way, information indicating the degree of wear and tear of the vehicle 12 is updated as needed and used for diagnosis by the server diagnostic model 108 later.

[0065] Through the operation procedure described above, the in-vehicle device 13 stores in-vehicle information 139 and updated reference information. Therefore, the in-vehicle device 13 can use the accumulated in-vehicle information 139 and reference information as training data at any time to train the in-vehicle diagnostic model 138. In this way, it becomes possible to adjust the in-vehicle diagnostic model 138 to output diagnostic results that are more suitable for the driver.

[0066] Through the operation procedure described above, the in-vehicle device 13 stores in-vehicle information 139 and updated reference information. Therefore, the in-vehicle device 13 can use the stored in-vehicle information 139 and reference information as training data at any time to train the in-vehicle diagnostic model 138. In this way, it becomes possible to adjust the in-vehicle diagnostic model 138 to output diagnostic results that are more suitable for the driver.

[0067] Furthermore, since the server device 10 stores server information 109 sent from multiple vehicles 12, this server information 109 may be used as training data at any time to train the server diagnostic model 108.

[0068] Figure 4 is a sequence diagram showing an example of the operation procedure of the information processing system 1 in a modified example. The example of the operation procedure in Figure 4 is an example in which the in-vehicle device 13 operates instead of the user terminal 14, and differs from Figure 2 in that steps S213' and S214' are executed by the in-vehicle device 13 instead of steps S213 and S214 in Figure 2. That is, in step S213', the server device 10 sends notification information based on the diagnostic result to the in-vehicle device 13. Then, in step S214', the in-vehicle device 13 outputs a notification in the same way as the user terminal 14 does in step S214 in Figure 2.

[0069] As shown in Figure 2 or Figure 4, the in-vehicle device 13 notifies the driver of the diagnostic results in near real-time, depending on the driver's driving operations or driving conditions. Meanwhile, the server device 10 generates diagnostic results based on the driver's driving tendencies, which are acquired over time, for example, several days to several weeks. The notification is then output to the driver by the in-vehicle device 13 or the user terminal 14. When the user terminal 14 outputs the notification as in the example in Figure 2, the driver can receive notifications about their own driving tendencies even when they are away from the vehicle 12.

[0070] Furthermore, by evaluating the driver's response to notifications through the procedure shown in Figure 3, if the driver finds the notifications output by the in-vehicle diagnostic model 138, which are generated in near real-time, bothersome, the relevant information based on such notifications can be filtered out so that the in-vehicle diagnostic model 138 does not use it again, and sent to the server device 10 as server information 109, thereby reducing bothersome notifications.

[0071] Figure 5 is a flowchart illustrating an example procedure for evaluating the driver's response to a notification in a further modified case. The example procedure in Figure 5 differs from the example procedure in Figure 3 in that steps S34' and S34'' are inserted after step S34.

[0072] If the driver's response in step S34 is negative (Yes), the control unit 133 of the in-vehicle device 13 adds the count value of the negative response associated with each piece of target information in step S34' and proceeds to step S34''. If the response is not negative (No), it bypasses step S34' and proceeds to step S34''.

[0073] In step S34'', if the count value is greater than or equal to an arbitrary reference value, for example, any value between 3 and 10 times (Yes), the control unit 133 separates the in-vehicle information 139 identified in step S31 as server information 109. If the count value is less than the arbitrary reference value (No), it bypasses step S35 and terminates the process.

[0074] According to this modification, the risk of unnecessarily classifying the in-vehicle information 139 as server information 109 when the driver accidentally makes a negative reaction can be reduced.

[0075] In a further modification, instead of evaluating the driver's response, the in-vehicle device 13 may separate the in-vehicle information 139 as server information 109 in response to explicit input from the driver, such as a touch operation on a button on the touch panel to stop notification of diagnostic results. Such operation input may be conditional on being made within any time after notification of the diagnostic results, for example, within a few seconds to 10 seconds. Doing so makes it possible to more reliably separate the target information in accordance with the driver's intentions.

[0076] According to the embodiment described above, the driver can obtain near real-time diagnostic results about their driving operations or condition according to their preferences, and can also obtain diagnostic results about their driving tendencies after a certain period of time, allowing them to review their tendencies. Therefore, it is possible to improve the driver's convenience when notifying the driver of diagnostic results regarding the driving of the vehicle 12. In addition, by sending server information 109 that does not require near real-time diagnosis to the server device 10, the in-vehicle device 13 can save resources.

[0077] In the embodiments described above, the order of steps S211 to S213 performed by the server device 10 in Figure 2 and the order of each step in the in-vehicle device 13 are not limited to the example in Figure 2. Also, the order of steps S211 to S213' performed by the server device 10 in Figure 4 and the order of steps S201 to S209 in the in-vehicle device 13 are not limited to the example in Figure 4.

[0078] In the above-described embodiment, the processing and control program that defines the operation of the vehicle 12 and the user terminal 14 may be stored in the storage unit 102 of the server device 10 or in the storage unit of another server device and downloaded to each device via the network 11, or it may be stored in a non-transient recording and storage medium that can be read by each device and read from the medium by each device.

[0079] As described above, embodiments have been explained based on various drawings and examples, but it should be noted that those skilled in the art will find it easy to make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each means, each step, etc., can be rearranged in a logically consistent manner, and multiple means, steps, etc., can be combined into one or divided.

[0080] Some embodiments of the present disclosure are described below. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Note 1] A system comprising a terminal device used by the vehicle driver and a server device that communicates with the terminal device, The terminal device has a first diagnostic model for notifying the driver of the diagnostic results corresponding to the driver's driving in a first time, The server device has a second diagnostic model for notifying the driver of the diagnostic results corresponding to the driver's driving in a second time that is longer than the first time. The terminal device holds first information used for diagnosis by the first diagnostic model from the information acquired during the driver's operation, and sends second information other than the first information to the server device for use in diagnosis by the second diagnostic model. system. [Note 2] In Appendix 1, The terminal device does not send the first information to the server device. system. [Note 3] In Appendix 1 or 2, The terminal device trains the first diagnostic model using the first information. system. [Note 4] In any of the appendices 1 to 3, The server device trains the second diagnostic model using the second information. system. [Note 5] In any of the appendices 1 to 4, The first information is information relating to the driver's driving operations or driving conditions, The information in the second above is information regarding the driver's driving tendencies. system. [Note 6] In any of the appendices 1 to 5, The terminal device determines the first and second pieces of information from the information acquired during the driver's operation, in accordance with a predetermined operation performed by the driver. system. [Note 7] In any of the appendices 1 to 6, The terminal device determines the first and second pieces of information from among the information acquired by the driver during operation, in accordance with the driver's response to the diagnostic results of the first diagnostic model. system. [Note 8] In Appendix 7, The terminal device acquires the response based on the driver's captured image or voice. system. [Note 9] A method for operating a system having a terminal device used by a vehicle driver and a server device that communicates with the terminal device, The terminal device has a first diagnostic model for notifying the driver of the diagnostic results corresponding to the driver's driving in a first time, The server device has a second diagnostic model for notifying the driver of the diagnostic results corresponding to the driver's driving in a second time that is longer than the first time, The terminal device includes storing first information used for diagnosis by the first diagnostic model from the information acquired during the driver's operation, and sending second information other than the first information to the server device for use in diagnosis by the second diagnostic model. How the system works. [Claim 10] In Appendix 9, The terminal device does not send the first information to the server device. How the system works. [Claim 11] In Appendix 9 or 10, The terminal device trains the first diagnostic model using the first information. How the system works. [Claim 12] In any of the appendices 9 to 11, The server device trains the second diagnostic model using the second information. How the system works. [Claim 13] In any of the appendices 9 to 12, The first information is information relating to the driver's driving operations or driving conditions, The information in the second above is information regarding the driver's driving tendencies. How the system works. [Claim 14] In any of the appendices 9 to 13, The terminal device determines the first and second pieces of information from the information acquired during the driver's operation, in accordance with a predetermined operation performed by the driver. How the system works. [Claim 15] In appendices 9-14, The terminal device determines the first and second pieces of information from among the information acquired by the driver during operation, in accordance with the driver's response to the diagnostic results of the first diagnostic model. How the system works. [Claim 16] In claim 15, The terminal device acquires the response based on the driver's captured image or voice. How the system works. [Explanation of Symbols]

[0081] 1. Information Processing System 10 Server devices 11 Network 12 vehicles 13 Terminal devices 100 Server Agents 101, 131, 141 Communications Department 102, 132, 142 storage section 103, 133, 143 Control Unit 108 Server Diagnostic Models 109 Server Information 130 In-vehicle agents 134, 144 Positioning Unit 135, 145 Input section 136, 146 Output section 138 In-vehicle diagnostic models 139 Automotive information

Claims

1. A system comprising a terminal device used by the vehicle driver and a server device that communicates with the terminal device, The terminal device has a detection unit that acquires operational characteristics or information indicating the driver's driving state, and a first diagnostic model for notifying the driver of a diagnostic result corresponding to the acquired information within a first time. The server device has a second diagnostic model for acquiring information indicating the driver's driving tendencies from the terminal device and notifying the driver of a diagnostic result corresponding to the acquired information within a second time period longer than the first time period. The terminal device stores information acquired during the driver's operation, including operational feature quantities or information indicating the state of operation used for diagnosis by the first diagnostic model, and sends other information indicating driving tendencies to the server device for use in diagnosis by the second diagnostic model. The server device then acquires the diagnostic results output by the second diagnostic model and outputs the diagnostic results. system.

2. A system comprising a terminal device used by the vehicle driver and a server device that communicates with the terminal device, The terminal device has a first diagnostic model for notifying the driver of the diagnostic results corresponding to the driver's driving in a first time, The server device has a second diagnostic model for notifying the driver of the diagnostic results corresponding to the driver's driving in a second time that is longer than the first time. The terminal device holds first information used for diagnosis by the first diagnostic model from the information acquired during the driver's operation, and sends second information other than the first information to the server device for use in diagnosis by the second diagnostic model. system.

3. In claim 2, The terminal device does not send the first information to the server device. system.

4. In claim 2, The terminal device trains the first diagnostic model using the first information. system.

5. In claim 2, The server device trains the second diagnostic model using the second information. system.

6. In claim 2, The first piece of information is information relating to the driver's driving operations or driving conditions, The second piece of information described above is information relating to the driver's driving tendencies. system.

7. In claim 2, The terminal device determines the first and second pieces of information from the information acquired during the driver's operation, in accordance with a predetermined operation performed by the driver. system.

8. In claim 2, The terminal device determines the first and second pieces of information from among the information acquired by the driver during operation, in accordance with the driver's response to the diagnostic results of the first diagnostic model. system.

9. In claim 8, The terminal device acquires the response based on the driver's captured image or voice. system.

10. A method for operating a system having a terminal device used by a vehicle driver and a server device that communicates with the terminal device, The terminal device has a first diagnostic model for notifying the driver of the diagnostic results corresponding to the driver's driving in a first time, The server device has a second diagnostic model for notifying the driver of the diagnostic results corresponding to the driver's driving in a second time that is longer than the first time, The terminal device includes holding first information used for diagnosis by the first diagnostic model from the information acquired during the driver's operation, and sending second information other than the first information to the server device for use in diagnosis by the second diagnostic model. How the system works.

11. In claim 10, The terminal device does not send the first information to the server device. How the system works.

12. In claim 10, The terminal device trains the first diagnostic model using the first information. How the system works.

13. In claim 10, The server device trains the second diagnostic model using the second information. How the system works.

14. In claim 10, The first piece of information is information relating to the driver's driving operations or driving conditions, The second piece of information described above is information relating to the driver's driving tendencies. How the system works.

15. In claim 10, The terminal device determines the first and second pieces of information from the information acquired during the driver's operation, in accordance with a predetermined operation performed by the driver. How the system works.

16. In claim 10, The terminal device determines the first and second pieces of information from among the information acquired by the driver during operation, in accordance with the driver's response to the diagnostic results of the first diagnostic model. How the system works.

Images