System and method of operation of a system

Abstract

This disclosure provides a system and a method for improving driver convenience by notifying the driver of diagnostic results related to vehicle driving. The system includes a terminal device for use by the driver of the vehicle and a server device communicating with the terminal device. The terminal device has a first diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving at a first time. The server device has a second diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving at a second time, which is longer than the first time. The terminal device retains first information from information acquired during the driver's driving that is used in the diagnosis implemented by the first diagnostic model, and sends second information other than the first information to the server device for use in the diagnosis implemented by the second diagnostic model.

Description

Technical Field

[0001] This disclosure relates to a system and a method of operating the system. Background Technology

[0002] A technique for diagnosing the state of a moving body, including a vehicle, is known. For example, Patent Documents 1 to 3 disclose a system that uses a cloud server to process information obtained from detecting the movements of vehicles and the like, and performs diagnoses on the vehicles and the like.

[0003] Prior art literature Patent documents Patent Document 1: Japanese Patent Application Publication No. 2017-13742 Patent Document 2: Japanese Patent Application Publication No. 2019-131187 Patent Document 3: Japanese Patent Application Publication No. 2021-196678 Summary of the Invention The problem that the invention aims to solve There is room to improve convenience for drivers by using AI (Artificial Intelligence) and other technologies to notify them of various diagnostic results related to vehicle driving.

[0004] The following describes a system that improves driver convenience by informing the driver of diagnostic results related to vehicle driving.

[0005] Methods for solving problems The system disclosed herein includes a terminal device and a server device. The terminal device is used by a driver of a vehicle, and the server device communicates with the terminal device. The terminal device has a first diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving at a first time. The server device has a second diagnostic model for notifying the driver of diagnostic results corresponding to the driver's driving at a second time, which is longer than the first time. The terminal device retains first information from information acquired during the driver's driving that is used in a diagnosis implemented by the first diagnostic model, and sends second information other than the first information to the server device for use in a diagnosis implemented by the second diagnostic model.

[0006] In another embodiment of the system disclosed herein, the system includes a terminal device and a server device. The terminal device is used by a driver of a vehicle, and the server device communicates with the terminal device. The terminal device has a first diagnostic model for acquiring information representing the driver's operational characteristics or driving state via a detection unit and notifying the driver of a diagnostic result corresponding to the acquired information at a first time. The server device has a second diagnostic model for acquiring information representing the driver's driving tendencies from the terminal device and notifying the driver of a diagnostic result corresponding to the acquired information at a second time, longer than the first time. The system operation includes: the terminal device retaining information representing the operational characteristics or driving state used in a diagnosis implemented by the first diagnostic model from information acquired during the driver's driving, and sending information representing other driving tendencies to the server device for use in a diagnosis implemented by the second diagnostic model; the server device acquiring and outputting a diagnostic result output by the second diagnostic model.

[0007] Invention Effects According to the system and other components disclosed herein, driver convenience can be improved by informing the driver of diagnostic results related to vehicle driving. Attached Figure Description

[0008] Figure 1 A diagram illustrating the structure of an information processing system.

[0009] Figure 2 A sequence diagram illustrating the working process of an information processing system.

[0010] Figure 3 A flowchart illustrating the operation of an on-board device.

[0011] Figure 4 This is a sequence diagram illustrating the working process of the information processing system in the example of the change.

[0012] Figure 5 This is a flowchart illustrating the operation of the on-board device in the example of the change. Detailed Implementation

[0013] The implementation method will be described below.

[0014] Figure 1This diagram illustrates a structural example of an information processing system in one embodiment. The information processing system 1 includes one or more server devices 10, vehicle-mounted devices 13, and user terminals 14 connected together via a network 11 in a manner capable of mutual information communication. The server device 10 is, for example, a server computer belonging to a cloud computing system or other computing system, functioning as a server with various installed functions. The vehicle-mounted device 13 is, for example, a navigation system with communication and information processing functions, and is mounted on a vehicle 12. The vehicle 12 is a passenger car, commercial vehicle, etc., and is partially or entirely driven by a driver. The vehicle 12 is, for example, a gasoline car, a battery electric vehicle (BEV), a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), a fuel cell electric vehicle (FCEV), etc. The user terminal 14 is an information processing terminal used by the driver of the vehicle 12, such as a smartphone, tablet computer, or PC (personal computer). Network 11 is, for example, the Internet, but can also be a self-organizing network, a LAN, a MAN (Metropolitan Area Network), or other networks or any combination thereof.

[0015] In this embodiment, the information processing system 1 assists in diagnosing the vehicle 12 and notifying the driver of the diagnostic results. The information processing system 1 includes an in-vehicle device 13 or a user terminal 14, which serves as a "terminal device" for use by the driver of the vehicle 12, and a server device 10 that communicates with the in-vehicle device 13 and the user terminal 14. The in-vehicle device 13 has a first diagnostic model (hereinafter referred to as the in-vehicle diagnostic model) 138 for notifying the driver of diagnostic results corresponding to the driver's driving at a first time. Furthermore, the server device 10 has a second diagnostic model (hereinafter referred to as the server diagnostic model) 108 for notifying the driver of diagnostic results corresponding to the driver's driving at a second time, which is longer than the first time. The in-vehicle device 13 retains information (hereinafter referred to as object information) obtained during the driver's driving, specifically information used in the diagnosis implemented by the in-vehicle diagnostic model 138 (hereinafter referred to as in-vehicle information) 139, and sends information other than the in-vehicle information 139 to the server device 10 for use in the diagnosis implemented by the server diagnostic model 108. Therefore, the vehicle information 139, excluding information sent by the vehicle device 13 from the vehicle device 13 to the server device 10, is stored in the server device 10 as server information 109. Furthermore, the vehicle device 13 can use the vehicle information 139 to perform diagnosis using the vehicle diagnostic model 138, and in addition to notifying the driver of the diagnostic results, it can also use the vehicle information 139 to train or strengthen the vehicle diagnostic model 138. On the other hand, the server device 10 can use the server information 109 to perform diagnosis using the server diagnostic model 108, and in addition to notifying the driver of the diagnostic results through the user terminal 14, it can also use the server information 109 to train or strengthen the server diagnostic model 108.

[0016] When the server diagnostic model 108 is used to derive a diagnostic result corresponding to the driver's driving and to notify the driver from the server device 10, information obtained based on driving is sent from the vehicle device 13 to the server device 10. During the transmission of the diagnostic result from the server device 10 to the vehicle device 13, there is a possibility that the time before the driver receives the diagnostic result notification may become longer due to communication delays. In this respect, in this embodiment, the vehicle device 13 can obtain the diagnostic result requiring an earlier time, such as approximately real-time notification, through the vehicle diagnostic model 138, and notify the driver. On the other hand, the server device 10 can obtain the diagnostic result that does not cause any hindrance even if a certain amount of time is required to reach the notification, through the server diagnostic model 108, and convey it to the driver. Therefore, compared with the case of communicating with the server device 10, the vehicle device 13 can obtain the notification required by the driver in a shorter time. That is, the convenience of the driver can be improved when the diagnostic result related to the driving of the vehicle 12 is notified to the driver.

[0017] Next, the structure of the server device 10 will be described.

[0018] Server device 10 includes a communication unit 101, a storage unit 102, and a control unit 103. Server device 10 can be a single computer, or it can be composed of two or more computers connected in a manner capable of information communication and working together. When server device 10 is composed of two or more computers... Figure 1 The structure shown is appropriately configured in more than two computers.

[0019] The communication unit 101 includes one or more communication interfaces. These communication interfaces may be, for example, LAN interfaces. The communication unit 101 receives information used in the operation of the control unit 103 and also transmits information obtained through the operation of the control unit 103. The server device 10 is connected to the network 11 via the communication unit 101 and communicates with the vehicle-mounted device 13 and the user terminal 14 via the network 11.

[0020] Storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or combinations of at least two of these types, functioning as main storage devices, auxiliary storage devices, or cache memories. Semiconductor memories are, for example, RAM (Random Access Memory) or ROM (Read Only Memory). RAM is, for example, SRAM (Static RAM) or DRAM (Dynamic RAM). ROM is, for example, EEPROM (Electrically Erasable Programmable ROM). Storage unit 102 stores information used in the operation of control unit 103 and information obtained through the operation of control unit 103.

[0021] The control unit 103 includes one or more processors, one or more dedicated circuits, or combinations thereof. The processor may be, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) for specific processing. Dedicated circuits may include, for example, a FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 103 controls various parts of the server device 10 and performs information processing related to the operation of the server device 10.

[0022] The functions of the server device 10 are implemented by a control program executed by a processor included in the control unit 103. The control program is a program for enabling the computer to perform functions corresponding to the steps included in the operation of the server device 10. That is, the control program is a program for enabling the computer to function as the server device 10. Furthermore, some or all of the functions of the server device 10 can also be implemented by dedicated circuitry included in the control unit 103. Additionally, the control program can be stored in a non-temporary recording and storage medium readable by the server device 10, and can be read from the medium by the server device 10.

[0023] In this embodiment, the storage unit 102 stores the server diagnostic model 108, server information 109, and the server agent 100. The server diagnostic model 108 is an AI model obtained by learning from past diagnostic results corresponding to the server information 109 in order to diagnose the driver's driving tendencies corresponding to the server information 109. The server information 109 contains information for diagnosing driving tendencies generated by the driver, including information such as the number of dangerous driving incidents and the vehicle's wear and tear. Dangerous driving is diagnosed, for example, in the vehicle-mounted device 13, as described later. Furthermore, information indicating the wear and tear of the vehicle 12 is also obtained in the vehicle-mounted device 13. This information includes information such as remaining fuel level, brake pad deterioration level, and onboard battery deterioration level. The server device 10 obtains the server information 109 containing diagnostic history from the vehicle-mounted device 13. Server agent 100 is a module for generating a conversational AI that conveys the diagnostic results obtained by server diagnostic model 108 to the driver, and has a knowledge base related to natural language processing capabilities, diagnostic results, and driver preferences. The server diagnostic model 108, executed by control unit 103, diagnoses a tendency towards dangerous driving, for example, if the number of dangerous driving incidents exceeds a certain level. Furthermore, the server diagnostic model 108 diagnoses a tendency towards vehicle wear and tear, for example, if the vehicle's wear and tear exceeds a certain level. The diagnostic criteria for the number of dangerous driving incidents and the vehicle's wear and tear can be appropriately set or generated in the server diagnostic model 108 during the learning process. Moreover, the server agent 100, executed by control unit 103, generates a notification to the driver corresponding to the diagnostic results.

[0024] Next, an example of the structure of the vehicle-mounted device 13 will be described.

[0025] The vehicle-mounted device 13 includes a communication unit 131, a storage unit 132, a control unit 133, a positioning unit 134, an input unit 135, an output unit 136, and a detection unit 137. These components can be configured as a single control device, or as two or more control devices, or as a control device and other devices such as a communication unit. Control devices include, for example, ECUs (Electronic Control Units). Communication units include, for example, DCMs (Data Communication Modules). Each component is connected to each other or to the equipment of the vehicle 12 via an in-vehicle network according to standards such as CAN (Controller Area Network) in a manner capable of information communication. Furthermore, the vehicle-mounted device 13 may also be configured to partially include equipment equivalent to the user terminal 14.

[0026] The communication unit 131 includes a communication module corresponding to wired or wireless LAN standards, and modules corresponding to mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The vehicle-mounted device 13 connects to the network 11 via the communication unit 131 and through a nearby router or mobile communication base station, and communicates with other devices via the network 11.

[0027] Storage unit 132 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. Semiconductor memories are, for example, RAM or ROM. RAM is, for example, SRAM or DRAM. ROM is, for example, EEPROM. Storage unit 132 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. Storage unit 132 stores information used in the operation of control unit 133 and information obtained through the operation of control unit 133.

[0028] The control unit 133 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU, or a dedicated processor such as a GPU for specific processing. The dedicated circuit is, for example, an FPGA or an ASIC. The control unit 133 controls various parts of the vehicle-mounted device 13 and performs information processing related to the operation of the vehicle-mounted device 13.

[0029] The functions of the control unit 133 are implemented by a processor included in the control unit 133 executing control and processing programs. These control and processing programs are programs used to enable the computer to perform functions corresponding to the steps included in the operation of the control unit 133. In other words, the control and processing programs are programs used to enable the computer to function as the control unit 133. Furthermore, some or all of the functions of the control unit 133 can also be implemented using dedicated circuitry included in the control unit 133.

[0030] The positioning unit 134 includes one or more GNSS (Global Navigation Satellite System) receivers. GNSS receivers may include, for example, GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System), BeiDou, GLONASS (Global Navigation Satellite System), and Galileo. The positioning unit 134 transmits the positioning results to the control unit 133, and the control unit 133 uses the control unit 133 to obtain the position information of the on-board device 13, i.e., the vehicle 12.

[0031] The input unit 135 includes one or more input interfaces. Input interfaces may include, for example, a microphone for accepting voice input, physical buttons, electrostatic capacitive buttons, indicator devices, or a touchscreen integrated with a display. The input interface includes an interface with a camera installed on the vehicle 12 to capture images of the interior or exterior of the vehicle 12. The camera may be built into the vehicle-mounted device 13 or be a separate unit. The input unit 135 accepts input from users such as the driver for information used in the operation of the control unit 133, including voice input, or images captured by the camera of the driver, and transmits the received information to the control unit 133.

[0032] The output unit 136 includes one or more output interfaces. These output interfaces may be, for example, a speaker for outputting voice, or a display for outputting images. The display may be, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 136 outputs information obtained through the operation of the control unit 133.

[0033] The detection unit 137 includes sensor-type components that detect various events occurring in the vehicle 12, or interfaces with sensor-type components. These sensor-type components include, for example, those that detect the vehicle 12's speed, longitudinal acceleration, lateral acceleration, deceleration, accelerator operation amount, brake operation amount, steering angle, turn signal illumination time, fuel consumption per unit time, energy-saving mode selection status, odometer reading, safety equipment operation information, remaining engine oil level, brake pad wear, and battery degradation. Furthermore, sensor-type components include millimeter-wave and infrared radar that detects objects around the vehicle 12. The detection unit 137 transmits vehicle information indicating various states of the vehicle 12 detected by the sensor-type components to the control unit 133.

[0034] The control unit 133 receives and transmits various information to the communication unit 131, storage unit 132, positioning unit 134, input unit 135, output unit 136, and detection unit 137, and controls these components while simultaneously controlling the operation of the vehicle 12. During vehicle 12 operation, the control unit 133 provides navigation functionality by providing the driver with route information and other necessary information via the output unit 136, or controls partial automatic driving of the vehicle 12.

[0035] In this embodiment, the storage unit 132 stores the vehicle diagnostic model 138, vehicle information 139, and vehicle intelligent agent 130. The vehicle diagnostic model 138 is an AI model obtained by learning from past diagnostic results corresponding to the vehicle information 139 in order to diagnose the characteristics of the driver's operations or the driver's state corresponding to the vehicle information 139. The vehicle information 139 represents information indicating the operational characteristic quantities or driving state generated by the driver. The operational characteristic quantities include information indicating control quantities in driving operations or the motion state of the vehicle 12. Control quantities in driving operations include control quantities of the brakes, accelerator, steering device, turn signals, etc. Control quantities include the change in control quantities per unit time. Furthermore, information indicating the motion state of the vehicle 12 includes the vehicle 12's speed, direction of travel, lateral acceleration, and distance to other vehicles detected by radar or external images of the vehicle 12. Information indicating the driver's driving state includes images of the driver, voice, etc. The vehicle-mounted intelligent agent 130 is a module for generating a conversational AI that conveys the diagnostic results obtained by the vehicle-mounted diagnostic model 138 to the driver, and has a knowledge base related to natural language processing functions, diagnostic results, and driver preferences. The vehicle-mounted diagnostic model 138, executed by the control unit 133, diagnoses dangerous driving, for example, when the control quantity of the driving operation or the motion state of the vehicle 12 represents a value corresponding to dangerous driving. Furthermore, the vehicle-mounted diagnostic model 138 diagnoses fatigue accumulation, for example, when images or voice recordings indicating the driver's state while driving indicate drowsiness or fatigue. The criteria for diagnosing dangerous driving and fatigue accumulation can be appropriately set or generated in the vehicle-mounted diagnostic model 138 during the learning process. Moreover, the vehicle-mounted intelligent agent 130 generates a notification to the driver corresponding to the diagnostic results.

[0036] Next, the structure of user terminal 14 will be described.

[0037] User terminal 14 includes a communication unit 141, a storage unit 142, a control unit 143, a positioning unit 144, an input unit 145, and an output unit 146.

[0038] The communication unit 141 includes a communication module corresponding to wired or wireless LAN standards, and a module corresponding to mobile communication standards such as LTE, 4G, or 5G. The user terminal 14 connects to the network 11 through the communication unit 141 and via a nearby router device or a mobile communication base station, and performs information communication with other devices through the network 11.

[0039] Storage unit 142 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. Semiconductor memories are, for example, RAM or ROM. RAM is, for example, SRAM or DRAM. ROM is, for example, EEPROM. Storage unit 142 functions as, for example, a main storage device, an auxiliary storage device, or a high-speed cache memory. Storage unit 142 stores information used in the operation of control unit 143 and information obtained through the operation of control unit 143.

[0040] The control unit 143 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU, or a dedicated processor such as a GPU, which is dedicated to specific processing. The dedicated circuit is, for example, an FPGA or an ASIC. The control unit 143 controls various parts of the user terminal 14 and performs information processing related to the operation of the user terminal 14.

[0041] The positioning unit 144 includes one or more GNSS receivers. GNSS receivers may include, for example, any of GPS, QZSS, BeiDou, GLONASS, and Galileo. The positioning unit 144 transmits the positioning results to the control unit 143, and uses the control unit 143 to obtain the location information of the user terminal 14.

[0042] The input unit 145 includes one or more input interfaces. Examples of input interfaces include microphones for accepting voice input, physical buttons, electrostatic / capacitive buttons, indicator devices, touchscreens integrated with the display, and cameras for capturing images. The input unit 145 accepts input operations for information used in the operation of the control unit 143 and sends the input information to the control unit 143.

[0043] The output unit 146 includes one or more output interfaces. These output interfaces may be, for example, speakers or displays. The displays may be, for example, LCD or OLED displays. The output unit 146 outputs information obtained through the operation of the control unit 143.

[0044] The functions of the control unit 143 are implemented by a processor included in the control unit 143 executing control and processing programs. These control and processing programs are programs used to enable the computer to perform functions corresponding to the steps included in the operation of the control unit 143. In other words, the control and processing programs are programs used to enable the computer to function as the control unit 143. Furthermore, some or all of the functions of the control unit 143 can also be implemented using dedicated circuitry included in the control unit 143.

[0045] Next, use Figures 2 to 5 The operation of information processing system 1 will be explained.

[0046] Figure 2 This is a sequence diagram used to explain the working steps of the information processing system 1 in this embodiment. Figure 2 This describes the steps related to the joint operation of the server device 10, the vehicle-mounted device 13, and the user terminal 14. Figure 2 The steps related to various information processing of the server device 10, vehicle-mounted device 13, and user terminal 14 are executed by control units 103, 133, and 143, respectively. Furthermore, the steps related to the transmission and reception of various information of the server device 10, vehicle-mounted device 13, and user terminal 14 are executed by the respective control units 103, 133, and 143 transmitting and receiving information to each other via communication units 101, 131, and 141. In the server device 10, vehicle-mounted device 13, and user terminal 14, the information transmitted, received, and processed by the control units 103, 133, and 143 is appropriately stored in storage units 102, 132, and 142, respectively. Furthermore, in the vehicle-mounted device 13 and user terminal 14, control units 133 and 143 accept various information inputs via input units 135 and 145, respectively, and output various information via output units 136 and 146, respectively.

[0047] Figure 2 Steps S201 to S209 of the vehicle-mounted device 13 are executed, for example, at any interval of tens of milliseconds to several seconds when the vehicle 12 is in motion.

[0048] In S201, the vehicle-mounted device 13 acquires object information. The control unit 133 acquires object information, including control quantities of the brakes, accelerator, steering wheel, and turn signals, the vehicle's speed, direction of travel, lateral acceleration, distance to other vehicles, images captured by the driver, and voice messages, through various sensors installed on the vehicle 12 and input interfaces. A timestamp may also be appended to each piece of information.

[0049] In S202, the vehicle-mounted device 13 classifies the object information. The control unit 133 classifies the object information into vehicle-mounted information 139 and server-mounted information 109, respectively. For example, the control unit 133 classifies the object information by attaching tags indicating whether it is for vehicle-mounted or server-mounted information according to each type of object information. In the storage unit 132, for example, reference information is stored that is pre-classified into vehicle-mounted information 139 and server-mounted information 109 according to each type of object information. The control unit 133 refers to the reference information and applies tags corresponding to the type of object information being acquired.

[0050] In S203, the vehicle-mounted device 13 sends the server information 109 to the server device 10. Alternatively, after sending the server information 109 to the server device 10, the vehicle-mounted device 13 can delete the server information 109 from the storage unit 102 to release storage capacity.

[0051] In S204, the vehicle-mounted device 13 stores the vehicle-mounted information 139.

[0052] In S205, the vehicle-mounted device 13 performs a diagnosis using the vehicle-mounted information 139. In the vehicle-mounted device 13, the vehicle-mounted diagnostic model 138 is executed by the control unit 133, and a diagnosis using the classified vehicle-mounted information 139 is performed.

[0053] In S206, the vehicle-mounted device 13 generates a notification based on the diagnostic results. When the vehicle-mounted intelligent agent 130 diagnoses, for example, emergency braking, emergency acceleration, or emergency steering based on brake control quantities or acceleration, the control unit 133 generates notifications with warnings of danger such as "Emergency braking danger!", "Emergency start danger!", or "Emergency steering caution!". Furthermore, when the driver's condition is diagnosed as including drowsiness, fatigue, or low attention, notifications generate notifications with warnings to promote alertness and attention such as "Wake up!" or "Look ahead!". Alternatively, the notification may be replaced by flashing warning lights or audible warnings, in addition to or in lieu of warnings.

[0054] In S207, the vehicle-mounted device 13 outputs a notification to the driver. The notification wording is displayed on the display of the output unit 136 or output as voice through a speaker. In addition, warning lights may flash or warning sounds may be output. Such notification output can help suppress dangerous driving or drowsy driving by the driver.

[0055] In step S208, the on-board unit 13 evaluates the driver's reaction to receiving the notification. Detailed steps of step S208 are provided below. Figure 3 As shown in the image.

[0056] Figure 3 A flowchart illustrating an example of the steps for evaluating the driver's reaction in the vehicle-mounted device 13. Figure 3 Each step is an information processing step executed by the control unit 133.

[0057] In S31, the control unit 133 specifies the vehicle information 139. The control unit 133 reads and specifies the vehicle information 139 on which the diagnosis is based from the storage unit 102.

[0058] In step S32, the control unit 133 acquires response information. This response information indicates the driver's reaction to the notification and includes captured images and voice messages from the driver. The control unit 133 acquires the captured images and voice messages from the input interface.

[0059] In S33, the control unit 133 determines whether the reaction is positive or negative. For example, the control unit 133 determines the positive or negative reaction based on the driver's facial expression in the captured image. For example, the control unit 133 performs arbitrary image processing on the captured image, extracts an image of the driver's face, detects markings in the facial image, and determines whether the driver's expression belongs to a pattern indicating displeasure based on the shape of the markings. Alternatively, the control unit 133 analyzes the driver's speech based on voice information to determine whether it contains statements indicating displeasure. Information such as patterns and statements indicating displeasure is appropriately preset. Furthermore, the control unit 133 determines a negative reaction if the driver's expression or speech indicates displeasure, and a positive reaction otherwise. If the control unit 133 determines the response to be negative (yes in step S34), it proceeds to step S35; if the response to be positive (i.e., not negative, as determined in step S34), it bypasses step S35 and ends the process. Figure 3 The steps.

[0060] In S35, the control unit 133 classifies the vehicle information 139 specified in step S31 as server information 109.

[0061] If you return to Figure 2 In step S209, the vehicle-mounted device 13 updates the classification criteria for the object information. For example, the control unit 133 of the vehicle-mounted device 13 updates the classification criteria for the object information based on... Figure 3 The classification results in step S35 are used to update the reference information that serves as the basis for classifying object information into vehicle information 139 or server information 109. The control unit 133 updates the basis, for example, by changing the label for each category of object information. In this way, object information classified from vehicle information 139 to server information 109 is processed as server information 109 in subsequent processing.

[0062] Figure 2 Steps S211 to S213 of the server device 10 are executed, for example, at any interval of every few days to every few weeks.

[0063] In S211, the server device 10 performs the diagnosis implemented by the server information 109. In the server device 10, the server diagnostic model 108 is executed by the control unit 103, and the diagnosis performed using the server information 109 is implemented.

[0064] In S212, the server device 10 generates a notification based on the diagnostic results. If, for example, the server agent 100 diagnoses a tendency towards dangerous driving, the control unit 103 generates notifications such as "Control your speed," "Apply the brakes as early as possible," "Turn slowly around curves," "Use the turn signals as early as possible," and "Maintain sufficient distance from other vehicles." Alternatively, if the control unit 103 diagnoses a tendency towards vehicle wear and tear, it generates notifications such as "It's time for vehicle maintenance" and "Check the brakes."

[0065] In S213, the vehicle-mounted device 13 sends notification information to the user terminal 14. The notification information is information used to output a notification.

[0066] In S214, the user terminal 14 outputs a notification to the driver. The wording of the notification is displayed on the display of the output unit 146, or output as voice through a speaker. Such notification output can help to prompt the driver to become aware of dangerous driving tendencies, or prompt maintenance and inspection of the vehicle 12. Furthermore, when the amount of information that can be notified to the driver while driving in the vehicle device 13 is limited, the driver can review his driving tendencies afterward.

[0067] When the server device 10 sends server information 109 from the vehicle-mounted device 13 in step S203, it updates the server information 109 in step S215 as needed. For example, the control unit 103 updates information indicating the wear level of the vehicle 12, the remaining oil level, the degree of brake pad deterioration, and the degree of on-board battery deterioration to the latest information and stores it in the storage unit 102. In this way, the information indicating the wear level of the vehicle 12 is updated in a timely manner for use in subsequent diagnoses performed by the server diagnostic model 108.

[0068] Through these operational steps, the vehicle information 139 and the updated reference information are maintained in the vehicle-mounted device 13. Therefore, the vehicle-mounted device 13 can use the accumulated vehicle information 139 and reference information as teaching data at any time to train the vehicle diagnostic model 138. By employing this method, the vehicle diagnostic model 138 can be adjusted to output diagnostic results that are more suitable for the driver. Through the aforementioned steps, the vehicle-mounted device 13 accumulates the vehicle-mounted information 139 and the updated reference information. Therefore, the vehicle-mounted device 13 can use the accumulated vehicle-mounted information 139 and reference information as teaching data at any time to train the vehicle-mounted diagnostic model 138. By employing this method, the vehicle-mounted diagnostic model 138 can be adjusted to output diagnostic results that are more suitable for the driver. Furthermore, since the server information 109 sent from multiple vehicles 12 is stored in the server device 10, such server information 109 can be used as teaching data at any time to train the server diagnostic model 108. Figure 4 This is a sequence diagram illustrating the working steps of information processing system 1 in the example of the change. Figure 4 The following are examples of working steps that replace the user terminal 14 to perform operations for the vehicle-mounted device 13, and in replacing... Figure 2 In terms of executing steps S213' and S214' through the vehicle-mounted device 13, and in this respect, with Figure 2 The difference is that, in step S213', the server device 10 sends the notification information obtained based on the diagnostic results to the vehicle-mounted device 13. Thus, in step S214', the vehicle-mounted device 13 uses the same method as in... Figure 2 In step S214, the user terminal 14 implements the same method to output the notification.

[0069] By execution Figure 2 or Figure 4 The steps shown illustrate how the onboard device 13 can notify the driver of approximately real-time diagnostic results based on the driver's driving actions or driving state. On the other hand, the server device 10 generates diagnostic results corresponding to the driver's driving tendencies acquired over time, for example, spanning several days to several weeks. Furthermore, the notification is output to the driver via the onboard device 13 or the user terminal 14. In a manner similar to... Figure 2 In the case where notifications are output by user terminal 14 as in the example, the driver can receive notifications related to their driving tendencies even when they are away from vehicle 12.

[0070] In addition, by utilizing Figure 3 The steps shown are used to evaluate the driver's reaction to the notification, so that if the notification output by the vehicle diagnostic model 138 in roughly real-time annoys the driver, the annoying notification can be reduced by classifying the object information on which such notification is based in a way that will not be used by the vehicle diagnostic model 138 in the future and sending it to the server device 10 as server information 109.

[0071] Figure 5 This is a flowchart illustrating the steps involved in evaluating a driver's response to a notification in a further modified example. Figure 5 The example of the steps is that steps S34' and S34'' are inserted after step S34, which is similar to... Figure 3 The steps are different.

[0072] If the driver's response in step S34 is a negative response (yes), the control unit 133 of the vehicle device 13 will add up the count of negative responses that have been established according to each object information in step S34' and proceed to step S34''. If the response is not a negative response (no), it will bypass step S34' and proceed to step S34''.

[0073] In step S34'', if the count value is any reference value, such as any value of 3 to 10 times or more (yes), the control unit 133 classifies the vehicle information 139 specified in step S31 into server information 109; if the count value is less than any reference value (no), it bypasses step S35 and ends the process.

[0074] According to the modified example, it is possible to reduce the possibility that the vehicle information 139 may be unnecessarily classified as server information 109 in the event that the driver accidentally gives a negative response.

[0075] Furthermore, according to a further modification, instead of evaluating the driver's reaction, the vehicle-mounted device 13 can also classify the vehicle-mounted information 139 to the server-mounted information 109 based on the driver's explicit input, such as a touch operation on the touch panel to stop the notification of diagnostic results. The operation input can, for example, be conditional upon occurring at any time after the notification of diagnostic results, for example, within a few seconds to ten seconds. By adopting this method, the classification of object information according to the driver's intention can be performed more reliably.

[0076] According to this embodiment described above, the driver can obtain approximately real-time diagnostic results based on their driving operation or state according to their preferences, and obtain diagnostic results related to their driving tendencies after a certain period of time, allowing them to repeatedly review their tendencies. Therefore, driver convenience can be improved by notifying the driver of diagnostic results related to the driving of vehicle 12. Furthermore, by sending server information 109, which does not require approximately real-time diagnostics, to server device 10, the vehicle-mounted device 13 can save resources.

[0077] In the above embodiments, Figure 2The order of steps S211 to S213 performed by the server device 10 and the steps in the vehicle device 13 is not limited to... Figure 2 Examples. Furthermore, Figure 4 The order in which steps S211 to S213' implemented by the server device 10 and steps S201 to S209 in the vehicle device 13 are not limited to... Figure 4 Examples.

[0078] In the above embodiments, the prescribed processing and control program for the operation of vehicle 12 and user terminal 14 may also be stored in the storage unit 102 of server device 10 or the storage unit of other server devices, and downloaded to each device via network 11, or stored in a non-temporary recording and storage medium readable by each device, and read from the medium by each device.

[0079] While the embodiments have been described above based on the accompanying drawings, it should be noted that various modifications and alterations can be easily made based on this disclosure if one is skilled in the art. Therefore, it should be understood that such modifications and alterations are included within the scope of this disclosure. For example, the functions included in each unit or step can be reconfigured in a logically consistent manner, and multiple units or steps can be combined into one or divided.

[0080] The following are examples of some embodiments of this disclosure. However, please note that the embodiments of this disclosure are not limited to these contents.

[0081] [Postscript 1] A system comprising a terminal device and a server device, wherein the terminal device is used by a driver of a vehicle, and the server device communicates with the terminal device, wherein... The terminal device has a first diagnostic model, which is used to notify the driver of diagnostic results corresponding to the driver's driving behavior as soon as possible. The server device has a second diagnostic model, which is used to notify the driver of diagnostic results corresponding to the driver's driving at a second time that is longer than the first time. The terminal device retains first information from the information acquired while the driver is driving, which is used in the diagnosis implemented by the first diagnostic model, and sends second information other than the first information to the server device for use in the diagnosis implemented by the second diagnostic model.

[0082] [Postscript 2] The system as described in Appendix 1, wherein, The terminal device does not send the first information to the server device.

[0083] [Postscript 3] The system as described in Appendix 1 or 2, wherein, The terminal device uses the first information to train the first diagnostic model.

[0084] [Postscript 4] The system as described in any one of Appendices 1 to 3, wherein, The server device uses the second information to train the second diagnostic model.

[0085] [Postscript 5] The system as described in any one of Notes 1 to 4, wherein, The first information is information related to the driver's driving operations or driving state. The second information is information related to the driver's driving tendencies.

[0086] [Postscript 6] The system as described in any one of notes 1 to 5, wherein, The terminal device determines the first information and the second information based on the predetermined operation performed by the driver in the information obtained while the driver is driving.

[0087] [Postscript 7] The system as described in any one of Notes 1 to 6, wherein, The terminal device determines the first information and the second information based on the driver's reaction to the diagnostic results of the first diagnostic model, which is obtained from information acquired while the driver is driving.

[0088] [Postscript 8] The system as described in Appendix 7, wherein... The terminal device obtains the response through images or voice recorded by the driver.

[0089] [Postscript 9] A method for operating a system comprising a terminal device and a server device, wherein the terminal device is used by a driver of a vehicle, and the server device communicates with the terminal device, wherein... The terminal device has a first diagnostic model, which is used to acquire information representing the driver's operational characteristics or driving state through a detection unit, and to notify the driver of the diagnostic results corresponding to the acquired information in a timely manner. The server device has a second diagnostic model, which is used to obtain information representing the driver's driving tendency from the terminal device, and to notify the driver of the diagnostic results corresponding to the obtained information at a second time that is longer than the first time. The system's operating method includes: The terminal device retains information representing the operational characteristic quantity or the state of driving, which is used in the diagnosis implemented by the first diagnostic model, from the information obtained when the driver is driving. It also sends information representing other driving tendencies to the server device for use in the diagnosis implemented by the second diagnostic model. The server device then obtains the diagnostic result output by the second diagnostic model and outputs the diagnostic result.

[0090] [Postscript 10] The system's operating method as described in Appendix 9, wherein... The terminal device does not send the first information to the server device.

[0091] [Postscript 11] The system operation method as described in Appendix 9 or 10, wherein... The terminal device uses the first information to train the first diagnostic model.

[0092] [Postscript 12] The system's operating method as described in any of Appendix 9 to 11, wherein... The server device uses the second information to train the second diagnostic model.

[0093] [Postscript 13] The system's operating method as described in any of Appendix 9 to 12, wherein... The first information is information related to the driver's driving operations or driving state. The second information is information related to the driver's driving tendencies.

[0094] [Postscript 14] The system's operating method is as described in any of Appendix 9 to 13, wherein... The terminal device determines the first information and the second information based on the predetermined operation performed by the driver in the information obtained while the driver is driving.

[0095] [Postscript 15] The system's operation method, as described in Appendices 9 to 14, wherein... The terminal device determines the first information and the second information based on the driver's reaction to the diagnostic results of the first diagnostic model, which is obtained from information acquired while the driver is driving.

[0096] [Postscript 16] The system operation method as described in Appendix 15, wherein... The terminal device obtains the response through images or voice recorded by the driver.

[0097] Symbol Explanation 1…Information processing system; 10… server devices; 11…Network; 12… vehicles; 13…Terminal device; 100… server intelligent agent; 101, 131, 141… Ministry of Communications; Storage units 102, 132, 142… 103, 133, 143… Control Department; 108…Server diagnostic model; 109… server information; 130…In-vehicle intelligent agent; Positioning parts 134, 144… Input sections 135, 145… Output sections 136, 146… 138…On-board diagnostic model; 139…In-vehicle information.

Claims

1. A system comprising a terminal device and a server device, the terminal device being used by a driver of a vehicle, the server device communicating with the terminal device, wherein, The terminal device has a first diagnostic model, which is used to acquire information representing the driver's operational characteristics or driving state through a detection unit, and to notify the driver of the diagnostic results corresponding to the acquired information in a timely manner. The server device has a second diagnostic model, which is used to obtain information representing the driver's driving tendency from the terminal device, and to notify the driver of the diagnostic results corresponding to the obtained information at a second time that is longer than the first time. The terminal device retains information representing the operational characteristic quantity or the state of driving, which is used in the diagnosis implemented by the first diagnostic model, from the information obtained when the driver is driving. It also sends information representing other driving tendencies to the server device for use in the diagnosis implemented by the second diagnostic model. The server device then obtains the diagnostic result output by the second diagnostic model and outputs the diagnostic result.

2. A system comprising a terminal device and a server device, the terminal device being used by a driver of a vehicle, the server device communicating with the terminal device, wherein, The terminal device has a first diagnostic model, which is used to notify the driver of diagnostic results corresponding to the driver's driving behavior as soon as possible. The server device has a second diagnostic model, which is used to notify the driver of diagnostic results corresponding to the driver's driving at a second time that is longer than the first time. The terminal device retains first information from the information acquired while the driver is driving, which is used in the diagnosis implemented by the first diagnostic model, and sends second information other than the first information to the server device for use in the diagnosis implemented by the second diagnostic model.

3. The system as described in claim 2, wherein, The terminal device does not send the first information to the server device.

4. The system as described in claim 2, wherein, The terminal device uses the first information to train the first diagnostic model.

5. The system as described in claim 2, wherein, The server device uses the second information to train the second diagnostic model.

6. The system as claimed in claim 2, wherein, The first information is information related to the driver's driving operations or driving state. The second information is information related to the driver's driving tendencies.

7. The system as claimed in claim 2, wherein, The terminal device determines the first information and the second information based on the predetermined operation performed by the driver in the information obtained while the driver is driving.

8. The system as claimed in claim 2, wherein, The terminal device determines the first information and the second information based on the driver's reaction to the diagnostic results of the first diagnostic model, which is obtained from information acquired while the driver is driving.

9. The system of claim 8, wherein, The terminal device obtains the response through images or voice recorded by the driver.

10. A method of operating a system, the system comprising a terminal device and a server device, the terminal device being used by a driver of a vehicle, the server device communicating with the terminal device, wherein... The terminal device has a first diagnostic model, which is used to acquire information representing the driver's operational characteristics or driving state through a detection unit, and to notify the driver of the diagnostic results corresponding to the acquired information in a timely manner. The server device has a second diagnostic model, which is used to obtain information representing the driver's driving tendency from the terminal device and notify the driver of the diagnostic results corresponding to the obtained information at a second time, which is longer than the first time. The system's operating method includes: The terminal device retains information representing the operational characteristic quantity or the state of driving, which is used in the diagnosis implemented by the first diagnostic model, from the information obtained when the driver is driving. It also sends information representing other driving tendencies to the server device for use in the diagnosis implemented by the second diagnostic model. The server device then obtains the diagnostic result output by the second diagnostic model and outputs the diagnostic result.

11. The method of operating the system as described in claim 10, wherein, The terminal device does not send the first information to the server device.

12. The method of operating the system as described in claim 10, wherein, The terminal device uses the first information to train the first diagnostic model.

13. The method of operating the system as described in claim 10, wherein, The server device uses the second information to train the second diagnostic model.

14. The method of operating the system as described in claim 10, wherein, The first information is information related to the driver's driving operations or driving state. The second information is information related to the driver's driving tendencies.

15. The method of operating the system as described in claim 10, wherein, The terminal device determines the first information and the second information based on the predetermined operation performed by the driver in the information obtained while the driver is driving.

16. The method of operating the system as described in claim 10, wherein, The terminal device determines the first information and the second information based on the driver's reaction to the diagnostic results of the first diagnostic model, which is obtained from information acquired while the driver is driving.

Images