system

Abstract

The system according to this embodiment aims to detect dangerous situations and abnormalities during driving in real time and to respond appropriately. [Solution] The system according to the embodiment comprises a monitoring unit, a detection unit, a warning unit, a video recording and storage unit, an emergency notification unit, and an advice unit. The monitoring unit monitors the driving situation in real time. The detection unit detects dangerous situations or abnormalities based on the data collected by the monitoring unit. The warning unit issues a warning for dangerous situations or abnormalities detected by the detection unit. The video recording and storage unit automatically saves the recording when an accident occurs. The emergency notification unit makes an emergency notification based on the video saved by the video recording and storage unit. The advice unit analyzes past driving data and provides advice on driving skills.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, dangerous situations and abnormalities during driving are not sufficiently detected in real time and appropriately handled, leaving room for improvement.

[0005] The system according to the embodiment aims to detect dangerous situations and abnormalities during driving in real time and appropriately handle them.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a monitoring unit, a detection unit, a warning unit, a video recording and storage unit, an emergency notification unit, and an advice unit. The monitoring unit monitors the driving conditions in real time. The detection unit detects dangerous situations or abnormalities based on the data collected by the monitoring unit. The warning unit issues a warning for dangerous situations or abnormalities detected by the detection unit. The video recording and storage unit automatically saves recordings in the event of an accident. The emergency notification unit makes an emergency notification based on the video recorded and stored by the video recording and storage unit. The advice unit analyzes past driving data and provides advice on driving skills. [Effects of the Invention]

[0007] The system according to this embodiment can detect dangerous situations and abnormalities during driving in real time and respond appropriately. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F manages communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The AI ​​assistant for a drive recorder according to an embodiment of the present invention is a system that provides real-time warnings of dangerous situations and abnormalities while driving, automatic recording and saving in the event of an accident, emergency call functions, and advice on driving skills based on past driving data. The AI ​​assistant for the drive recorder monitors the driving situation in real time and detects dangerous situations and abnormalities. Next, it issues a warning for the detected dangerous situations and abnormalities. Furthermore, if an accident occurs, it automatically saves the recording and makes an emergency call. It also analyzes past driving data and provides advice on driving skills. For example, the AI ​​assistant for the drive recorder monitors the vehicle's speed, the movement of surrounding vehicles, and road conditions. In this process, the AI ​​analyzes this data and detects dangerous situations and abnormalities. For example, it can detect situations such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. Next, it issues a warning for the detected dangerous situations and abnormalities. For example, it warns the driver by voice or on-screen display. This allows the driver to respond quickly to dangerous situations. Furthermore, if an accident occurs, it automatically saves the recording and makes an emergency call. For example, if the impact sensor is activated, it automatically saves video footage before and after the accident. It also notifies emergency contacts of the accident through its emergency call function, enabling a quick response. Furthermore, it analyzes past driving data to provide driving skill advice. For instance, it analyzes the driver's driving patterns and habits to provide advice for safer driving, allowing the driver to improve their skills. In this way, the AI ​​assistant in the dashcam supports safe driving by providing real-time warnings of dangerous situations and anomalies while driving, automatic recording and saving of accident footage, emergency call functions, and driving skill advice based on past driving data.

[0029] The AI ​​assistant for the drive recorder according to this embodiment comprises a monitoring unit, a detection unit, a warning unit, a recording storage unit, an emergency call unit, and an advice unit. The monitoring unit monitors the driving situation in real time. The monitoring unit collects data such as vehicle speed, the movement of surrounding vehicles, and road conditions. The monitoring unit can analyze this data using AI to detect dangerous situations or abnormalities. The detection unit detects dangerous situations or abnormalities based on the data collected by the monitoring unit. The detection unit detects situations such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. The detection unit can analyze these situations using AI to detect dangerous situations or abnormalities. The warning unit issues a warning for dangerous situations or abnormalities detected by the detection unit. The warning unit warns the driver, for example, by voice or on-screen display. The warning unit can adjust the content and timing of the warning using AI. The recording storage unit automatically saves the recording in the event of an accident. The recording and storage unit automatically saves video footage before and after an accident, for example, when the impact sensor is activated. The recording and storage unit can adjust the recording storage conditions and storage period using AI. The emergency call unit makes an emergency call based on the video footage saved by the recording and storage unit. The emergency call unit notifies emergency contacts of the occurrence of an accident, for example. The emergency call unit can adjust the content and timing of the call using AI. The advice unit analyzes past driving data and provides advice on driving skills. The advice unit analyzes the driver's driving patterns and habits, for example, and provides advice for safe driving. The advice unit can adjust the content and timing of the advice using AI. As a result, the AI ​​assistant of the drive recorder according to this embodiment can support safe driving by providing real-time warnings of dangerous situations and abnormalities while driving, automatic recording and saving in the event of an accident, emergency call functions, and advice on driving skills based on past driving data.

[0030] The monitoring unit monitors driving conditions in real time. For example, it collects data such as vehicle speed, the movement of surrounding vehicles, and road conditions. Specifically, it uses the vehicle's speed sensor and GPS to determine the current speed and position, and cameras and radar sensors to detect the movement of surrounding vehicles. Regarding road conditions, it analyzes camera footage to recognize lane positions, traffic light status, and road signs. This data is analyzed in real time using AI to detect dangerous situations and anomalies. For example, the AI ​​uses image recognition technology to measure the distance to the vehicle ahead from camera footage and issues a warning if it is approaching too closely. It also analyzes speed sensor data to detect anomalies such as sudden braking or sharp steering maneuvers. Furthermore, it considers changes in road conditions and weather, enabling it to detect slippery surfaces and poor visibility. This allows the monitoring unit to monitor various driving conditions in real time and provide appropriate information to the driver.

[0031] The detection unit detects dangerous situations or anomalies based on data collected by the monitoring unit. For example, the detection unit can detect situations such as sudden braking, sudden steering maneuvers, or a rapid decrease in the distance to the vehicle in front. By using AI to analyze these situations, it can detect dangerous situations and anomalies. Specifically, the AI ​​learns past driving data using machine learning algorithms and identifies abnormal patterns. For example, in the case of sudden braking, it analyzes data on speed changes and steering maneuvers before and after the braking and detects anomalies by comparing them with normal driving patterns. Also, in the case of a rapid decrease in the distance to the vehicle in front, the AI ​​analyzes camera images and radar sensor data to assess the risk of collision. Furthermore, the detection unit can learn the driver's behavior patterns and habits and perform anomaly detection tailored to the individual driving style. As a result, the detection unit can quickly and accurately detect specific dangerous situations and anomalies while driving and issue appropriate warnings to the driver.

[0032] The warning unit issues warnings for dangerous situations or anomalies detected by the detection unit. The warning unit warns the driver, for example, through voice or screen display. The content and timing of warnings can be adjusted using AI. Specifically, the AI ​​selects the optimal warning method according to the type and urgency of the detected danger. For example, if the distance to the vehicle in front decreases rapidly, it will issue a voice warning of "Pay attention ahead" and simultaneously display a warning message on the dashboard screen. In the event of sudden braking, it can also use a vibration alert to draw the driver's attention. Furthermore, the warning unit can monitor the driver's response and adjust the intensity and frequency of warnings as needed. For example, if the driver does not take an appropriate action in response to a warning, it will increase the volume of the warning or issue repeated warnings to draw attention. In this way, the warning unit can provide the driver with effective warnings at the appropriate time and help them avoid dangerous situations.

[0033] The recording storage unit automatically saves recordings in the event of an accident. For example, if an impact sensor is activated, the recording storage unit automatically saves footage before and after an accident. AI can be used to adjust the recording storage conditions and storage period. Specifically, if the impact sensor exceeds a certain threshold, the recording storage unit immediately starts recording and saves footage for several seconds before and after the accident. The AI ​​can analyze the saved footage and automatically extract important scenes and parts that can serve as evidence. The AI ​​also sets an appropriate storage period according to the severity of the accident and legal requirements. For example, a short storage period is sufficient for minor accidents, but a long storage period is necessary for serious accidents. Furthermore, the recording storage unit can automatically upload the saved footage to a cloud server, ensuring data security and easy access. This allows the recording storage unit to reliably save important footage from the time of the accident, which can then be used for later analysis and evidence submission.

[0034] The emergency call unit makes emergency calls based on video footage saved by the video storage unit. For example, the emergency call unit notifies emergency contacts of the occurrence of an accident. AI can be used to adjust the content and timing of the call. Specifically, the AI ​​analyzes the severity and location information of the accident and selects the most appropriate emergency contacts. For example, in the event of a serious accident, the emergency call unit immediately notifies the police and ambulance services and provides detailed information and location information about the accident. It also automatically notifies the driver's emergency contacts to encourage a quick response. Furthermore, the AI ​​can automatically generate the content of the call, clearly communicating the situation of the accident and the necessary actions. For example, it can generate a specific call such as, "A collision has occurred with the vehicle in front. Our current location is XX. Please dispatch an ambulance." This allows the emergency call unit to make quick and appropriate calls when an accident occurs, preventing the escalation of damage.

[0035] The advice unit analyzes past driving data and provides advice on driving skills. For example, it analyzes the driver's driving patterns and habits and provides advice for safe driving. The content and timing of the advice can be adjusted using AI. Specifically, the AI ​​learns from past driving data and identifies the driver's characteristics and areas for improvement. For example, if sudden braking or sudden steering is frequent, the AI ​​analyzes the cause and provides appropriate advice. For example, it may give specific advice such as, "You brake suddenly a lot, so try to maintain a little more distance from the vehicle in front." It can also provide individualized advice tailored to the driver's driving style. For example, if the driver drives for long periods of time, it may also give health-related advice such as, "Take regular breaks." Furthermore, the advice unit can collect driver feedback and continuously improve the content and method of advice. In this way, the advice unit can support the improvement of drivers' driving skills and contribute to the realization of safe driving.

[0036] The monitoring unit can collect data on vehicle speed, the movement of surrounding vehicles, road conditions, and other data. For example, the monitoring unit can measure the vehicle speed with a sensor and collect the data. The monitoring unit can also monitor the movement of surrounding vehicles with a camera and collect the data. The monitoring unit can also monitor road conditions with sensors and cameras and collect the data. The monitoring unit can also collect other data such as weather information and road construction information. This allows for detailed monitoring of driving conditions by collecting data such as vehicle speed, the movement of surrounding vehicles, and road conditions. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input vehicle speed data into a generating AI and have the generating AI perform analysis of the speed data.

[0037] The detection unit can detect dangerous situations or abnormalities such as sudden braking or sudden steering maneuvers, or a rapid decrease in the distance to the vehicle in front. For example, the detection unit can detect sudden braking by measuring the speed at which the brake pedal is pressed using a sensor. The detection unit can also detect sudden steering maneuvers by measuring the angle of rotation of the steering wheel using a sensor. The detection unit can also detect dangerous situations by measuring the rate at which the distance to the vehicle in front decreases rapidly using a sensor. In this way, dangerous situations and abnormalities such as sudden braking, sudden steering maneuvers, and a rapid decrease in the distance to the vehicle in front can be detected early, allowing for the early detection of dangers while driving. Some or all of the above-described processes in the detection unit may be performed using AI, or they may not be performed using AI. For example, the detection unit can input sudden braking data into a generating AI and have the generating AI perform sudden braking detection.

[0038] The warning unit can warn the driver by voice or on screen. For example, the warning unit can alert the driver to danger by voice. The warning unit can also alert the driver to danger by on screen. The warning unit can use AI to adjust the content and timing of the warning. This allows the driver to respond quickly to dangerous situations by providing warnings by voice or on screen. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit can input data on dangerous situations into a generating AI and have the generating AI adjust the content and timing of the warning.

[0039] The recording and storage unit can automatically save video footage immediately before and after an accident when the impact sensor is activated. For example, the recording and storage unit saves video footage immediately before the accident when the impact sensor is activated. The recording and storage unit can also save video footage immediately after the accident. The recording and storage unit can use AI to adjust the recording storage conditions and storage period. This allows for a detailed record of the accident by automatically saving video footage before and after the accident when the impact sensor is activated. Some or all of the above-described processes in the recording and storage unit may be performed using AI or not. For example, the recording and storage unit can input impact sensor data into a generating AI and have the generating AI adjust the recording storage conditions and storage period.

[0040] The emergency call unit can notify emergency contacts of the occurrence of a specific accident. For example, the emergency call unit can notify the driver's family of the accident. The emergency call unit can also notify the driver's workplace of the accident. The emergency call unit can use AI to adjust the content and timing of the notification. This enables a quick response by notifying emergency contacts of the accident. Some or all of the above processes in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input accident data into a generating AI and have the generating AI adjust the content and timing of the notification.

[0041] The advice unit can analyze the driver's driving patterns or habits and provide advice for safe driving. For example, the advice unit can analyze the frequency of the driver's sudden braking and provide advice to reduce sudden braking. The advice unit can also analyze the characteristics of the driver's steering and provide advice for smoother steering. The advice unit can comprehensively analyze the driver's driving patterns and provide specific advice for safe driving. In this way, by analyzing the driver's driving patterns and habits and providing advice for safe driving, the driver's driving skills can be improved. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's driving data into a generating AI and have the generating AI execute advice on driving skills.

[0042] The monitoring unit can monitor the vehicle's internal environment and collect specific data to improve driver comfort. For example, the monitoring unit can measure the interior temperature with a sensor and collect data. The monitoring unit can also measure the interior humidity with a sensor and collect data. The monitoring unit can also measure the interior air quality with a sensor and collect data. By monitoring the vehicle's internal environment and collecting data to improve driver comfort, driver comfort can be improved. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input interior environment data into a generating AI and have the generating AI perform data collection for comfort improvement.

[0043] The monitoring unit can monitor the driver's gaze and facial orientation and detect specific signs of distraction. For example, if the driver's gaze is diverting from the road, the AI ​​in the monitoring unit can detect signs of distraction. The monitoring unit can also detect signs of distraction if the driver's face is frequently turned to the side. The monitoring unit can also detect signs of distraction if the driver is frequently using their smartphone. In this way, by monitoring the driver's gaze and facial orientation and detecting signs of distraction, the driver's attention can be maintained. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input the driver's gaze data into a generating AI and have the generating AI perform the detection of signs of distraction.

[0044] The monitoring unit can monitor the vehicle's external environment and provide specific driving advice to the driver. For example, the monitoring unit can monitor weather and traffic conditions and collect data. The monitoring unit can analyze this data using AI and provide appropriate driving advice to the driver. For example, the monitoring unit can advise the driver to slow down in rainy weather. The monitoring unit can also suggest alternative routes to the driver during traffic congestion. The monitoring unit can also advise the driver to check the use of lights when driving at night. In this way, by monitoring the vehicle's external environment and providing appropriate driving advice to the driver, safe driving can be supported. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input external environment data into a generating AI and have the generating AI perform the task of providing driving advice.

[0045] The monitoring unit can monitor the vehicle's maintenance status and propose specific maintenance. For example, the monitoring unit can measure tire pressure with a sensor and collect data. The monitoring unit can also measure oil levels with a sensor and collect data. The monitoring unit can also measure brake pad wear with a sensor and collect data. This allows the vehicle's safety to be maintained by monitoring its maintenance status and proposing necessary maintenance. Some or all of the above-described processes in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input maintenance data into a generating AI and have the generating AI execute maintenance proposals.

[0046] The detection unit can analyze the vehicle's behavior and detect specific abnormal driving patterns. For example, the detection unit can analyze data on sudden acceleration and sudden deceleration to detect abnormal driving patterns. The detection unit can also analyze data from a gyro sensor to detect sudden steering maneuvers. The detection unit can also detect vehicle skidding and detect abnormal driving patterns. By analyzing the vehicle's behavior and detecting abnormal driving patterns, abnormalities during driving can be detected early. Some or all of the above-described processes in the detection unit may be performed using AI, or they may not be performed using AI. For example, the detection unit can input vehicle behavior data into a generating AI and have the generating AI perform the detection of abnormal driving patterns.

[0047] The detection unit can recognize road signs or signals and provide specific driving instructions to the driver. For example, the detection unit can recognize road signs and notify the driver of speed limits. The detection unit can also recognize the color of traffic signals and provide the driver with instructions to stop or proceed. The detection unit can also recognize stop signs and instruct the driver to stop. In this way, by recognizing road signs and signals and providing appropriate driving instructions to the driver, safe driving can be supported. Some or all of the above processing in the detection unit may be performed using AI or not. For example, the detection unit can input road sign and signal data into a generating AI and have the generating AI perform the task of providing driving instructions.

[0048] The detection unit can communicate with other vehicles and predict the movement of surrounding vehicles to detect specific hazards. For example, the detection unit can acquire the speed and location information of other vehicles and predict the risk of collision. The detection unit can also detect the braking operation of other vehicles and predict the risk of sudden stopping. The detection unit can also detect lane changes by other vehicles and predict the risk of contact. In this way, by communicating with other vehicles and predicting the movement of surrounding vehicles to detect hazards, the risk of collision can be reduced. Some or all of the above processing in the detection unit may be performed using AI or not. For example, the detection unit can input data from other vehicles into a generating AI and have the generating AI perform hazard prediction.

[0049] The detection unit can detect the movement of pedestrians or cyclists and issue warnings to drivers. For example, the detection unit can issue a warning to a driver when a pedestrian is crossing a crosswalk. The detection unit can also issue a warning to a driver when a cyclist is riding on the roadway. The detection unit can also issue a warning to a driver when a pedestrian suddenly steps into the roadway. By detecting the movement of pedestrians and cyclists and issuing warnings to drivers, the risk of collisions with pedestrians and cyclists can be reduced. Some or all of the above processing in the detection unit may be performed using AI or not. For example, the detection unit can input pedestrian and cyclist data into a generating AI and have the generating AI issue warnings.

[0050] The warning unit can select a specific warning method by referring to the driver's past response data when issuing a warning. For example, the warning unit may prioritize using warning methods to which the driver has responded quickly in the past. The warning unit may also avoid warning methods that the driver has ignored in the past. Based on past response data, the warning unit may also select the most effective warning method for the driver. This allows for effective warnings for the driver by selecting the optimal warning method by referring to the driver's past response data. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit may input the driver's past response data into a generating AI and have the generating AI perform the selection of a warning method.

[0051] The warning unit can customize the warning content to suit the driver's language or culture. For example, the warning unit can issue warnings in the driver's native language. The warning unit can also issue warnings using language appropriate to the driver's culture. The warning unit can also issue warnings based on the traffic rules of the driver's region. By customizing the warning content to suit the driver's language and culture, it becomes possible to provide warnings that are easy for the driver to understand. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit can input the driver's language and culture data into a generating AI and have the generating AI perform the customization of the warning content.

[0052] The warning unit can provide specific warnings when a warning is issued, taking into account the driver's health condition. For example, if the driver's heart rate is high, the warning unit will issue a warning in a gentle voice. If the driver's stress level is high, the warning unit can also prioritize visual warnings. If the driver's health condition is good, the warning unit can use the normal warning method. This allows for optimal warnings for the driver by considering their health condition. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit can input the driver's health data into a generating AI and have the generating AI adjust the warning content.

[0053] The warning unit can be customized to the driver's preferences. For example, the warning unit can issue warnings using a voice preferred by the driver. The warning unit can also issue warnings using a visual display method preferred by the driver. The warning unit can also issue warnings at a time preferred by the driver. By customizing the warning content to the driver's preferences, it becomes possible to provide warnings that are easy for the driver to understand. Some or all of the above processes in the warning unit may be performed using AI or not. For example, the warning unit can input driver preference data into a generating AI and have the generating AI perform the customization of the warning content.

[0054] The recording and storage unit can determine the priority of saving footage based on its specific importance during recording. For example, the recording and storage unit may prioritize saving footage before and after an accident. It can also prioritize saving footage when sudden braking or steering maneuvers occur. It can also prioritize saving footage when an abnormal driving pattern is detected. By determining the priority of saving based on the importance of the footage, important footage can be saved preferentially. Some or all of the above processing in the recording and storage unit may be performed using AI or not. For example, the recording and storage unit can input video data into a generating AI and have the generating AI determine the priority of saving.

[0055] The recording storage unit can automatically back up recorded data to the cloud. For example, the recording storage unit will automatically back up recorded data to the cloud if it exceeds a certain capacity. The recording storage unit can also automatically back up accident footage to the cloud when it is saved. The recording storage unit can also periodically back up recorded data to the cloud. This ensures data security by automatically backing up recorded data to the cloud. Some or all of the above processes in the recording storage unit may be performed using AI or not. For example, the recording storage unit can input recorded data into a generating AI and have the generating AI perform the cloud backup.

[0056] The recording and storage unit can add specific vehicle location information to the recording and save it. For example, the recording and storage unit can add GPS information to the recording data and save it. The recording and storage unit can also add vehicle speed information and save the recording data. By adding and saving the vehicle's location information during recording, the location information of the video can be determined. Some or all of the above processing in the recording and storage unit may be performed using AI or not. For example, the recording and storage unit can input location information data into a generating AI and have the generating AI perform the addition of location information.

[0057] The recording storage unit can analyze the recorded data and provide specific feedback to the driver. For example, the recording storage unit can analyze the recorded data and provide the driver with advice on safe driving. The recording storage unit can also analyze the recorded data and provide the driver with feedback to improve their driving skills. The recording storage unit can analyze the recorded data and suggest areas for improvement in driving to the driver. In this way, by analyzing the recorded data and providing feedback to the driver, the driver's driving skills can be improved. Some or all of the above processing in the recording storage unit may be performed using AI or not. For example, the recording storage unit can input the recorded data into a generating AI and have the generating AI perform the task of providing feedback.

[0058] The emergency call unit can automatically transmit specific details of an accident when an emergency call is made. For example, the emergency call unit can include the intensity of the impact of the accident in the report based on impact sensor data. The emergency call unit can also include the location of the accident in the report based on GPS data. The emergency call unit can also include the vehicle's speed at the time of the accident in the report based on vehicle speed data. This enables a rapid response by automatically transmitting detailed accident information when an emergency call is made. Some or all of the above processing in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input accident data into a generating AI and have the generating AI execute the transmission of detailed information.

[0059] The emergency call unit can monitor the driver's health condition during an emergency call and reflect this information in the specific content of the call. For example, if the driver's heart rate is abnormally high, the emergency call unit will include that information in the call. The emergency call unit can also include information if the driver is unconscious. The emergency call unit can also include information if the driver's health condition is poor. This allows for appropriate emergency response by monitoring the driver's health condition during an emergency call and reflecting it in the call content. Some or all of the above processing in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input the driver's health data into a generating AI and have the generating AI adjust the content of the call.

[0060] The emergency call unit can add specific vehicle maintenance information to the emergency call before transmitting it. For example, the emergency call unit can include information on the vehicle's tire pressure in the emergency call. It can also include information on the vehicle's oil level in the emergency call. It can also include information on the wear of the vehicle's brake pads in the emergency call. By adding vehicle maintenance information to the emergency call before transmitting it, the cause and circumstances of the accident can be understood in more detail. Some or all of the above processing in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input maintenance data into a generating AI and have the generating AI perform the addition of emergency call information.

[0061] The emergency call unit can automatically send specific notifications to the driver's emergency contacts in the event of an emergency. For example, the emergency call unit can automatically notify the driver's family in the event of an emergency. The emergency call unit can also automatically notify the driver's workplace in the event of an emergency. The emergency call unit can also automatically notify emergency contacts designated by the driver in the event of an emergency. This enables a rapid response by automatically notifying the driver's emergency contacts in the event of an emergency. Some or all of the above processes in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input emergency contact data into a generating AI and have the generating AI execute the notifications.

[0062] The advice unit can provide personalized and specific advice by referring to the driver's past driving data when providing advice. For example, the advice unit can provide advice that takes into account the driver's habits based on past driving data. The advice unit can also provide advice to improve the driver's weaknesses based on past driving data. The advice unit can also provide advice to leverage the driver's strengths based on past driving data. In this way, the driver's driving skills can be improved by providing personalized advice by referring to the driver's past driving data. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's past driving data into a generating AI and have the generating AI perform the task of providing personalized advice.

[0063] The advice unit can adjust the advice content to match the driver's driving style or preferences. For example, if the driver prioritizes safe driving, the advice unit will provide safe driving advice. If the driver prioritizes efficient driving, the advice unit can also provide efficient driving advice. If the driver prioritizes comfortable driving, the advice unit can also provide comfortable driving advice. By adjusting the advice content to match the driver's driving style and preferences, it becomes possible to provide the optimal advice for the driver. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's driving style and preference data into a generating AI and have the generating AI perform the adjustment of the advice content.

[0064] The advice unit can provide specific advice while considering the driver's health condition. For example, if the driver is tired, the advice unit may advise them to take a break. If the driver's stress level is high, the advice unit may also provide advice on how to relax. If the driver's health condition is not good, the advice unit may also provide advice on safe driving. By considering the driver's health condition when providing advice, it becomes possible to provide the most appropriate advice for the driver. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit may input the driver's health data into a generating AI and have the generating AI adjust the content of the advice.

[0065] The advice unit can customize the advice to suit the driver's language and culture. For example, the advice unit can provide advice in the driver's native language. The advice unit can also provide advice in language appropriate to the driver's culture. The advice unit can also provide advice based on the traffic rules of the driver's region. By customizing the advice to suit the driver's language and culture, it becomes possible to provide advice that is easy for the driver to understand. Some or all of the above processes in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's language and cultural data into a generating AI and have the generating AI perform the customization of the advice.

[0066] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0067] The AI ​​assistant in the dashcam can monitor the driver's health and provide driving advice based on that health condition. For example, it can measure the driver's heart rate and blood pressure with sensors and advise them to take a break if an abnormality is detected. It can also advise the driver to take regular breaks if they are driving for a long time. Furthermore, if the driver's health is not good, it can advise them to refrain from driving. In this way, it can support safe driving by providing driving advice that takes the driver's health condition into consideration.

[0068] The AI ​​assistant in the dashcam can monitor the driver's gaze and provide driving advice based on their eye movements. For example, if the driver's gaze is diverting from the road, it can provide a warning. If the driver is frequently using their smartphone, it can advise them to reduce their smartphone use. If the driver's gaze is focused on the road ahead, it can advise them to continue driving safely. In this way, by providing driving advice based on the driver's gaze, it can support safe driving.

[0069] The AI ​​assistant in the dashcam can analyze the driver's driving style and provide advice to improve fuel efficiency based on that style. For example, if the driver frequently accelerates or brakes suddenly, it can advise them to drive more smoothly. If the driver frequently drives on highways, it can also advise them to maintain a constant speed. If the driver frequently drives in urban areas, it can advise them to consider traffic light timings. In this way, by providing advice to improve fuel efficiency based on driving style, it can support economical driving.

[0070] The AI ​​assistant in the dashcam can analyze the driver's driving history and suggest insurance premium discounts based on that history. For example, if a driver has a long history of safe driving, it can suggest a discount to the insurance company. It can also suggest a discount if the driver has never had an accident. It can also suggest a discount if the driver performs regular maintenance. In this way, by suggesting insurance premium discounts based on driving history, the financial burden on the driver can be reduced.

[0071] The AI ​​assistant in the dashcam can analyze the driver's driving patterns and suggest vehicle maintenance schedules based on those patterns. For example, if there are frequent sudden braking and acceleration, it may suggest replacing the brake pads sooner. If long-distance driving is common, it may suggest increasing the frequency of oil changes. If driving mostly in urban areas, it may suggest checking tire wear earlier. In this way, vehicle safety can be maintained by suggesting maintenance schedules based on driving patterns.

[0072] The following briefly describes the processing flow for example form 1.

[0073] Step 1: The monitoring unit monitors the driving conditions in real time. The monitoring unit collects data such as vehicle speed, the movement of surrounding vehicles, and road conditions. The monitoring unit uses AI to analyze this data and detect dangerous situations or anomalies. Step 2: The detection unit detects dangerous situations or anomalies based on the data collected by the monitoring unit. The detection unit can detect situations such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. The detection unit can use AI to analyze these situations and detect dangerous situations or anomalies. Step 3: The warning unit issues a warning for dangerous situations or abnormalities detected by the detection unit. The warning unit warns the driver, for example, through voice or on-screen display. The warning unit can use AI to adjust the content and timing of the warning. Step 4: The recording storage unit automatically saves recordings in the event of an accident. For example, if the impact sensor is activated, the recording storage unit automatically saves video footage before and after the accident. The recording storage unit can use AI to adjust the recording storage conditions and storage period. Step 5: The emergency call unit makes an emergency call based on the video footage saved by the video storage unit. For example, the emergency call unit notifies emergency contacts of the occurrence of an accident. The emergency call unit can use AI to adjust the content and timing of the call. Step 6: The advice unit analyzes past driving data and provides advice on driving skills. For example, the advice unit analyzes the driver's driving patterns and habits and provides advice for safe driving. The advice unit can use AI to adjust the content and timing of the advice.

[0074] (Example of form 2) The AI ​​assistant for a drive recorder according to an embodiment of the present invention is a system that provides real-time warnings of dangerous situations and abnormalities while driving, automatic recording and saving in the event of an accident, emergency call functions, and advice on driving skills based on past driving data. The AI ​​assistant for the drive recorder monitors the driving situation in real time and detects dangerous situations and abnormalities. Next, it issues a warning for the detected dangerous situations and abnormalities. Furthermore, if an accident occurs, it automatically saves the recording and makes an emergency call. It also analyzes past driving data and provides advice on driving skills. For example, the AI ​​assistant for the drive recorder monitors the vehicle's speed, the movement of surrounding vehicles, and road conditions. In this process, the AI ​​analyzes this data and detects dangerous situations and abnormalities. For example, it can detect situations such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. Next, it issues a warning for the detected dangerous situations and abnormalities. For example, it warns the driver by voice or on-screen display. This allows the driver to respond quickly to dangerous situations. Furthermore, if an accident occurs, it automatically saves the recording and makes an emergency call. For example, if the impact sensor is activated, it automatically saves video footage before and after the accident. It also notifies emergency contacts of the accident through its emergency call function, enabling a quick response. Furthermore, it analyzes past driving data to provide driving skill advice. For instance, it analyzes the driver's driving patterns and habits to provide advice for safer driving, allowing the driver to improve their skills. In this way, the AI ​​assistant in the dashcam supports safe driving by providing real-time warnings of dangerous situations and anomalies while driving, automatic recording and saving of accident footage, emergency call functions, and driving skill advice based on past driving data.

[0075] The AI ​​assistant for the drive recorder according to this embodiment comprises a monitoring unit, a detection unit, a warning unit, a recording storage unit, an emergency call unit, and an advice unit. The monitoring unit monitors the driving situation in real time. The monitoring unit collects data such as vehicle speed, the movement of surrounding vehicles, and road conditions. The monitoring unit can analyze this data using AI to detect dangerous situations or abnormalities. The detection unit detects dangerous situations or abnormalities based on the data collected by the monitoring unit. The detection unit detects situations such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. The detection unit can analyze these situations using AI to detect dangerous situations or abnormalities. The warning unit issues a warning for dangerous situations or abnormalities detected by the detection unit. The warning unit warns the driver, for example, by voice or on-screen display. The warning unit can adjust the content and timing of the warning using AI. The recording storage unit automatically saves the recording in the event of an accident. The recording and storage unit automatically saves video footage before and after an accident, for example, when the impact sensor is activated. The recording and storage unit can adjust the recording storage conditions and storage period using AI. The emergency call unit makes an emergency call based on the video footage saved by the recording and storage unit. The emergency call unit notifies emergency contacts of the occurrence of an accident, for example. The emergency call unit can adjust the content and timing of the call using AI. The advice unit analyzes past driving data and provides advice on driving skills. The advice unit analyzes the driver's driving patterns and habits, for example, and provides advice for safe driving. The advice unit can adjust the content and timing of the advice using AI. As a result, the AI ​​assistant of the drive recorder according to this embodiment can support safe driving by providing real-time warnings of dangerous situations and abnormalities while driving, automatic recording and saving in the event of an accident, emergency call functions, and advice on driving skills based on past driving data.

[0076] The monitoring unit monitors driving conditions in real time. For example, it collects data such as vehicle speed, the movement of surrounding vehicles, and road conditions. Specifically, it uses the vehicle's speed sensor and GPS to determine the current speed and position, and cameras and radar sensors to detect the movement of surrounding vehicles. Regarding road conditions, it analyzes camera footage to recognize lane positions, traffic light status, and road signs. This data is analyzed in real time using AI to detect dangerous situations and anomalies. For example, the AI ​​uses image recognition technology to measure the distance to the vehicle ahead from camera footage and issues a warning if it is approaching too closely. It also analyzes speed sensor data to detect anomalies such as sudden braking or sharp steering maneuvers. Furthermore, it considers changes in road conditions and weather, enabling it to detect slippery surfaces and poor visibility. This allows the monitoring unit to monitor various driving conditions in real time and provide appropriate information to the driver.

[0077] The detection unit detects dangerous situations or anomalies based on data collected by the monitoring unit. For example, the detection unit can detect situations such as sudden braking, sudden steering maneuvers, or a rapid decrease in the distance to the vehicle in front. By using AI to analyze these situations, it can detect dangerous situations and anomalies. Specifically, the AI ​​learns past driving data using machine learning algorithms and identifies abnormal patterns. For example, in the case of sudden braking, it analyzes data on speed changes and steering maneuvers before and after the braking and detects anomalies by comparing them with normal driving patterns. Also, in the case of a rapid decrease in the distance to the vehicle in front, the AI ​​analyzes camera images and radar sensor data to assess the risk of collision. Furthermore, the detection unit can learn the driver's behavior patterns and habits and perform anomaly detection tailored to the individual driving style. As a result, the detection unit can quickly and accurately detect specific dangerous situations and anomalies while driving and issue appropriate warnings to the driver.

[0078] The warning unit issues warnings for dangerous situations or anomalies detected by the detection unit. The warning unit warns the driver, for example, through voice or screen display. The content and timing of warnings can be adjusted using AI. Specifically, the AI ​​selects the optimal warning method according to the type and urgency of the detected danger. For example, if the distance to the vehicle in front decreases rapidly, it will issue a voice warning of "Pay attention ahead" and simultaneously display a warning message on the dashboard screen. In the event of sudden braking, it can also use a vibration alert to draw the driver's attention. Furthermore, the warning unit can monitor the driver's response and adjust the intensity and frequency of warnings as needed. For example, if the driver does not take an appropriate action in response to a warning, it will increase the volume of the warning or issue repeated warnings to draw attention. In this way, the warning unit can provide the driver with effective warnings at the appropriate time and help them avoid dangerous situations.

[0079] The recording storage unit automatically saves recordings in the event of an accident. For example, if an impact sensor is activated, the recording storage unit automatically saves footage before and after an accident. AI can be used to adjust the recording storage conditions and storage period. Specifically, if the impact sensor exceeds a certain threshold, the recording storage unit immediately starts recording and saves footage for several seconds before and after the accident. The AI ​​can analyze the saved footage and automatically extract important scenes and parts that can serve as evidence. The AI ​​also sets an appropriate storage period according to the severity of the accident and legal requirements. For example, a short storage period is sufficient for minor accidents, but a long storage period is necessary for serious accidents. Furthermore, the recording storage unit can automatically upload the saved footage to a cloud server, ensuring data security and easy access. This allows the recording storage unit to reliably save important footage from the time of the accident, which can then be used for later analysis and evidence submission.

[0080] The emergency call unit makes emergency calls based on video footage saved by the video storage unit. For example, the emergency call unit notifies emergency contacts of the occurrence of an accident. AI can be used to adjust the content and timing of the call. Specifically, the AI ​​analyzes the severity and location information of the accident and selects the most appropriate emergency contacts. For example, in the event of a serious accident, the emergency call unit immediately notifies the police and ambulance services and provides detailed information and location information about the accident. It also automatically notifies the driver's emergency contacts to encourage a quick response. Furthermore, the AI ​​can automatically generate the content of the call, clearly communicating the situation of the accident and the necessary actions. For example, it can generate a specific call such as, "A collision has occurred with the vehicle in front. Our current location is XX. Please dispatch an ambulance." This allows the emergency call unit to make quick and appropriate calls when an accident occurs, preventing the escalation of damage.

[0081] The advice unit analyzes past driving data and provides advice on driving skills. For example, it analyzes the driver's driving patterns and habits and provides advice for safe driving. The content and timing of the advice can be adjusted using AI. Specifically, the AI ​​learns from past driving data and identifies the driver's characteristics and areas for improvement. For example, if sudden braking or sudden steering is frequent, the AI ​​analyzes the cause and provides appropriate advice. For example, it may give specific advice such as, "You brake suddenly a lot, so try to maintain a little more distance from the vehicle in front." It can also provide individualized advice tailored to the driver's driving style. For example, if the driver drives for long periods of time, it may also give health-related advice such as, "Take regular breaks." Furthermore, the advice unit can collect driver feedback and continuously improve the content and method of advice. In this way, the advice unit can support the improvement of drivers' driving skills and contribute to the realization of safe driving.

[0082] The monitoring unit can collect data on vehicle speed, the movement of surrounding vehicles, road conditions, and other data. For example, the monitoring unit can measure the vehicle speed with a sensor and collect the data. The monitoring unit can also monitor the movement of surrounding vehicles with a camera and collect the data. The monitoring unit can also monitor road conditions with sensors and cameras and collect the data. The monitoring unit can also collect other data such as weather information and road construction information. This allows for detailed monitoring of driving conditions by collecting data such as vehicle speed, the movement of surrounding vehicles, and road conditions. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input vehicle speed data into a generating AI and have the generating AI perform analysis of the speed data.

[0083] The detection unit can detect dangerous situations or abnormalities such as sudden braking or sudden steering maneuvers, or a rapid decrease in the distance to the vehicle in front. For example, the detection unit can detect sudden braking by measuring the speed at which the brake pedal is pressed using a sensor. The detection unit can also detect sudden steering maneuvers by measuring the angle of rotation of the steering wheel using a sensor. The detection unit can also detect dangerous situations by measuring the rate at which the distance to the vehicle in front decreases rapidly using a sensor. In this way, dangerous situations and abnormalities such as sudden braking, sudden steering maneuvers, and a rapid decrease in the distance to the vehicle in front can be detected early, allowing for the early detection of dangers while driving. Some or all of the above-described processes in the detection unit may be performed using AI, or they may not be performed using AI. For example, the detection unit can input sudden braking data into a generating AI and have the generating AI perform sudden braking detection.

[0084] The warning unit can warn the driver by voice or on screen. For example, the warning unit can alert the driver to danger by voice. The warning unit can also alert the driver to danger by on screen. The warning unit can use AI to adjust the content and timing of the warning. This allows the driver to respond quickly to dangerous situations by providing warnings by voice or on screen. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit can input data on dangerous situations into a generating AI and have the generating AI adjust the content and timing of the warning.

[0085] The recording and storage unit can automatically save video footage immediately before and after an accident when the impact sensor is activated. For example, the recording and storage unit saves video footage immediately before the accident when the impact sensor is activated. The recording and storage unit can also save video footage immediately after the accident. The recording and storage unit can use AI to adjust the recording storage conditions and storage period. This allows for a detailed record of the accident by automatically saving video footage before and after the accident when the impact sensor is activated. Some or all of the above-described processes in the recording and storage unit may be performed using AI or not. For example, the recording and storage unit can input impact sensor data into a generating AI and have the generating AI adjust the recording storage conditions and storage period.

[0086] The emergency call unit can notify emergency contacts of the occurrence of a specific accident. For example, the emergency call unit can notify the driver's family of the accident. The emergency call unit can also notify the driver's workplace of the accident. The emergency call unit can use AI to adjust the content and timing of the notification. This enables a quick response by notifying emergency contacts of the accident. Some or all of the above processes in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input accident data into a generating AI and have the generating AI adjust the content and timing of the notification.

[0087] The advice unit can analyze the driver's driving patterns or habits and provide advice for safe driving. For example, the advice unit can analyze the frequency of the driver's sudden braking and provide advice to reduce sudden braking. The advice unit can also analyze the characteristics of the driver's steering and provide advice for smoother steering. The advice unit can comprehensively analyze the driver's driving patterns and provide specific advice for safe driving. In this way, by analyzing the driver's driving patterns and habits and providing advice for safe driving, the driver's driving skills can be improved. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's driving data into a generating AI and have the generating AI execute advice on driving skills.

[0088] The monitoring unit can estimate the driver's emotions and adjust the monitoring accuracy based on the estimated emotions. For example, if the driver is stressed, the AI ​​can increase the monitoring accuracy and collect more detailed data. If the driver is relaxed, the AI ​​can return the monitoring accuracy to a normal level and collect only the minimum necessary data. If the driver is tired, the AI ​​can adjust the monitoring accuracy to a moderate level and continuously monitor the driver's condition. This allows for more appropriate monitoring by adjusting the monitoring accuracy based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input driver emotion data into a generative AI and have the generative AI adjust the monitoring accuracy.

[0089] The monitoring unit can monitor the vehicle's internal environment and collect specific data to improve driver comfort. For example, the monitoring unit can measure the interior temperature with a sensor and collect data. The monitoring unit can also measure the interior humidity with a sensor and collect data. The monitoring unit can also measure the interior air quality with a sensor and collect data. By monitoring the vehicle's internal environment and collecting data to improve driver comfort, driver comfort can be improved. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input interior environment data into a generating AI and have the generating AI perform data collection for comfort improvement.

[0090] The monitoring unit can monitor the driver's gaze and facial orientation and detect specific signs of distraction. For example, if the driver's gaze is diverting from the road, the AI ​​in the monitoring unit can detect signs of distraction. The monitoring unit can also detect signs of distraction if the driver's face is frequently turned to the side. The monitoring unit can also detect signs of distraction if the driver is frequently using their smartphone. In this way, by monitoring the driver's gaze and facial orientation and detecting signs of distraction, the driver's attention can be maintained. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input the driver's gaze data into a generating AI and have the generating AI perform the detection of signs of distraction.

[0091] The monitoring unit can estimate the driver's emotions and determine the specific priorities of the data to monitor based on the estimated emotions. For example, if the driver is stressed, the AI ​​in the monitoring unit can prioritize monitoring the vehicle's speed and the movement of surrounding vehicles. If the driver is relaxed, the AI ​​in the monitoring unit can also prioritize monitoring road conditions and the vehicle's internal environment. If the driver is tired, the AI ​​in the monitoring unit can also prioritize monitoring the driver's gaze and facial direction. This allows for prioritizing important data by determining the priorities of the data to monitor based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input driver emotion data into a generative AI and have the generative AI determine the priorities of the monitoring data.

[0092] The monitoring unit can monitor the vehicle's external environment and provide specific driving advice to the driver. For example, the monitoring unit can monitor weather and traffic conditions and collect data. The monitoring unit can analyze this data using AI and provide appropriate driving advice to the driver. For example, the monitoring unit can advise the driver to slow down in rainy weather. The monitoring unit can also suggest alternative routes to the driver during traffic congestion. The monitoring unit can also advise the driver to check the use of lights when driving at night. In this way, by monitoring the vehicle's external environment and providing appropriate driving advice to the driver, safe driving can be supported. Some or all of the above processing in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input external environment data into a generating AI and have the generating AI perform the task of providing driving advice.

[0093] The monitoring unit can monitor the vehicle's maintenance status and propose specific maintenance. For example, the monitoring unit can measure tire pressure with a sensor and collect data. The monitoring unit can also measure oil levels with a sensor and collect data. The monitoring unit can also measure brake pad wear with a sensor and collect data. This allows the vehicle's safety to be maintained by monitoring its maintenance status and proposing necessary maintenance. Some or all of the above-described processes in the monitoring unit may be performed using AI or not. For example, the monitoring unit can input maintenance data into a generating AI and have the generating AI execute maintenance proposals.

[0094] The detection unit can estimate the driver's emotions and adjust specific detection criteria for dangerous situations based on the estimated emotions. For example, if the driver is stressed, the detection unit's AI can set the detection criteria for dangerous situations more strictly. If the driver is relaxed, the detection unit can also set the detection criteria for dangerous situations to a normal level. If the driver is tired, the detection unit can also set the detection criteria for dangerous situations to a moderate level. This allows for more accurate hazard detection by adjusting the detection criteria for dangerous situations based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the detection unit may be performed using AI or not. For example, the detection unit can input driver emotion data into a generative AI and have the generative AI adjust the hazard detection criteria.

[0095] The detection unit can analyze the vehicle's behavior and detect specific abnormal driving patterns. For example, the detection unit can analyze data on sudden acceleration and sudden deceleration to detect abnormal driving patterns. The detection unit can also analyze data from a gyro sensor to detect sudden steering maneuvers. The detection unit can also detect vehicle skidding and detect abnormal driving patterns. By analyzing the vehicle's behavior and detecting abnormal driving patterns, abnormalities during driving can be detected early. Some or all of the above-described processes in the detection unit may be performed using AI, or they may not be performed using AI. For example, the detection unit can input vehicle behavior data into a generating AI and have the generating AI perform the detection of abnormal driving patterns.

[0096] The detection unit can recognize road signs or signals and provide specific driving instructions to the driver. For example, the detection unit can recognize road signs and notify the driver of speed limits. The detection unit can also recognize the color of traffic signals and provide the driver with instructions to stop or proceed. The detection unit can also recognize stop signs and instruct the driver to stop. In this way, by recognizing road signs and signals and providing appropriate driving instructions to the driver, safe driving can be supported. Some or all of the above processing in the detection unit may be performed using AI or not. For example, the detection unit can input road sign and signal data into a generating AI and have the generating AI perform the task of providing driving instructions.

[0097] The detection unit can estimate the driver's emotions and adjust the specific display method of the detection results based on the estimated emotions of the driver. For example, if the driver is tense, the detection unit can provide a simple and highly visible display method. If the driver is relaxed, the detection unit can also provide a display method that includes detailed information. If the driver is in a hurry, the detection unit can also provide a display method that gets straight to the point. By adjusting the display method of the detection results based on the driver's emotions, it is possible to provide the optimal display for the driver. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the detection unit may be performed using AI or not. For example, the detection unit can input the driver's emotion data into the generative AI and have the generative AI perform the adjustment of the display method.

[0098] The detection unit can communicate with other vehicles and predict the movement of surrounding vehicles to detect specific hazards. For example, the detection unit can acquire the speed and location information of other vehicles and predict the risk of collision. The detection unit can also detect the braking operation of other vehicles and predict the risk of sudden stopping. The detection unit can also detect lane changes by other vehicles and predict the risk of contact. In this way, by communicating with other vehicles and predicting the movement of surrounding vehicles to detect hazards, the risk of collision can be reduced. Some or all of the above processing in the detection unit may be performed using AI or not. For example, the detection unit can input data from other vehicles into a generating AI and have the generating AI perform hazard prediction.

[0099] The detection unit can detect the movement of pedestrians or cyclists and issue warnings to drivers. For example, the detection unit can issue a warning to a driver when a pedestrian is crossing a crosswalk. The detection unit can also issue a warning to a driver when a cyclist is riding on the roadway. The detection unit can also issue a warning to a driver when a pedestrian suddenly steps into the roadway. By detecting the movement of pedestrians and cyclists and issuing warnings to drivers, the risk of collisions with pedestrians and cyclists can be reduced. Some or all of the above processing in the detection unit may be performed using AI or not. For example, the detection unit can input pedestrian and cyclist data into a generating AI and have the generating AI issue warnings.

[0100] The warning unit can estimate the driver's emotions and adjust the intensity or method of the warning based on the estimated emotions. For example, if the driver is tense, the warning unit can issue a warning in a calm voice. If the driver is relaxed, the warning unit can also issue a warning in a normal voice. If the driver is in a hurry, the warning unit can also issue a warning in a quick and concise voice. This allows for optimal warnings for the driver by adjusting the intensity and method of the warning based on their emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit can input driver emotion data into a generative AI and have the generative AI adjust the intensity and method of the warning.

[0101] The warning unit can select a specific warning method by referring to the driver's past response data when issuing a warning. For example, the warning unit may prioritize using warning methods to which the driver has responded quickly in the past. The warning unit may also avoid warning methods that the driver has ignored in the past. Based on past response data, the warning unit may also select the most effective warning method for the driver. This allows for effective warnings for the driver by selecting the optimal warning method by referring to the driver's past response data. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit may input the driver's past response data into a generating AI and have the generating AI perform the selection of a warning method.

[0102] The warning unit can customize the warning content to suit the driver's language or culture. For example, the warning unit can issue warnings in the driver's native language. The warning unit can also issue warnings using language appropriate to the driver's culture. The warning unit can also issue warnings based on the traffic rules of the driver's region. By customizing the warning content to suit the driver's language and culture, it becomes possible to provide warnings that are easy for the driver to understand. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit can input the driver's language and culture data into a generating AI and have the generating AI perform the customization of the warning content.

[0103] The warning unit can estimate the driver's emotions and adjust the timing of warnings based on the estimated emotions. For example, if the driver is tense, the warning unit may issue a warning earlier. If the driver is relaxed, the warning unit may issue a warning at the normal time. If the driver is in a hurry, the warning unit may issue a warning quickly. By adjusting the timing of warnings based on the driver's emotions, it is possible to issue warnings at the optimal time for the driver. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit can input driver emotion data into a generative AI and have the generative AI adjust the warning timing.

[0104] The warning unit can provide specific warnings when a warning is issued, taking into account the driver's health condition. For example, if the driver's heart rate is high, the warning unit will issue a warning in a gentle voice. If the driver's stress level is high, the warning unit can also prioritize visual warnings. If the driver's health condition is good, the warning unit can use the normal warning method. This allows for optimal warnings for the driver by considering their health condition. Some or all of the above processing in the warning unit may be performed using AI or not. For example, the warning unit can input the driver's health data into a generating AI and have the generating AI adjust the warning content.

[0105] The warning unit can be customized to the driver's preferences. For example, the warning unit can issue warnings using a voice preferred by the driver. The warning unit can also issue warnings using a visual display method preferred by the driver. The warning unit can also issue warnings at a time preferred by the driver. By customizing the warning content to the driver's preferences, it becomes possible to provide warnings that are easy for the driver to understand. Some or all of the above processes in the warning unit may be performed using AI or not. For example, the warning unit can input driver preference data into a generating AI and have the generating AI perform the customization of the warning content.

[0106] The recording storage unit can estimate the driver's emotions and adjust the specific recording storage period based on the estimated emotions. For example, if the driver is stressed, the recording storage unit can set a longer storage period. If the driver is relaxed, the recording storage unit can also set a normal storage period. If the driver is tired, the recording storage unit can also set a medium storage period. This allows the necessary footage to be stored for an appropriate period by adjusting the recording storage period based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the recording storage unit may be performed using AI or not. For example, the recording storage unit can input driver emotion data into a generative AI and have the generative AI adjust the recording storage period.

[0107] The recording and storage unit can determine the priority of saving footage based on its specific importance during recording. For example, the recording and storage unit may prioritize saving footage before and after an accident. It can also prioritize saving footage when sudden braking or steering maneuvers occur. It can also prioritize saving footage when an abnormal driving pattern is detected. By determining the priority of saving based on the importance of the footage, important footage can be saved preferentially. Some or all of the above processing in the recording and storage unit may be performed using AI or not. For example, the recording and storage unit can input video data into a generating AI and have the generating AI determine the priority of saving.

[0108] The recording storage unit can automatically back up recorded data to the cloud. For example, the recording storage unit will automatically back up recorded data to the cloud if it exceeds a certain capacity. The recording storage unit can also automatically back up accident footage to the cloud when it is saved. The recording storage unit can also periodically back up recorded data to the cloud. This ensures data security by automatically backing up recorded data to the cloud. Some or all of the above processes in the recording storage unit may be performed using AI or not. For example, the recording storage unit can input recorded data into a generating AI and have the generating AI perform the cloud backup.

[0109] The recording storage unit can estimate the driver's emotions and adjust the specific image quality of the recording based on the estimated emotions. For example, if the driver is stressed, the recording storage unit can set the image quality to high. If the driver is relaxed, the recording storage unit can also set the image quality to normal. If the driver is tired, the recording storage unit can also set the image quality to medium. In this way, by adjusting the image quality based on the driver's emotions, recording can be made at the desired quality. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the recording storage unit may be performed using AI or not. For example, the recording storage unit can input driver emotion data into a generative AI and have the generative AI perform the adjustment of the recording quality.

[0110] The recording and storage unit can add specific vehicle location information to the recording and save it. For example, the recording and storage unit can add GPS information to the recording data and save it. The recording and storage unit can also add vehicle speed information and save the recording data. By adding and saving the vehicle's location information during recording, the location information of the video can be determined. Some or all of the above processing in the recording and storage unit may be performed using AI or not. For example, the recording and storage unit can input location information data into a generating AI and have the generating AI perform the addition of location information.

[0111] The recording storage unit can analyze the recorded data and provide specific feedback to the driver. For example, the recording storage unit can analyze the recorded data and provide the driver with advice on safe driving. The recording storage unit can also analyze the recorded data and provide the driver with feedback to improve their driving skills. The recording storage unit can analyze the recorded data and suggest areas for improvement in driving to the driver. In this way, by analyzing the recorded data and providing feedback to the driver, the driver's driving skills can be improved. Some or all of the above processing in the recording storage unit may be performed using AI or not. For example, the recording storage unit can input the recorded data into a generating AI and have the generating AI perform the task of providing feedback.

[0112] The emergency call unit can estimate the driver's emotions and adjust the specific content of the emergency call based on the estimated emotions. For example, if the driver is in a state of panic, the emergency call unit will set a concise and quick message. If the driver is calm, the emergency call unit can also set a detailed message. If the driver is unconscious, the emergency call unit can automatically set the message. This allows for appropriate emergency calls by adjusting the content of the emergency call based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input driver emotion data into a generative AI and have the generative AI adjust the content of the emergency call.

[0113] The emergency call unit can automatically transmit specific details of an accident when an emergency call is made. For example, the emergency call unit can include the intensity of the impact of the accident in the report based on impact sensor data. The emergency call unit can also include the location of the accident in the report based on GPS data. The emergency call unit can also include the vehicle's speed at the time of the accident in the report based on vehicle speed data. This enables a rapid response by automatically transmitting detailed accident information when an emergency call is made. Some or all of the above processing in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input accident data into a generating AI and have the generating AI execute the transmission of detailed information.

[0114] The emergency call unit can monitor the driver's health condition during an emergency call and reflect this information in the specific content of the call. For example, if the driver's heart rate is abnormally high, the emergency call unit will include that information in the call. The emergency call unit can also include information if the driver is unconscious. The emergency call unit can also include information if the driver's health condition is poor. This allows for appropriate emergency response by monitoring the driver's health condition during an emergency call and reflecting it in the call content. Some or all of the above processing in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input the driver's health data into a generating AI and have the generating AI adjust the content of the call.

[0115] The emergency call unit can estimate the driver's emotions and adjust the specific timing of the emergency call based on the estimated emotions. For example, if the driver is in a state of panic, the emergency call unit will make an emergency call quickly. If the driver is calm, the emergency call unit can also assess the situation before making an emergency call. If the driver is unconscious, the emergency call unit can also automatically make an emergency call. This allows for emergency calls to be made at the appropriate time by adjusting the timing of the emergency call based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input driver emotion data into a generative AI and have the generative AI adjust the timing of the emergency call.

[0116] The emergency call unit can add specific vehicle maintenance information to the emergency call before transmitting it. For example, the emergency call unit can include information on the vehicle's tire pressure in the emergency call. It can also include information on the vehicle's oil level in the emergency call. It can also include information on the wear of the vehicle's brake pads in the emergency call. By adding vehicle maintenance information to the emergency call before transmitting it, the cause and circumstances of the accident can be understood in more detail. Some or all of the above processing in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input maintenance data into a generating AI and have the generating AI perform the addition of emergency call information.

[0117] The emergency call unit can automatically send specific notifications to the driver's emergency contacts in the event of an emergency. For example, the emergency call unit can automatically notify the driver's family in the event of an emergency. The emergency call unit can also automatically notify the driver's workplace in the event of an emergency. The emergency call unit can also automatically notify emergency contacts designated by the driver in the event of an emergency. This enables a rapid response by automatically notifying the driver's emergency contacts in the event of an emergency. Some or all of the above processes in the emergency call unit may be performed using AI or not. For example, the emergency call unit can input emergency contact data into a generating AI and have the generating AI execute the notifications.

[0118] The advice unit can estimate the driver's emotions and adjust the specific content of the advice based on the estimated emotions. For example, if the driver is feeling stressed, the advice unit can provide advice on how to relax. If the driver is relaxed, the advice unit can also provide normal driving advice. If the driver is tired, the advice unit can also advise the driver to take a break. By adjusting the content of the advice based on the driver's emotions, it becomes possible to provide the most appropriate advice for the driver. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's emotion data into the generative AI and have the generative AI adjust the content of the advice.

[0119] The advice unit can provide personalized and specific advice by referring to the driver's past driving data when providing advice. For example, the advice unit can provide advice that takes into account the driver's habits based on past driving data. The advice unit can also provide advice to improve the driver's weaknesses based on past driving data. The advice unit can also provide advice to leverage the driver's strengths based on past driving data. In this way, the driver's driving skills can be improved by providing personalized advice by referring to the driver's past driving data. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's past driving data into a generating AI and have the generating AI perform the task of providing personalized advice.

[0120] The advice unit can adjust the advice content to match the driver's driving style or preferences. For example, if the driver prioritizes safe driving, the advice unit will provide safe driving advice. If the driver prioritizes efficient driving, the advice unit can also provide efficient driving advice. If the driver prioritizes comfortable driving, the advice unit can also provide comfortable driving advice. By adjusting the advice content to match the driver's driving style and preferences, it becomes possible to provide the optimal advice for the driver. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's driving style and preference data into a generating AI and have the generating AI perform the adjustment of the advice content.

[0121] The advice unit can estimate the driver's emotions and adjust the timing of advice based on the estimated emotions. For example, if the driver is tense, the advice unit can provide advice to help them relax earlier. If the driver is relaxed, the advice unit can also provide advice at the normal timing. If the driver is in a hurry, the advice unit can also provide advice quickly. By adjusting the timing of advice based on the driver's emotions, it is possible to provide advice at the optimal time for the driver. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit can input driver emotion data into a generative AI and have the generative AI adjust the timing of advice.

[0122] The advice unit can provide specific advice while considering the driver's health condition. For example, if the driver is tired, the advice unit may advise them to take a break. If the driver's stress level is high, the advice unit may also provide advice on how to relax. If the driver's health condition is not good, the advice unit may also provide advice on safe driving. By considering the driver's health condition when providing advice, it becomes possible to provide the most appropriate advice for the driver. Some or all of the above processing in the advice unit may be performed using AI or not. For example, the advice unit may input the driver's health data into a generating AI and have the generating AI adjust the content of the advice.

[0123] The advice unit can customize the advice to suit the driver's language and culture. For example, the advice unit can provide advice in the driver's native language. The advice unit can also provide advice in language appropriate to the driver's culture. The advice unit can also provide advice based on the traffic rules of the driver's region. By customizing the advice to suit the driver's language and culture, it becomes possible to provide advice that is easy for the driver to understand. Some or all of the above processes in the advice unit may be performed using AI or not. For example, the advice unit can input the driver's language and cultural data into a generating AI and have the generating AI perform the customization of the advice.

[0124] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0125] The AI ​​assistant in the dashcam can monitor the driver's health and provide driving advice based on that health condition. For example, it can measure the driver's heart rate and blood pressure with sensors and advise them to take a break if an abnormality is detected. It can also advise the driver to take regular breaks if they are driving for a long time. Furthermore, if the driver's health is not good, it can advise them to refrain from driving. In this way, it can support safe driving by providing driving advice that takes the driver's health condition into consideration.

[0126] The AI ​​assistant in the dashcam can estimate the driver's emotions and select music and entertainment based on those emotions while driving. For example, if the driver is stressed, it will play relaxing music. If the driver is relaxed, it can play normal music. If the driver is tired, it can play uplifting music. This improves driving comfort by selecting music and entertainment based on the driver's emotions.

[0127] The AI ​​assistant in the dashcam can monitor the driver's gaze and provide driving advice based on their eye movements. For example, if the driver's gaze is diverting from the road, it can provide a warning. If the driver is frequently using their smartphone, it can advise them to reduce their smartphone use. If the driver's gaze is focused on the road ahead, it can advise them to continue driving safely. In this way, by providing driving advice based on the driver's gaze, it can support safe driving.

[0128] The AI ​​assistant in the dashcam can estimate the driver's emotions and adjust the lighting and climate control settings based on those estimates. For example, if the driver is stressed, the lighting can be softened and the climate control set to a comfortable temperature. If the driver is relaxed, the normal lighting and climate control settings can be maintained. If the driver is tired, the lighting can be brightened and the climate control set to cooler. This improves driving comfort by adjusting lighting and climate control settings based on the driver's emotions.

[0129] The AI ​​assistant in the dashcam can analyze the driver's driving style and provide advice to improve fuel efficiency based on that style. For example, if the driver frequently accelerates or brakes suddenly, it can advise them to drive more smoothly. If the driver frequently drives on highways, it can also advise them to maintain a constant speed. If the driver frequently drives in urban areas, it can advise them to consider traffic light timings. In this way, by providing advice to improve fuel efficiency based on driving style, it can support economical driving.

[0130] The AI ​​assistant in the dashcam can estimate the driver's emotions and suggest break times based on those emotions. For example, if the driver is feeling stressed, it may suggest taking a break earlier. If the driver is relaxed, it may suggest regular break times. If the driver is tired, it may suggest taking breaks more frequently. By suggesting break times based on the driver's emotions, it can reduce fatigue while driving and support safe driving.

[0131] The AI ​​assistant in the dashcam can analyze the driver's driving history and suggest insurance premium discounts based on that history. For example, if a driver has a long history of safe driving, it can suggest a discount to the insurance company. It can also suggest a discount if the driver has never had an accident. It can also suggest a discount if the driver performs regular maintenance. In this way, by suggesting insurance premium discounts based on driving history, the financial burden on the driver can be reduced.

[0132] The AI ​​assistant in the dashcam can estimate the driver's emotions and adjust the navigation route based on those emotions. For example, if the driver is stressed, it can suggest a route that avoids traffic. If the driver is relaxed, it can suggest a normal route. If the driver is in a hurry, it can suggest the shortest route. By adjusting the navigation route based on the driver's emotions, it can reduce stress while driving and support a more comfortable driving experience.

[0133] The AI ​​assistant in the dashcam can analyze the driver's driving patterns and suggest vehicle maintenance schedules based on those patterns. For example, if there are frequent sudden braking and acceleration, it may suggest replacing the brake pads sooner. If long-distance driving is common, it may suggest increasing the frequency of oil changes. If driving mostly in urban areas, it may suggest checking tire wear earlier. In this way, vehicle safety can be maintained by suggesting maintenance schedules based on driving patterns.

[0134] The AI ​​assistant in the dashcam can estimate the driver's emotions and support communication while driving based on those emotions. For example, if the driver is stressed, it can offer relaxing topics. If the driver is relaxed, it can offer normal topics. If the driver is tired, it can offer uplifting topics. By supporting communication based on the driver's emotions, it can reduce stress while driving and support a more comfortable driving experience.

[0135] The following briefly describes the processing flow for example form 2.

[0136] Step 1: The monitoring unit monitors the driving conditions in real time. The monitoring unit collects data such as vehicle speed, the movement of surrounding vehicles, and road conditions. The monitoring unit uses AI to analyze this data and detect dangerous situations or anomalies. Step 2: The detection unit detects dangerous situations or anomalies based on the data collected by the monitoring unit. The detection unit can detect situations such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. The detection unit can use AI to analyze these situations and detect dangerous situations or anomalies. Step 3: The warning unit issues a warning for dangerous situations or abnormalities detected by the detection unit. The warning unit warns the driver, for example, through voice or on-screen display. The warning unit can use AI to adjust the content and timing of the warning. Step 4: The recording storage unit automatically saves recordings in the event of an accident. For example, if the impact sensor is activated, the recording storage unit automatically saves video footage before and after the accident. The recording storage unit can use AI to adjust the recording storage conditions and storage period. Step 5: The emergency call unit makes an emergency call based on the video footage saved by the video storage unit. For example, the emergency call unit notifies emergency contacts of the occurrence of an accident. The emergency call unit can use AI to adjust the content and timing of the call. Step 6: The advice unit analyzes past driving data and provides advice on driving skills. For example, the advice unit analyzes the driver's driving patterns and habits and provides advice for safe driving. The advice unit can use AI to adjust the content and timing of the advice.

[0137] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0138] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0139] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0140] For example, the monitoring unit can monitor the vehicle's speed, the movement of surrounding vehicles, and road conditions in real time using the camera 42 and sensors of the smart device 14. The detection unit analyzes the data from the monitoring unit using the specific processing unit 290 of the data processing device 12 and detects dangerous situations or abnormalities such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. The warning unit uses the control unit 46A of the smart device 14 to warn the driver with voice and screen display. The recording and storage unit automatically saves video footage before and after an accident to the storage 50 of the smart device 14. The emergency notification unit notifies emergency contacts of the occurrence of an accident via the communication I / F 26 of the data processing device 12. The advice unit analyzes past driving data using the specific processing unit 290 of the data processing device 12 and provides advice on driving skills. The correspondence between each unit and the devices and control units is not limited to the example described above and can be modified in various ways.

[0141] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0142] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0143] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0144] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0145] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0146] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0147] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0148] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0149] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0150] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0151] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0152] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0153] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0154] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0155] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0156] For example, the monitoring unit can monitor the vehicle's speed, the movement of surrounding vehicles, and road conditions in real time using the camera 42 and sensors of the smart glasses 214. The detection unit analyzes the data from the monitoring unit using the specific processing unit 290 of the data processing device 12 and detects dangerous situations or abnormalities such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. The warning unit uses the control unit 46A of the smart glasses 214 to warn the driver with voice and screen display. The recording and storage unit automatically saves video footage before and after an accident to the storage 50 of the smart glasses 214. The emergency notification unit notifies emergency contacts of the accident via the communication I / F 26 of the data processing device 12. The advice unit analyzes past driving data using the specific processing unit 290 of the data processing device 12 and provides advice on driving skills. The correspondence between each unit and the devices and control units is not limited to the example described above and can be modified in various ways.

[0157] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0158] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0159] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0160] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0161] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0162] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0163] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0164] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0165] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0166] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0167] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0168] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0169] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0170] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0171] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0172] For example, the monitoring unit can monitor the vehicle's speed, the movement of surrounding vehicles, and road conditions in real time using the camera 42 and sensors of the headset terminal 314. The detection unit analyzes the data from the monitoring unit using the specific processing unit 290 of the data processing device 12 and detects dangerous situations or abnormalities such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. The warning unit uses the control unit 46A of the headset terminal 314 to warn the driver with voice and screen display. The recording and storage unit automatically saves video footage before and after an accident to the storage 50 of the headset terminal 314. The emergency notification unit notifies emergency contacts of the accident via the communication I / F 26 of the data processing device 12. The advice unit analyzes past driving data using the specific processing unit 290 of the data processing device 12 and provides advice on driving skills. The correspondence between each unit and the devices and control units is not limited to the example described above and can be modified in various ways.

[0173] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0174] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0175] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0176] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0177] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0178] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0179] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0180] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0181] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0182] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0183] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0184] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0185] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0186] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0187] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0188] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0189] For example, the monitoring unit can use the camera 42 and sensors of the robot 414 to monitor the vehicle's speed, the movement of surrounding vehicles, and road conditions in real time. The detection unit, using the specific processing unit 290 of the data processing device 12, analyzes the data from the monitoring unit and detects dangerous situations or abnormalities such as sudden braking, sudden steering, or a rapid decrease in the distance to the vehicle in front. The warning unit, using the control unit 46A of the robot 414, warns the driver with voice and on-screen displays. The recording and storage unit automatically saves video footage before and after an accident to the storage 50 of the robot 414. The emergency notification unit notifies emergency contacts of the accident via the communication I / F 26 of the data processing device 12. The advice unit, using the specific processing unit 290 of the data processing device 12, analyzes past driving data and provides advice on driving skills. The correspondence between each unit and the devices and control units is not limited to the example described above and can be modified in various ways.

[0190] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0191] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0192] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0193] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0194] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0195] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0196] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0197] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0198] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0199] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0200] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0201] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0202] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0203] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0204] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0205] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0206] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0207] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0208] (Note 1) A monitoring unit that monitors the driving conditions in real time, A detection unit that detects dangerous situations or anomalies based on data collected by the monitoring unit, A warning unit that issues a warning in response to a dangerous situation or abnormality detected by the aforementioned detection unit, A recording storage unit that automatically saves recordings in the event of an accident, An emergency call unit that makes an emergency call based on the video stored by the aforementioned recording and storage unit, It includes an advice unit that analyzes past driving data and provides advice on driving skills. A system characterized by the following features. (Note 2) The aforementioned monitoring unit, It collects data such as vehicle speed, the movement of surrounding vehicles, road conditions, and other information. The system described in Appendix 1, characterized by the features described herein. (Note 3) The detection unit, It detects dangerous situations or abnormalities such as sudden braking or steering, or a rapid decrease in the distance to the vehicle in front. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned warning unit is The system warns the driver via voice or on-screen display. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned recording storage unit is The system automatically saves video footage immediately before and after an accident if the impact sensor is activated. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned emergency call unit, Notify emergency contacts of the specific occurrence of an accident. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned advice section, Analyze the driver's driving patterns or habits and provide advice for safe driving. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned monitoring unit, The system estimates the driver's emotions and adjusts the monitoring accuracy based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned monitoring unit, The vehicle's internal environment is monitored, and specific data is collected to improve driver comfort. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned monitoring unit, It monitors the driver's gaze and facial direction to detect specific signs of distraction. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned monitoring unit, The system estimates the driver's emotions and determines specific data to monitor based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned monitoring unit, It monitors the vehicle's external environment and provides specific driving advice to the driver. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned monitoring unit, We monitor the vehicle's maintenance status and propose specific maintenance measures. The system described in Appendix 1, characterized by the features described herein. (Note 14) The detection unit, The system estimates the driver's emotions and adjusts specific detection criteria for dangerous situations based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The detection unit, It analyzes vehicle behavior and detects specific abnormal driving patterns. The system described in Appendix 1, characterized by the features described herein. (Note 16) The detection unit, It recognizes road signs or signals and provides specific driving instructions to the driver. The system described in Appendix 1, characterized by the features described herein. (Note 17) The detection unit, The system estimates the driver's emotions and adjusts the specific display method of the detection results based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 18) The detection unit, It communicates with other vehicles, predicts the movements of surrounding vehicles, and detects specific dangers. The system described in Appendix 1, characterized by the features described herein. (Note 19) The detection unit, It detects the movement of pedestrians or cyclists and issues a warning to the driver. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned warning unit is The system estimates the driver's emotions and adjusts the intensity or method of warnings based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned warning unit is When a warning is issued, the system selects a specific warning method by referring to the driver's past response data. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned warning unit is Customize warning messages to match the driver's language or culture. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned warning unit is The system estimates the driver's emotions and adjusts the specific timing of warnings based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned warning unit is When issuing a warning, the specific warning should take into account the driver's health condition. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned warning unit is Customize the warning content to suit the driver's preferences. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned recording storage unit is The system estimates the driver's emotions and adjusts the specific retention period of the recording based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned recording storage unit is During recording, the system prioritizes saving based on the specific importance of the footage. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned recording storage unit is Automatically back up recorded data to the cloud. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned recording storage unit is The system estimates the driver's emotions and adjusts the specific image quality of the recording based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned recording storage unit is When recording, the system will add and save the vehicle's specific location information. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned recording storage unit is The system analyzes recorded data and provides specific feedback to the driver. The system described in Appendix 1, characterized by the features described herein. (Note 32) The aforementioned emergency call unit, The system estimates the driver's emotions and adjusts the specific content of the emergency call based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 33) The aforementioned emergency call unit, When an emergency call is made, specific details of the accident will be automatically transmitted. The system described in Appendix 1, characterized by the features described herein. (Note 34) The aforementioned emergency call unit, During emergency calls, the driver's health condition is monitored and reflected in the specific details of the call. The system described in Appendix 1, characterized by the features described herein. (Note 35) The aforementioned emergency call unit, The system estimates the driver's emotions and adjusts the specific timing of emergency calls based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 36) The aforementioned emergency call unit, When making an emergency call, include specific vehicle maintenance information when sending the message. The system described in Appendix 1, characterized by the features described herein. (Note 37) The aforementioned emergency call unit, In the event of an emergency call, the driver's emergency contacts will be automatically notified. The system described in Appendix 1, characterized by the features described herein. (Note 38) The aforementioned advice section, The system estimates the driver's emotions and adjusts the specific content of the advice based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 39) The aforementioned advice section, When providing advice, we refer to the driver's past driving data to provide personalized and specific advice. The system described in Appendix 1, characterized by the features described herein. (Note 40) The aforementioned advice section, The advice can be adjusted to suit the driver's driving style or preferences. The system described in Appendix 1, characterized by the features described herein. (Note 41) The aforementioned advice section, The system estimates the driver's emotions and adjusts the timing of advice based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 42) The aforementioned advice section, When providing advice, we will take the driver's health condition into consideration and provide specific advice accordingly. The system described in Appendix 1, characterized by the features described herein. (Note 43) The aforementioned advice section, Customize the advice to suit the driver's language or culture. The system described in Appendix 1, characterized by the features described herein. [Explanation of Symbols]

[0209] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A monitoring unit that monitors the driving conditions in real time, A detection unit that detects dangerous situations or anomalies based on data collected by the monitoring unit, A warning unit that issues a warning in response to a dangerous situation or abnormality detected by the aforementioned detection unit, A recording storage unit that automatically saves recordings in the event of an accident, An emergency call unit that makes an emergency call based on the video stored by the aforementioned recording and storage unit, It includes an advice unit that analyzes past driving data and provides advice on driving skills. A system characterized by the following features.

2. The aforementioned monitoring unit, It collects data on vehicle speed, the movement of surrounding vehicles, and road conditions. The system according to feature 1.

3. The detection unit, The system detects sudden braking, sudden steering maneuvers, and a rapid decrease in the distance to the vehicle ahead as dangerous or abnormal situations. The system according to feature 1.

4. The aforementioned warning unit is The system warns the driver via voice or on-screen display. The system according to feature 1.

5. The aforementioned recording storage unit is The system automatically saves video footage immediately before and after an accident if the impact sensor is activated. The system according to feature 1.

6. The aforementioned emergency call unit, Notify emergency contacts of the accident. The system according to feature 1.

7. The aforementioned advice section, Analyze the driver's driving patterns or habits and provide advice for safe driving. The system according to feature 1.

8. The aforementioned monitoring unit, The system estimates the driver's emotions and adjusts the monitoring accuracy based on those estimated emotions. The system according to feature 1.

Images