system

Abstract

The system according to this embodiment aims to monitor in-vehicle behavior, respond immediately to abnormal behavior or emergencies, and analyze driving behavior in real time to evaluate safety. [Solution] The system according to the embodiment comprises a monitoring unit, a detection unit, a response unit, an analysis unit, and an evaluation unit. The monitoring unit monitors the behavior inside the vehicle. The detection unit detects abnormal behavior or emergencies based on the behavior monitored by the monitoring unit. The response unit immediately responds to the abnormal behavior or emergencies detected by the detection unit. The analysis unit analyzes the driver's driving behavior in real time. The evaluation unit evaluates safety based on the driving behavior analyzed by the analysis unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art, there are problems that it is difficult to quickly respond to abnormal behaviors and emergencies in the vehicle, and the safety evaluation is not performed in real time.

[0005] The system according to the embodiment aims to monitor the behavior in the vehicle, immediately respond to abnormal behaviors and emergencies, and analyze the driving behavior in real time to evaluate the safety.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a monitoring unit, a detection unit, a response unit, an analysis unit, and an evaluation unit. The monitoring unit monitors the behavior inside the vehicle. The detection unit detects abnormal behavior or emergencies based on the behavior monitored by the monitoring unit. The response unit immediately responds to the abnormal behavior or emergencies detected by the detection unit. The analysis unit analyzes the driver's driving behavior in real time. The evaluation unit evaluates safety based on the driving behavior analyzed by the analysis unit. [Effects of the Invention]

[0007] The system according to this embodiment can monitor in-vehicle behavior, respond immediately to abnormal behavior or emergencies, and analyze driving behavior in real time to evaluate safety. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10]This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F manages communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The rideshare safety system according to an embodiment of the present invention is a system that significantly improves the safety and peace of mind of rideshare users by utilizing AI technology. This rideshare safety system monitors the behavior inside the vehicle, detects abnormal behavior and emergencies, and responds immediately. The rideshare safety system also analyzes the driver's driving behavior in real time and evaluates safety. For example, the rideshare safety system monitors the behavior of passengers and drivers through cameras installed inside the vehicle. If a passenger behaves abnormally or an emergency occurs, the AI ​​detects this and responds immediately. This eliminates anxiety about troubles and emergencies inside the vehicle. Next, the rideshare safety system analyzes the driver's driving behavior in real time. For example, if a driver brakes suddenly or makes a sudden turn, the AI ​​detects this and evaluates safety. This eliminates anxiety about the driver's driving skills and safety. Furthermore, the rideshare safety system verifies the identity of drivers and passengers to prevent fraudulent use. For example, if a person other than the driver booked by the passenger is driving, the AI ​​detects this and issues a warning. This improves the safety of rideshare users. This system significantly improves the safety and peace of mind of rideshare users. For example, it eliminates anxieties about troubles and emergencies inside the vehicle, as well as concerns about the driver's driving skills and safety. Furthermore, users' anxieties are alleviated because they receive a quick and appropriate response when problems arise. This builds trust in the rideshare service and provides an environment where users can use rideshare with peace of mind. In short, rideshare safety systems can significantly improve the safety and peace of mind of rideshare users.

[0029] The rideshare safety system according to this embodiment comprises a monitoring unit, a detection unit, a response unit, an analysis unit, and an evaluation unit. The monitoring unit monitors the behavior inside the vehicle. The monitoring unit monitors the behavior of passengers and the driver, for example, through cameras installed inside the vehicle. The monitoring unit detects, for example, when a passenger behaves abnormally or when an emergency occurs, using cameras inside the vehicle. The monitoring unit can also detect, for example, when a passenger commits an act of violence or experiences a sudden illness, using cameras inside the vehicle. The monitoring unit can also detect, for example, when a passenger is involved in an accident or when a fire occurs, using cameras inside the vehicle. The detection unit detects abnormal behavior and emergencies based on the behavior monitored by the monitoring unit. The detection unit detects abnormal behavior and emergencies of passengers, for example, using AI. The detection unit can also detect acts of violence or sudden illness of passengers, for example, using AI. The detection unit can also detect accidents or fires involving passengers, for example, using AI. The response unit responds immediately to abnormal behavior and emergencies detected by the detection unit. The response unit, for example, uses AI to immediately respond to passengers' unusual behavior or emergencies. The response unit can also immediately respond to passengers' violent behavior or sudden illness using AI. The response unit can also immediately respond to passenger accidents or fires using AI. The analysis unit analyzes the driver's driving behavior in real time. The analysis unit analyzes the driver's driving behavior in real time, for example, using AI. The analysis unit can also detect, for example, when the driver brakes suddenly or makes a sudden steering turn using AI. The analysis unit can also detect, for example, when the driver accelerates suddenly or decelerates suddenly using AI. The evaluation unit evaluates safety based on the driving behavior analyzed by the analysis unit. The evaluation unit evaluates safety based on the driver's driving behavior, for example, using AI. The evaluation unit can also evaluate, for example, when the driver brakes suddenly or makes a sudden steering turn using AI. The evaluation unit can also evaluate, for example, when the driver accelerates suddenly or decelerates suddenly using AI.As a result, the rideshare safety system according to this embodiment can monitor in-vehicle behavior, detect and immediately respond to abnormal behavior or emergencies, analyze the driver's driving behavior in real time, and evaluate safety.

[0030] The monitoring unit monitors the behavior inside the vehicle. For example, the monitoring unit monitors the behavior of passengers and the driver through cameras installed inside the vehicle. Specifically, the in-vehicle cameras capture high-resolution video in real time and transmit it to a central monitoring system. The cameras are installed to cover the entire interior of the vehicle using wide-angle lenses and to minimize blind spots. Furthermore, by using infrared cameras in conjunction, clear images can be obtained even at night or in low-light environments. For example, the monitoring unit uses the in-vehicle cameras to detect if a passenger behaves abnormally or if an emergency occurs. Abnormal behavior includes passengers suddenly standing up or behaving aggressively towards other passengers or the driver. Emergencies include passengers suddenly collapsing or a fire breaking out inside the vehicle. The monitoring unit can also use the in-vehicle cameras to detect if a passenger engages in violent behavior or experiences a sudden illness. Violent behavior includes physical attacks such as punching and kicking, and illness includes passengers suddenly losing consciousness or experiencing severe vomiting. The monitoring unit can, for example, use in-vehicle cameras to detect incidents such as passenger accidents or fires. Accidents include collisions or sudden stops, while fires include smoke or obvious flames inside the vehicle. This allows the monitoring unit to constantly monitor the situation inside the vehicle and quickly detect abnormal behavior or emergencies.

[0031] The detection unit detects abnormal behavior and emergencies based on the behavior monitored by the monitoring unit. For example, the detection unit uses AI to detect abnormal passenger behavior and emergencies. Specifically, the AI ​​uses video analysis technology to analyze the movement patterns of passengers and drivers in real time. The AI ​​detects abnormal behavior by comparing it with normal behavior patterns that have been trained in advance. For example, if a passenger suddenly stands up or acts aggressively towards other passengers, the AI ​​will detect this as abnormal behavior. The detection unit can also use AI to detect violent behavior or sudden illness by passengers. The AI ​​analyzes changes in movement and facial expressions in the video to detect signs of violent behavior or illness early. For example, if a passenger suddenly collapses or their complexion changes rapidly, the AI ​​will detect this as illness. The detection unit can also use AI to detect accidents or fires involving passengers. The AI ​​analyzes in-vehicle video and sensor data to detect signs of collisions or fires early. For example, if the vehicle suddenly stops or smoke is generated inside the vehicle, the AI ​​will detect this as an accident or fire. This allows the detection unit to quickly and accurately detect abnormal behavior or emergencies based on data provided by the monitoring unit.

[0032] The response unit immediately responds to abnormal behavior or emergencies detected by the detection unit. For example, the response unit uses AI to immediately respond to abnormal behavior or emergencies involving passengers. Specifically, the AI ​​automatically executes appropriate actions based on pre-configured response protocols. For example, if a passenger engages in violent behavior, the AI ​​will sound an alarm in the vehicle and instruct the driver to make an emergency stop. It can also automatically notify the police or security company. The response unit can also immediately respond to violent behavior or sudden illness of passengers using AI. If illness is detected, the AI ​​will activate the emergency button in the vehicle to call an ambulance and guide the driver to the nearest hospital. The response unit can also immediately respond to accidents or fires involving passengers using AI. In the event of an accident, the AI ​​will instruct the vehicle to make an emergency stop and urge passengers to evacuate to a safe place. In the event of a fire, the AI ​​will activate the vehicle's fire suppression system and simultaneously notify the fire department. This allows the response unit to respond quickly and appropriately to detected abnormal behavior or emergencies, ensuring the safety of passengers and the driver.

[0033] The analysis unit analyzes the driver's driving behavior in real time. For example, the analysis unit uses AI to analyze the driver's driving behavior in real time. Specifically, the AI ​​analyzes vehicle sensor data and camera footage to monitor the driver's driving patterns. The AI ​​compares abnormal driving behaviors, such as sudden braking, sudden steering, sudden acceleration, and sudden deceleration, with pre-trained normal driving patterns to detect them. For example, if a driver suddenly brakes or makes a sudden steering turn, the AI ​​detects this as abnormal driving behavior. The analysis unit can also use AI to detect sudden acceleration or deceleration. The AI ​​analyzes data from the vehicle's acceleration and gyroscope sensors to detect sudden changes in speed. This allows the analysis unit to monitor the driver's driving behavior in real time and quickly detect abnormal driving patterns. Furthermore, the analysis unit can evaluate the driver's driving tendencies and risk factors based on past driving data. For example, if past data reveals that a particular driver frequently brakes suddenly, the analysis unit can provide guidance to that driver to improve their driving. This allows the analysis unit to continuously monitor the driver's driving behavior and contribute to promoting safe driving.

[0034] The evaluation unit assesses safety based on driving behavior analyzed by the analysis unit. For example, the evaluation unit uses AI to evaluate safety based on the driver's driving behavior. Specifically, the AI ​​calculates the driver's driving score based on driving data provided by the analysis unit. The driving score is evaluated based on the frequency and severity of abnormal driving behaviors such as sudden braking, sudden steering, sudden acceleration, and sudden deceleration. For example, if a driver suddenly brakes or makes a sudden steering turn, the AI ​​evaluates this and deducts points from the driving score. The evaluation unit can also use AI to evaluate sudden acceleration or sudden deceleration by the driver. The AI ​​analyzes data from the vehicle's acceleration and gyroscope sensors to evaluate sudden speed changes. This allows the evaluation unit to quantitatively assess safety based on the driver's driving behavior. Furthermore, the evaluation unit can provide feedback to the driver based on the driving score. For example, drivers with low driving scores can be provided with advice and training programs for safe driving. Drivers with high driving scores can be motivated by implementing reward programs. This allows the evaluation unit to continuously assess the driver's driving behavior, contributing to the promotion of safe driving and accident prevention.

[0035] The monitoring unit can monitor the behavior of passengers and the driver through cameras installed inside the vehicle. For example, the monitoring unit can use a front camera installed inside the vehicle to monitor the behavior of passengers and the driver. The monitoring unit can also use a side camera installed inside the vehicle to monitor the behavior of passengers and the driver. The monitoring unit can also use an infrared camera installed inside the vehicle to monitor the behavior of passengers and the driver. This allows for the early detection of abnormal behavior or emergencies by monitoring the behavior of passengers and the driver through cameras installed inside the vehicle. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input video data from cameras installed inside the vehicle into a generating AI and have the generating AI detect abnormal behavior or emergencies from the video data.

[0036] The monitoring unit can verify the identities of drivers and passengers to prevent fraudulent use. The monitoring unit can verify the identities of drivers and passengers using, for example, facial recognition technology. The monitoring unit can also verify the identities of drivers and passengers using, for example, ID card scanning. The monitoring unit can also verify the identities of drivers and passengers using, for example, fingerprint authentication. By verifying the identities of drivers and passengers, fraudulent use can be prevented. Some or all of the above processes in the monitoring unit may be performed using, for example, AI, or not using AI. For example, the monitoring unit can input facial data acquired using facial recognition technology into a generating AI and have the generating AI perform identity verification from the facial data.

[0037] The detection unit can detect abnormal behavior or emergencies. For example, the detection unit can use AI to detect abnormal behavior or emergencies involving passengers. For example, the detection unit can also use AI to detect violent behavior or sudden illness by passengers. For example, the detection unit can use AI to detect accidents or fires involving passengers. This enables a rapid response by detecting abnormal behavior or emergencies. Some or all of the above-described processes in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input passenger behavior data into a generating AI and have the generating AI perform the detection of abnormal behavior or emergencies from the behavior data.

[0038] The response unit can respond immediately to abnormal behavior or emergencies. For example, the response unit can use AI to respond immediately to abnormal behavior or emergencies involving passengers. For example, the response unit can use AI to respond immediately to violent behavior or sudden illness by passengers. For example, the response unit can use AI to respond immediately to accidents or fires involving passengers. This ensures user safety by responding immediately to abnormal behavior or emergencies. Some or all of the above-described processes in the response unit may be performed using AI, for example, or without AI. For example, the response unit can input data on abnormal behavior or emergencies into a generating AI and have the generating AI select a response method.

[0039] The analysis unit can analyze the driver's driving behavior in real time. The analysis unit can analyze the driver's driving behavior in real time, for example, using AI. The analysis unit can also detect, for example, when the driver applies the brakes suddenly or makes a sudden steering turn, using AI. The analysis unit can also detect, for example, when the driver accelerates suddenly or decelerates suddenly, using AI. This makes it possible to evaluate safety by analyzing the driver's driving behavior in real time. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the driver's driving data into a generating AI and have the generating AI perform the analysis of the driving behavior.

[0040] The evaluation unit can assess safety based on the analyzed driving behavior. The evaluation unit can, for example, use AI to assess safety based on the driver's driving behavior. The evaluation unit can also, for example, use AI to evaluate when the driver applies the brakes suddenly or makes a sudden steering turn. The evaluation unit can also, for example, use AI to evaluate when the driver accelerates suddenly or decelerates suddenly. By assessing safety based on the analyzed driving behavior, it is possible to alleviate the driver's concerns about their driving skills and safety. Some or all of the above processing in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input the driver's driving data into a generating AI and have the generating AI perform the safety assessment.

[0041] The monitoring unit can monitor environmental data such as temperature and humidity inside the vehicle and detect abnormal environmental changes. For example, if the temperature inside the vehicle rises rapidly, the AI ​​in the monitoring unit can detect the abnormality and automatically adjust the air conditioning. For example, if the humidity inside the vehicle becomes abnormally high, the AI ​​in the monitoring unit can detect the abnormality and prompt ventilation. For example, if the carbon dioxide concentration inside the vehicle becomes high, the AI ​​in the monitoring unit can detect the abnormality and instruct the system to open the windows. In this way, by monitoring environmental data inside the vehicle and detecting abnormal environmental changes, a comfortable environment inside the vehicle can be maintained. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input environmental data inside the vehicle into a generating AI and have the generating AI perform the detection of abnormal environmental changes.

[0042] The monitoring unit can predict abnormal behavior by referring to the passenger's past behavioral history during monitoring. For example, if a passenger has a history of abnormal behavior in the past, the AI ​​will focus its monitoring on that passenger's behavior. For example, if a passenger has a history of causing trouble in the past, the AI ​​can also monitor that passenger's behavior in detail. For example, if a passenger has a history of causing an emergency in the past, the AI ​​can predict that passenger's behavior and respond early. This improves the accuracy of predicting abnormal behavior by referring to the passenger's past behavioral history. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input the passenger's past behavioral history data into a generating AI and have the generating AI perform the prediction of abnormal behavior.

[0043] The monitoring unit can monitor audio data inside the vehicle and detect abnormal audio patterns. For example, if loud noises or shouting occur inside the vehicle, the AI ​​in the monitoring unit will detect the anomaly and respond immediately. For example, if an abnormal audio pattern (e.g., sounds of a fight) occurs inside the vehicle, the AI ​​in the monitoring unit can detect the anomaly and issue a warning. For example, if an abnormal sound (e.g., the sound of breaking glass) occurs inside the vehicle, the AI ​​in the monitoring unit can detect the anomaly and take emergency action. This enables a rapid response by monitoring audio data inside the vehicle and detecting abnormal audio patterns. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input audio data inside the vehicle into a generating AI and have the generating AI perform the detection of abnormal audio patterns.

[0044] The monitoring unit can simultaneously monitor the situation outside the vehicle during monitoring, enabling coordination between the inside and outside of the vehicle. For example, if an abnormal situation (e.g., an accident) occurs outside the vehicle, the AI ​​in the monitoring unit can issue a warning to the passengers inside the vehicle. For example, if the weather outside the vehicle changes suddenly, the AI ​​in the monitoring unit can adjust the environment inside the vehicle. For example, if an emergency (e.g., a fire) occurs outside the vehicle, the AI ​​in the monitoring unit can issue evacuation instructions to the passengers inside the vehicle. By simultaneously monitoring the situation outside the vehicle, coordination between the inside and outside of the vehicle is achieved, enabling a rapid response. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input external situation data into a generating AI and have the generating AI perform coordination between the inside and outside of the vehicle.

[0045] The detection unit can improve detection accuracy by referring to past abnormal behavior data when detecting abnormal behavior. For example, the detection unit can improve detection accuracy by having the AI ​​learn patterns of abnormal behavior based on past abnormal behavior data. The detection unit can also improve detection accuracy by having the AI ​​detect early signs of abnormal behavior by referring to past abnormal behavior data. The detection unit can also improve the accuracy of abnormal behavior prediction by having the AI ​​analyze past abnormal behavior data. As a result, the accuracy of abnormal behavior detection is improved by referring to past abnormal behavior data. Some or all of the above processing in the detection unit may be performed using AI, for example, or without using AI. For example, the detection unit can input past abnormal behavior data into a generating AI and have the generating AI perform abnormal behavior detection.

[0046] The detection unit can detect abnormal behavior by integrating data from multiple sensors inside the vehicle when detection occurs. For example, the detection unit can integrate camera data and audio data inside the vehicle, and the AI ​​can detect abnormal behavior. The detection unit can also integrate data from temperature and humidity sensors inside the vehicle, and the AI ​​can detect abnormal behavior. The detection unit can also integrate data from vibration sensors and position sensors inside the vehicle, and the AI ​​can detect abnormal behavior. By integrating data from multiple sensors inside the vehicle, the accuracy of abnormal behavior detection is improved. Some or all of the above processing in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input multiple sensor data into a generating AI and have the generating AI perform abnormal behavior detection.

[0047] The detection unit can improve detection accuracy by considering the lighting conditions inside the vehicle when detecting abnormal behavior. For example, if the vehicle is dark, the AI ​​can adjust the lighting to improve the accuracy of detecting abnormal behavior. The detection unit can also improve the accuracy of detecting abnormal behavior by adjusting the lighting if the vehicle is too bright. For example, if the lighting inside the vehicle fluctuates, the AI ​​can consider the lighting conditions to detect abnormal behavior. This improves the accuracy of detecting abnormal behavior by considering the lighting conditions inside the vehicle. Some or all of the above processing in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input vehicle lighting data into a generating AI and have the generating AI perform abnormal behavior detection.

[0048] The detection unit can detect abnormal behavior by combining audio and video data from inside the vehicle when detection occurs. For example, the detection unit combines audio and video data from inside the vehicle, and the AI ​​detects abnormal behavior. The detection unit can also integrate audio and video data from inside the vehicle, and the AI ​​can detect early signs of abnormal behavior. The detection unit can also analyze audio and video data from inside the vehicle, and the AI ​​can improve the accuracy of predicting abnormal behavior. As a result, the accuracy of detecting abnormal behavior is improved by combining audio and video data from inside the vehicle. Some or all of the above processing in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input audio and video data from inside the vehicle into a generating AI and have the generating AI perform abnormal behavior detection.

[0049] The response unit can select the optimal response by referring to past response history when responding to abnormal behavior or emergencies. For example, the response unit can use AI to select the optimal response method based on past response history. The response unit can also use AI to perform a rapid response by referring to past emergency response history. For example, the response unit can analyze past abnormal behavior response history and use AI to select the optimal response method. In this way, the optimal response method can be selected by referring to past response history. Some or all of the above processing in the response unit may be performed using AI, or not using AI. For example, the response unit can input past response history data into a generating AI and have the generating AI perform the selection of the optimal response method.

[0050] The response unit can select a response measure while considering the safety of other passengers in the vehicle. For example, if abnormal behavior occurs, the AI ​​in the response unit can select a response measure to ensure the safety of other passengers. For example, if an emergency occurs, the AI ​​in the response unit can also select a response measure that prioritizes the evacuation of other passengers. For example, if a problem occurs, the AI ​​in the response unit can also select a response measure that does not affect other passengers. In this way, overall safety can be ensured by considering the safety of other passengers. Some or all of the above processing in the response unit may be performed using AI, for example, or without AI. For example, the response unit can input safety data of other passengers into a generating AI and have the generating AI perform the selection of a response measure.

[0051] The response unit can take a rapid response to abnormal behavior or emergencies by utilizing in-vehicle voice alerts. For example, if abnormal behavior occurs, the AI ​​will issue a voice alert and take a rapid response. For example, if an emergency occurs, the AI ​​can issue a voice alert to prompt passengers to evacuate. For example, if trouble occurs, the AI ​​can issue a voice alert to alert passengers. This enables a rapid response by utilizing in-vehicle voice alerts. Some or all of the above processing in the response unit may be performed using AI, for example, or without AI. For example, the response unit can input data on abnormal behavior or emergencies into a generating AI and have the generating AI execute the activation of voice alerts.

[0052] The response unit can coordinate with external emergency services when responding to an emergency. For example, in the event of an emergency, the AI ​​can coordinate with the police and ambulance services to provide a rapid response. For example, if abnormal behavior occurs, the AI ​​can coordinate with a security company to provide a response. For example, if trouble occurs, the AI ​​can coordinate with a vehicle management company to provide a response. This allows for a rapid and appropriate response by coordinating with external emergency services. Some or all of the above-described processes in the response unit may be performed using AI, for example, or without AI. For example, the response unit can input emergency data into a generating AI and have the generating AI perform coordination with emergency services.

[0053] The analysis unit can improve the accuracy of its analysis by referring to past driving data when analyzing driving behavior. For example, the analysis unit can improve the accuracy of its analysis by having the AI ​​learn patterns of driving behavior based on past driving data. The analysis unit can also improve the accuracy of its analysis by having the AI ​​detect early signs of driving behavior by referring to past driving data. The analysis unit can also improve the accuracy of its AI in predicting driving behavior by analyzing past driving data. As a result, the accuracy of the analysis of driving behavior is improved by referring to past driving data. Some or all of the above processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input past driving data into a generating AI and have the generating AI perform the analysis of driving behavior.

[0054] The analysis unit can integrate vehicle state data during analysis to analyze driving behavior. For example, the analysis unit can integrate vehicle engine data and brake data, and the AI ​​can analyze driving behavior. The analysis unit can also integrate vehicle tire data and fuel data, and the AI ​​can analyze driving behavior. The analysis unit can also integrate vehicle speed data and position data, and the AI ​​can analyze driving behavior. By integrating vehicle state data, the accuracy of the driving behavior analysis is improved. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input vehicle state data into a generating AI and have the generating AI perform the driving behavior analysis.

[0055] The analysis unit can improve the accuracy of its analysis of driving behavior by utilizing in-vehicle audio data. For example, the analysis unit uses in-vehicle audio data to enable the AI ​​to analyze driving behavior. The analysis unit can also refer to in-vehicle audio data to enable the AI ​​to detect early signs of driving behavior. The analysis unit can also analyze in-vehicle audio data to improve the accuracy of its AI in predicting driving behavior. This improves the accuracy of driving behavior analysis by utilizing in-vehicle audio data. Some or all of the above-described processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input in-vehicle audio data into a generating AI and have the generating AI perform the analysis of driving behavior.

[0056] The analysis unit can analyze driving behavior by referring to external traffic data during analysis. For example, the analysis unit can use external traffic congestion data to have the AI ​​analyze driving behavior. The analysis unit can also use external traffic accident data to have the AI ​​detect early signs of driving behavior. The analysis unit can also use external traffic data to improve the accuracy of the AI's prediction of driving behavior. This improves the accuracy of the analysis of driving behavior by referring to external traffic data. Some or all of the above processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input external traffic data into a generating AI and have the generating AI perform the analysis of driving behavior.

[0057] The evaluation unit can improve evaluation accuracy by referring to past evaluation data when evaluating safety. For example, the evaluation unit can improve evaluation accuracy by having the AI ​​learn safety evaluation criteria based on past evaluation data. The evaluation unit can also improve safety indicators early by having the AI ​​refer to past evaluation data. The evaluation unit can also improve the accuracy of safety predictions by having the AI ​​analyze past evaluation data. As a result, the accuracy of safety evaluation is improved by referring to past evaluation data. Some or all of the above processes in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input past evaluation data into a generating AI and have the generating AI perform the safety evaluation.

[0058] The evaluation unit can integrate vehicle maintenance data during the evaluation process to assess safety. For example, the evaluation unit can integrate vehicle engine maintenance data and brake maintenance data, and the AI ​​can evaluate safety. The evaluation unit can also integrate vehicle tire maintenance data and fuel maintenance data, and the AI ​​can evaluate safety. The evaluation unit can also integrate vehicle speed maintenance data and position maintenance data, and the AI ​​can evaluate safety. This improves the accuracy of safety evaluation by integrating vehicle maintenance data. Some or all of the above processing in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input vehicle maintenance data into a generating AI and have the generating AI perform the safety evaluation.

[0059] The evaluation unit can improve the accuracy of safety evaluations by utilizing in-vehicle audio data. For example, the evaluation unit uses in-vehicle audio data to enable AI to evaluate safety. The evaluation unit can also use in-vehicle audio data to enable AI to detect signs of safety issues early. The evaluation unit can also analyze in-vehicle audio data to improve the accuracy of safety predictions. As a result, the accuracy of safety evaluations is improved by utilizing in-vehicle audio data. Some or all of the above processes in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input in-vehicle audio data into a generating AI and have the generating AI perform the safety evaluation.

[0060] The evaluation unit can assess safety by referring to external traffic data during the evaluation process. For example, the evaluation unit uses external traffic congestion data to enable the AI ​​to evaluate safety. The evaluation unit can also refer to external traffic accident data to enable the AI ​​to detect signs of safety issues early. The evaluation unit can also analyze external traffic data to improve the accuracy of safety predictions. This improves the accuracy of safety evaluation by referring to external traffic data. Some or all of the above processes in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input external traffic data into a generating AI and have the generating AI perform the safety evaluation.

[0061] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0062] The rideshare safety system can monitor the air quality inside the vehicle and detect abnormal conditions. For example, if the carbon dioxide concentration inside the vehicle becomes high, the AI ​​will prompt ventilation. The AI ​​can also issue a warning if the concentration of harmful substances inside the vehicle rises. If the oxygen concentration inside the vehicle drops, the AI ​​can instruct the driver to open the windows. This allows the system to monitor the air quality inside the vehicle and detect abnormal conditions, thereby protecting the health of passengers.

[0063] The rideshare safety system can monitor the vehicle's interior lighting and detect abnormal lighting conditions. For example, if the interior is too dark, the AI ​​can automatically adjust the lighting. If the interior is too bright, the AI ​​can also adjust the lighting to provide a comfortable brightness. If the interior lighting changes suddenly, the AI ​​can detect the anomaly and take appropriate action. In this way, by monitoring the interior lighting and detecting abnormal lighting conditions, a comfortable in-vehicle environment can be maintained.

[0064] The rideshare safety system monitors audio data inside the vehicle and can detect abnormal sound patterns. For example, if loud noises or shouting occur inside the vehicle, the AI ​​can detect the anomaly and respond immediately. If abnormal sound patterns (e.g., sounds of a fight) occur inside the vehicle, the AI ​​can detect the anomaly and issue a warning. If abnormal sounds (e.g., the sound of breaking glass) occur inside the vehicle, the AI ​​can detect the anomaly and take emergency action. This allows for a rapid response by monitoring audio data inside the vehicle and detecting abnormal sound patterns.

[0065] The rideshare safety system monitors in-vehicle vibration data and can detect abnormal vibrations. For example, if abnormal vibrations occur inside the vehicle, the AI ​​can detect the anomaly and take immediate action. If vibrations suddenly increase inside the vehicle, the AI ​​can detect the anomaly and issue a warning. If vibrations occur continuously inside the vehicle, the AI ​​can detect the anomaly and take appropriate action. This allows for a rapid response by monitoring in-vehicle vibration data and detecting abnormal vibrations.

[0066] The rideshare safety system can monitor the seating arrangement inside the vehicle and detect abnormal arrangements. For example, if seats are not positioned correctly, the AI ​​will detect the anomaly and issue a warning. If seats move suddenly, the AI ​​can also detect the anomaly and take appropriate action. If seats are positioned in a way that threatens passenger safety, the AI ​​can detect the anomaly and take immediate action. In this way, by monitoring the seating arrangement inside the vehicle and detecting abnormal arrangements, passenger safety can be ensured.

[0067] The following briefly describes the processing flow for example form 1.

[0068] Step 1: The monitoring unit monitors the behavior inside the vehicle. The monitoring unit monitors the behavior of passengers and the driver, for example, through cameras installed inside the vehicle. The monitoring unit can use the in-vehicle cameras to detect if a passenger behaves abnormally or if an emergency occurs. The monitoring unit can also use the in-vehicle cameras to detect if a passenger engages in violent behavior or experiences a sudden illness. The monitoring unit can also use the in-vehicle cameras to detect if a passenger is involved in an accident or if a fire occurs. Step 2: The detection unit detects abnormal behavior or emergencies based on the behavior monitored by the monitoring unit. The detection unit can, for example, use AI to detect abnormal behavior or emergencies involving passengers. The detection unit can also, for example, use AI to detect violent behavior or sudden illness by passengers. The detection unit can also, for example, use AI to detect accidents or fires involving passengers. Step 3: The response unit immediately responds to abnormal behavior or emergencies detected by the detection unit. The response unit can, for example, use AI to immediately respond to abnormal behavior or emergencies involving passengers. The response unit can, for example, use AI to immediately respond to violent behavior or sudden illness by passengers. The response unit can, for example, use AI to immediately respond to accidents or fires involving passengers. Step 4: The analysis unit analyzes the driver's driving behavior in real time. The analysis unit analyzes the driver's driving behavior in real time, for example, using AI. The analysis unit can also use AI to detect, for example, when the driver applies the brakes suddenly or makes a sudden steering turn. The analysis unit can also use AI to detect, for example, when the driver accelerates suddenly or decelerates suddenly. Step 5: The evaluation unit evaluates safety based on the driving behavior analyzed by the analysis unit. The evaluation unit evaluates safety based on the driver's driving behavior, for example, using AI. The evaluation unit can also evaluate, for example, situations where the driver brakes suddenly or makes a sudden steering turn, using AI. The evaluation unit can also evaluate, for example, situations where the driver accelerates suddenly or decelerates suddenly, using AI.

[0069] (Example of form 2) The rideshare safety system according to an embodiment of the present invention is a system that significantly improves the safety and peace of mind of rideshare users by utilizing AI technology. This rideshare safety system monitors the behavior inside the vehicle, detects abnormal behavior and emergencies, and responds immediately. The rideshare safety system also analyzes the driver's driving behavior in real time and evaluates safety. For example, the rideshare safety system monitors the behavior of passengers and drivers through cameras installed inside the vehicle. If a passenger behaves abnormally or an emergency occurs, the AI ​​detects this and responds immediately. This eliminates anxiety about troubles and emergencies inside the vehicle. Next, the rideshare safety system analyzes the driver's driving behavior in real time. For example, if a driver brakes suddenly or makes a sudden turn, the AI ​​detects this and evaluates safety. This eliminates anxiety about the driver's driving skills and safety. Furthermore, the rideshare safety system verifies the identity of drivers and passengers to prevent fraudulent use. For example, if a person other than the driver booked by the passenger is driving, the AI ​​detects this and issues a warning. This improves the safety of rideshare users. This system significantly improves the safety and peace of mind of rideshare users. For example, it eliminates anxieties about troubles and emergencies inside the vehicle, as well as concerns about the driver's driving skills and safety. Furthermore, users' anxieties are alleviated because they receive a quick and appropriate response when problems arise. This builds trust in the rideshare service and provides an environment where users can use rideshare with peace of mind. In short, rideshare safety systems can significantly improve the safety and peace of mind of rideshare users.

[0070] The rideshare safety system according to this embodiment comprises a monitoring unit, a detection unit, a response unit, an analysis unit, and an evaluation unit. The monitoring unit monitors the behavior inside the vehicle. The monitoring unit monitors the behavior of passengers and the driver, for example, through cameras installed inside the vehicle. The monitoring unit detects, for example, when a passenger behaves abnormally or when an emergency occurs, using cameras inside the vehicle. The monitoring unit can also detect, for example, when a passenger commits an act of violence or experiences a sudden illness, using cameras inside the vehicle. The monitoring unit can also detect, for example, when a passenger is involved in an accident or when a fire occurs, using cameras inside the vehicle. The detection unit detects abnormal behavior and emergencies based on the behavior monitored by the monitoring unit. The detection unit detects abnormal behavior and emergencies of passengers, for example, using AI. The detection unit can also detect acts of violence or sudden illness of passengers, for example, using AI. The detection unit can also detect accidents or fires involving passengers, for example, using AI. The response unit responds immediately to abnormal behavior and emergencies detected by the detection unit. The response unit, for example, uses AI to immediately respond to passengers' unusual behavior or emergencies. The response unit can also immediately respond to passengers' violent behavior or sudden illness using AI. The response unit can also immediately respond to passenger accidents or fires using AI. The analysis unit analyzes the driver's driving behavior in real time. The analysis unit analyzes the driver's driving behavior in real time, for example, using AI. The analysis unit can also detect, for example, when the driver brakes suddenly or makes a sudden steering turn using AI. The analysis unit can also detect, for example, when the driver accelerates suddenly or decelerates suddenly using AI. The evaluation unit evaluates safety based on the driving behavior analyzed by the analysis unit. The evaluation unit evaluates safety based on the driver's driving behavior, for example, using AI. The evaluation unit can also evaluate, for example, when the driver brakes suddenly or makes a sudden steering turn using AI. The evaluation unit can also evaluate, for example, when the driver accelerates suddenly or decelerates suddenly using AI.As a result, the rideshare safety system according to this embodiment can monitor in-vehicle behavior, detect and immediately respond to abnormal behavior or emergencies, analyze the driver's driving behavior in real time, and evaluate safety.

[0071] The monitoring unit monitors the behavior inside the vehicle. For example, the monitoring unit monitors the behavior of passengers and the driver through cameras installed inside the vehicle. Specifically, the in-vehicle cameras capture high-resolution video in real time and transmit it to a central monitoring system. The cameras are installed to cover the entire interior of the vehicle using wide-angle lenses and to minimize blind spots. Furthermore, by using infrared cameras in conjunction, clear images can be obtained even at night or in low-light environments. For example, the monitoring unit uses the in-vehicle cameras to detect if a passenger behaves abnormally or if an emergency occurs. Abnormal behavior includes passengers suddenly standing up or behaving aggressively towards other passengers or the driver. Emergencies include passengers suddenly collapsing or a fire breaking out inside the vehicle. The monitoring unit can also use the in-vehicle cameras to detect if a passenger engages in violent behavior or experiences a sudden illness. Violent behavior includes physical attacks such as punching and kicking, and illness includes passengers suddenly losing consciousness or experiencing severe vomiting. The monitoring unit can, for example, use in-vehicle cameras to detect incidents such as passenger accidents or fires. Accidents include collisions or sudden stops, while fires include smoke or obvious flames inside the vehicle. This allows the monitoring unit to constantly monitor the situation inside the vehicle and quickly detect abnormal behavior or emergencies.

[0072] The detection unit detects abnormal behavior and emergencies based on the behavior monitored by the monitoring unit. For example, the detection unit uses AI to detect abnormal passenger behavior and emergencies. Specifically, the AI ​​uses video analysis technology to analyze the movement patterns of passengers and drivers in real time. The AI ​​detects abnormal behavior by comparing it with normal behavior patterns that have been trained in advance. For example, if a passenger suddenly stands up or acts aggressively towards other passengers, the AI ​​will detect this as abnormal behavior. The detection unit can also use AI to detect violent behavior or sudden illness by passengers. The AI ​​analyzes changes in movement and facial expressions in the video to detect signs of violent behavior or illness early. For example, if a passenger suddenly collapses or their complexion changes rapidly, the AI ​​will detect this as illness. The detection unit can also use AI to detect accidents or fires involving passengers. The AI ​​analyzes in-vehicle video and sensor data to detect signs of collisions or fires early. For example, if the vehicle suddenly stops or smoke is generated inside the vehicle, the AI ​​will detect this as an accident or fire. This allows the detection unit to quickly and accurately detect abnormal behavior or emergencies based on data provided by the monitoring unit.

[0073] The response unit immediately responds to abnormal behavior or emergencies detected by the detection unit. For example, the response unit uses AI to immediately respond to abnormal behavior or emergencies involving passengers. Specifically, the AI ​​automatically executes appropriate actions based on pre-configured response protocols. For example, if a passenger engages in violent behavior, the AI ​​will sound an alarm in the vehicle and instruct the driver to make an emergency stop. It can also automatically notify the police or security company. The response unit can also immediately respond to violent behavior or sudden illness of passengers using AI. If illness is detected, the AI ​​will activate the emergency button in the vehicle to call an ambulance and guide the driver to the nearest hospital. The response unit can also immediately respond to accidents or fires involving passengers using AI. In the event of an accident, the AI ​​will instruct the vehicle to make an emergency stop and urge passengers to evacuate to a safe place. In the event of a fire, the AI ​​will activate the vehicle's fire suppression system and simultaneously notify the fire department. This allows the response unit to respond quickly and appropriately to detected abnormal behavior or emergencies, ensuring the safety of passengers and the driver.

[0074] The analysis unit analyzes the driver's driving behavior in real time. For example, the analysis unit uses AI to analyze the driver's driving behavior in real time. Specifically, the AI ​​analyzes vehicle sensor data and camera footage to monitor the driver's driving patterns. The AI ​​compares abnormal driving behaviors, such as sudden braking, sudden steering, sudden acceleration, and sudden deceleration, with pre-trained normal driving patterns to detect them. For example, if a driver suddenly brakes or makes a sudden steering turn, the AI ​​detects this as abnormal driving behavior. The analysis unit can also use AI to detect sudden acceleration or deceleration. The AI ​​analyzes data from the vehicle's acceleration and gyroscope sensors to detect sudden changes in speed. This allows the analysis unit to monitor the driver's driving behavior in real time and quickly detect abnormal driving patterns. Furthermore, the analysis unit can evaluate the driver's driving tendencies and risk factors based on past driving data. For example, if past data reveals that a particular driver frequently brakes suddenly, the analysis unit can provide guidance to that driver to improve their driving. This allows the analysis unit to continuously monitor the driver's driving behavior and contribute to promoting safe driving.

[0075] The evaluation unit assesses safety based on driving behavior analyzed by the analysis unit. For example, the evaluation unit uses AI to evaluate safety based on the driver's driving behavior. Specifically, the AI ​​calculates the driver's driving score based on driving data provided by the analysis unit. The driving score is evaluated based on the frequency and severity of abnormal driving behaviors such as sudden braking, sudden steering, sudden acceleration, and sudden deceleration. For example, if a driver suddenly brakes or makes a sudden steering turn, the AI ​​evaluates this and deducts points from the driving score. The evaluation unit can also use AI to evaluate sudden acceleration or sudden deceleration by the driver. The AI ​​analyzes data from the vehicle's acceleration and gyroscope sensors to evaluate sudden speed changes. This allows the evaluation unit to quantitatively assess safety based on the driver's driving behavior. Furthermore, the evaluation unit can provide feedback to the driver based on the driving score. For example, drivers with low driving scores can be provided with advice and training programs for safe driving. Drivers with high driving scores can be motivated by implementing reward programs. This allows the evaluation unit to continuously assess the driver's driving behavior, contributing to the promotion of safe driving and accident prevention.

[0076] The monitoring unit can monitor the behavior of passengers and the driver through cameras installed inside the vehicle. For example, the monitoring unit can use a front camera installed inside the vehicle to monitor the behavior of passengers and the driver. The monitoring unit can also use a side camera installed inside the vehicle to monitor the behavior of passengers and the driver. The monitoring unit can also use an infrared camera installed inside the vehicle to monitor the behavior of passengers and the driver. This allows for the early detection of abnormal behavior or emergencies by monitoring the behavior of passengers and the driver through cameras installed inside the vehicle. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input video data from cameras installed inside the vehicle into a generating AI and have the generating AI detect abnormal behavior or emergencies from the video data.

[0077] The monitoring unit can verify the identities of drivers and passengers to prevent fraudulent use. The monitoring unit can verify the identities of drivers and passengers using, for example, facial recognition technology. The monitoring unit can also verify the identities of drivers and passengers using, for example, ID card scanning. The monitoring unit can also verify the identities of drivers and passengers using, for example, fingerprint authentication. By verifying the identities of drivers and passengers, fraudulent use can be prevented. Some or all of the above processes in the monitoring unit may be performed using, for example, AI, or not using AI. For example, the monitoring unit can input facial data acquired using facial recognition technology into a generating AI and have the generating AI perform identity verification from the facial data.

[0078] The detection unit can detect abnormal behavior or emergencies. For example, the detection unit can use AI to detect abnormal behavior or emergencies involving passengers. For example, the detection unit can also use AI to detect violent behavior or sudden illness by passengers. For example, the detection unit can use AI to detect accidents or fires involving passengers. This enables a rapid response by detecting abnormal behavior or emergencies. Some or all of the above-described processes in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input passenger behavior data into a generating AI and have the generating AI perform the detection of abnormal behavior or emergencies from the behavior data.

[0079] The response unit can respond immediately to abnormal behavior or emergencies. For example, the response unit can use AI to respond immediately to abnormal behavior or emergencies involving passengers. For example, the response unit can use AI to respond immediately to violent behavior or sudden illness by passengers. For example, the response unit can use AI to respond immediately to accidents or fires involving passengers. This ensures user safety by responding immediately to abnormal behavior or emergencies. Some or all of the above-described processes in the response unit may be performed using AI, for example, or without AI. For example, the response unit can input data on abnormal behavior or emergencies into a generating AI and have the generating AI select a response method.

[0080] The analysis unit can analyze the driver's driving behavior in real time. The analysis unit can analyze the driver's driving behavior in real time, for example, using AI. The analysis unit can also detect, for example, when the driver applies the brakes suddenly or makes a sudden steering turn, using AI. The analysis unit can also detect, for example, when the driver accelerates suddenly or decelerates suddenly, using AI. This makes it possible to evaluate safety by analyzing the driver's driving behavior in real time. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the driver's driving data into a generating AI and have the generating AI perform the analysis of the driving behavior.

[0081] The evaluation unit can assess safety based on the analyzed driving behavior. The evaluation unit can, for example, use AI to assess safety based on the driver's driving behavior. The evaluation unit can also, for example, use AI to evaluate when the driver applies the brakes suddenly or makes a sudden steering turn. The evaluation unit can also, for example, use AI to evaluate when the driver accelerates suddenly or decelerates suddenly. By assessing safety based on the analyzed driving behavior, it is possible to alleviate the driver's concerns about their driving skills and safety. Some or all of the above processing in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input the driver's driving data into a generating AI and have the generating AI perform the safety assessment.

[0082] The monitoring unit can estimate passengers' emotions and adjust its monitoring focus based on the estimated emotions. For example, if a passenger is feeling anxious, the AI ​​intensifies monitoring of that passenger, aiming for early detection of abnormal behavior. For example, if a passenger is relaxed, the AI ​​can return monitoring of that passenger to a normal level and intensify monitoring of other passengers. For example, if a passenger is excited, the AI ​​can closely monitor that passenger's behavior and detect early signs of abnormal behavior. This allows for early detection of abnormal behavior by adjusting monitoring focus based on passengers' emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the monitoring unit may be performed using AI or not using AI. For example, the monitoring unit can input passenger facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0083] The monitoring unit can monitor environmental data such as temperature and humidity inside the vehicle and detect abnormal environmental changes. For example, if the temperature inside the vehicle rises rapidly, the AI ​​in the monitoring unit can detect the abnormality and automatically adjust the air conditioning. For example, if the humidity inside the vehicle becomes abnormally high, the AI ​​in the monitoring unit can detect the abnormality and prompt ventilation. For example, if the carbon dioxide concentration inside the vehicle becomes high, the AI ​​in the monitoring unit can detect the abnormality and instruct the system to open the windows. In this way, by monitoring environmental data inside the vehicle and detecting abnormal environmental changes, a comfortable environment inside the vehicle can be maintained. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input environmental data inside the vehicle into a generating AI and have the generating AI perform the detection of abnormal environmental changes.

[0084] The monitoring unit can predict abnormal behavior by referring to the passenger's past behavioral history during monitoring. For example, if a passenger has a history of abnormal behavior in the past, the AI ​​will focus its monitoring on that passenger's behavior. For example, if a passenger has a history of causing trouble in the past, the AI ​​can also monitor that passenger's behavior in detail. For example, if a passenger has a history of causing an emergency in the past, the AI ​​can predict that passenger's behavior and respond early. This improves the accuracy of predicting abnormal behavior by referring to the passenger's past behavioral history. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input the passenger's past behavioral history data into a generating AI and have the generating AI perform the prediction of abnormal behavior.

[0085] The monitoring unit can estimate passengers' emotions and adjust the monitoring frequency based on the estimated emotions. For example, if a passenger is feeling anxious, the AI ​​can increase the monitoring frequency for that passenger to aim for early detection of abnormal behavior. For example, if a passenger is relaxed, the AI ​​can return the monitoring frequency for that passenger to a normal level and increase the monitoring frequency for other passengers. For example, if a passenger is excited, the AI ​​can closely monitor that passenger's behavior to detect signs of abnormal behavior early. This allows for early detection of abnormal behavior by adjusting the monitoring frequency based on passengers' emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the monitoring unit may be performed using AI or not using AI. For example, the monitoring unit can input passenger facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0086] The monitoring unit can monitor audio data inside the vehicle and detect abnormal audio patterns. For example, if loud noises or shouting occur inside the vehicle, the AI ​​in the monitoring unit will detect the anomaly and respond immediately. For example, if an abnormal audio pattern (e.g., sounds of a fight) occurs inside the vehicle, the AI ​​in the monitoring unit can detect the anomaly and issue a warning. For example, if an abnormal sound (e.g., the sound of breaking glass) occurs inside the vehicle, the AI ​​in the monitoring unit can detect the anomaly and take emergency action. This enables a rapid response by monitoring audio data inside the vehicle and detecting abnormal audio patterns. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input audio data inside the vehicle into a generating AI and have the generating AI perform the detection of abnormal audio patterns.

[0087] The monitoring unit can simultaneously monitor the situation outside the vehicle during monitoring, enabling coordination between the inside and outside of the vehicle. For example, if an abnormal situation (e.g., an accident) occurs outside the vehicle, the AI ​​in the monitoring unit can issue a warning to the passengers inside the vehicle. For example, if the weather outside the vehicle changes suddenly, the AI ​​in the monitoring unit can adjust the environment inside the vehicle. For example, if an emergency (e.g., a fire) occurs outside the vehicle, the AI ​​in the monitoring unit can issue evacuation instructions to the passengers inside the vehicle. By simultaneously monitoring the situation outside the vehicle, coordination between the inside and outside of the vehicle is achieved, enabling a rapid response. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input external situation data into a generating AI and have the generating AI perform coordination between the inside and outside of the vehicle.

[0088] The detection unit can estimate the passenger's emotions and adjust the abnormal behavior detection criteria based on the estimated emotions. For example, if a passenger is feeling anxious, the AI ​​can tighten the abnormal behavior detection criteria to detect abnormalities early. For example, if a passenger is relaxed, the AI ​​can return the abnormal behavior detection criteria to a normal level and tighten the detection criteria for other passengers. For example, if a passenger is excited, the AI ​​can set the abnormal behavior detection criteria in detail to detect abnormalities early. This allows for early detection of abnormal behavior by adjusting the abnormal behavior detection criteria based on the passenger's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the detection unit may be performed using AI or not using AI. For example, the detection unit can input passenger facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0089] The detection unit can improve detection accuracy by referring to past abnormal behavior data when detecting abnormal behavior. For example, the detection unit can improve detection accuracy by having the AI ​​learn patterns of abnormal behavior based on past abnormal behavior data. The detection unit can also improve detection accuracy by having the AI ​​detect early signs of abnormal behavior by referring to past abnormal behavior data. The detection unit can also improve the accuracy of abnormal behavior prediction by having the AI ​​analyze past abnormal behavior data. As a result, the accuracy of abnormal behavior detection is improved by referring to past abnormal behavior data. Some or all of the above processing in the detection unit may be performed using AI, for example, or without using AI. For example, the detection unit can input past abnormal behavior data into a generating AI and have the generating AI perform abnormal behavior detection.

[0090] The detection unit can detect abnormal behavior by integrating data from multiple sensors inside the vehicle when detection occurs. For example, the detection unit can integrate camera data and audio data inside the vehicle, and the AI ​​can detect abnormal behavior. The detection unit can also integrate data from temperature and humidity sensors inside the vehicle, and the AI ​​can detect abnormal behavior. The detection unit can also integrate data from vibration sensors and position sensors inside the vehicle, and the AI ​​can detect abnormal behavior. By integrating data from multiple sensors inside the vehicle, the accuracy of abnormal behavior detection is improved. Some or all of the above processing in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input multiple sensor data into a generating AI and have the generating AI perform abnormal behavior detection.

[0091] The detection unit can estimate passengers' emotions and prioritize abnormal behavior based on the estimated emotions. For example, if a passenger is feeling anxious, the AI ​​in the detection unit will prioritize detecting that passenger's abnormal behavior. For example, if a passenger is relaxed, the AI ​​in the detection unit can also prioritize detecting abnormal behavior in other passengers. For example, if a passenger is excited, the AI ​​in the detection unit can also detect that passenger's abnormal behavior in detail. This allows for the priority detection of important abnormal behavior by prioritizing abnormal behavior based on passengers' emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input passenger facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0092] The detection unit can improve detection accuracy by considering the lighting conditions inside the vehicle when detecting abnormal behavior. For example, if the vehicle is dark, the AI ​​can adjust the lighting to improve the accuracy of detecting abnormal behavior. The detection unit can also improve the accuracy of detecting abnormal behavior by adjusting the lighting if the vehicle is too bright. For example, if the lighting inside the vehicle fluctuates, the AI ​​can consider the lighting conditions to detect abnormal behavior. This improves the accuracy of detecting abnormal behavior by considering the lighting conditions inside the vehicle. Some or all of the above processing in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input vehicle lighting data into a generating AI and have the generating AI perform abnormal behavior detection.

[0093] The detection unit can detect abnormal behavior by combining audio and video data from inside the vehicle when detection occurs. For example, the detection unit combines audio and video data from inside the vehicle, and the AI ​​detects abnormal behavior. The detection unit can also integrate audio and video data from inside the vehicle, and the AI ​​can detect early signs of abnormal behavior. The detection unit can also analyze audio and video data from inside the vehicle, and the AI ​​can improve the accuracy of predicting abnormal behavior. As a result, the accuracy of detecting abnormal behavior is improved by combining audio and video data from inside the vehicle. Some or all of the above processing in the detection unit may be performed using AI, for example, or without AI. For example, the detection unit can input audio and video data from inside the vehicle into a generating AI and have the generating AI perform abnormal behavior detection.

[0094] The response unit can estimate the passenger's emotions and adjust its response based on the estimated emotions. For example, if a passenger is feeling anxious, the AI ​​can provide reassuring support. If a passenger is relaxed, the AI ​​can provide normal support and enhance its response to other passengers. If a passenger is agitated, the AI ​​can provide calming support. By adjusting the response based on the passenger's emotions, a more appropriate response becomes possible. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the response unit may be performed using AI or not. For example, the response unit can input passenger facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0095] The response unit can select the optimal response by referring to past response history when responding to abnormal behavior or emergencies. For example, the response unit can use AI to select the optimal response method based on past response history. The response unit can also use AI to perform a rapid response by referring to past emergency response history. For example, the response unit can analyze past abnormal behavior response history and use AI to select the optimal response method. In this way, the optimal response method can be selected by referring to past response history. Some or all of the above processing in the response unit may be performed using AI, or not using AI. For example, the response unit can input past response history data into a generating AI and have the generating AI perform the selection of the optimal response method.

[0096] The response unit can select a response measure while considering the safety of other passengers in the vehicle. For example, if abnormal behavior occurs, the AI ​​in the response unit can select a response measure to ensure the safety of other passengers. For example, if an emergency occurs, the AI ​​in the response unit can also select a response measure that prioritizes the evacuation of other passengers. For example, if a problem occurs, the AI ​​in the response unit can also select a response measure that does not affect other passengers. In this way, overall safety can be ensured by considering the safety of other passengers. Some or all of the above processing in the response unit may be performed using AI, for example, or without AI. For example, the response unit can input safety data of other passengers into a generating AI and have the generating AI perform the selection of a response measure.

[0097] The response unit can estimate the passenger's emotions and determine the priority of responses based on the estimated emotions. For example, if a passenger is feeling anxious, the AI ​​will prioritize that passenger's response. For example, if a passenger is relaxed, the AI ​​may prioritize other passengers' responses. For example, if a passenger is agitated, the AI ​​may prioritize that passenger's response and take actions to calm them down. In this way, by determining the priority of responses based on the passenger's emotions, important responses can be prioritized. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the response unit may be performed using AI, or not using AI. For example, the response unit can input passenger facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0098] The response unit can take a rapid response to abnormal behavior or emergencies by utilizing in-vehicle voice alerts. For example, if abnormal behavior occurs, the AI ​​will issue a voice alert and take a rapid response. For example, if an emergency occurs, the AI ​​can issue a voice alert to prompt passengers to evacuate. For example, if trouble occurs, the AI ​​can issue a voice alert to alert passengers. This enables a rapid response by utilizing in-vehicle voice alerts. Some or all of the above processing in the response unit may be performed using AI, for example, or without AI. For example, the response unit can input data on abnormal behavior or emergencies into a generating AI and have the generating AI execute the activation of voice alerts.

[0099] The response unit can coordinate with external emergency services when responding to an emergency. For example, in the event of an emergency, the AI ​​can coordinate with the police and ambulance services to provide a rapid response. For example, if abnormal behavior occurs, the AI ​​can coordinate with a security company to provide a response. For example, if trouble occurs, the AI ​​can coordinate with a vehicle management company to provide a response. This allows for a rapid and appropriate response by coordinating with external emergency services. Some or all of the above-described processes in the response unit may be performed using AI, for example, or without AI. For example, the response unit can input emergency data into a generating AI and have the generating AI perform coordination with emergency services.

[0100] The analysis unit can estimate the driver's emotions and adjust the analysis criteria for driving behavior based on the estimated emotions. For example, if the driver is tense, the AI ​​can tighten the analysis criteria for driving behavior to detect abnormalities early. For example, if the driver is relaxed, the AI ​​can return the analysis criteria for driving behavior to a normal level and tighten the analysis criteria for other drivers. For example, if the driver is excited, the AI ​​can set the analysis criteria for driving behavior in detail to detect abnormalities early. This allows for early detection of abnormal behavior by adjusting the analysis criteria for driving behavior based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the analysis unit may be performed using AI or not using AI. For example, the analysis unit can input driver facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0101] The analysis unit can improve the accuracy of its analysis by referring to past driving data when analyzing driving behavior. For example, the analysis unit can improve the accuracy of its analysis by having the AI ​​learn patterns of driving behavior based on past driving data. The analysis unit can also improve the accuracy of its analysis by having the AI ​​detect early signs of driving behavior by referring to past driving data. The analysis unit can also improve the accuracy of its AI in predicting driving behavior by analyzing past driving data. As a result, the accuracy of the analysis of driving behavior is improved by referring to past driving data. Some or all of the above processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input past driving data into a generating AI and have the generating AI perform the analysis of driving behavior.

[0102] The analysis unit can integrate vehicle state data during analysis to analyze driving behavior. For example, the analysis unit can integrate vehicle engine data and brake data, and the AI ​​can analyze driving behavior. The analysis unit can also integrate vehicle tire data and fuel data, and the AI ​​can analyze driving behavior. The analysis unit can also integrate vehicle speed data and position data, and the AI ​​can analyze driving behavior. By integrating vehicle state data, the accuracy of the driving behavior analysis is improved. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input vehicle state data into a generating AI and have the generating AI perform the driving behavior analysis.

[0103] The analysis unit can estimate the driver's emotions and determine the priority of analysis based on the estimated emotions. For example, if a driver is tense, the AI ​​in the analysis unit will prioritize analyzing that driver's driving behavior. For example, if a driver is relaxed, the AI ​​in the analysis unit can also prioritize analyzing the driving behavior of other drivers. For example, if a driver is excited, the AI ​​in the analysis unit can also analyze that driver's driving behavior in detail. This allows for the prioritization of important driving behaviors by determining the priority of analysis based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the analysis unit may be performed using AI or not using AI. For example, the analysis unit can input driver facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0104] The analysis unit can improve the accuracy of its analysis of driving behavior by utilizing in-vehicle audio data. For example, the analysis unit uses in-vehicle audio data to enable the AI ​​to analyze driving behavior. The analysis unit can also refer to in-vehicle audio data to enable the AI ​​to detect early signs of driving behavior. The analysis unit can also analyze in-vehicle audio data to improve the accuracy of its AI in predicting driving behavior. This improves the accuracy of driving behavior analysis by utilizing in-vehicle audio data. Some or all of the above-described processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input in-vehicle audio data into a generating AI and have the generating AI perform the analysis of driving behavior.

[0105] The analysis unit can analyze driving behavior by referring to external traffic data during analysis. For example, the analysis unit can use external traffic congestion data to have the AI ​​analyze driving behavior. The analysis unit can also use external traffic accident data to have the AI ​​detect early signs of driving behavior. The analysis unit can also use external traffic data to improve the accuracy of the AI's prediction of driving behavior. This improves the accuracy of the analysis of driving behavior by referring to external traffic data. Some or all of the above processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input external traffic data into a generating AI and have the generating AI perform the analysis of driving behavior.

[0106] The evaluation unit can estimate the driver's emotions and adjust the safety evaluation criteria based on the estimated emotions. For example, if the driver is tense, the AI ​​can tighten the safety evaluation criteria to detect abnormalities early. For example, if the driver is relaxed, the AI ​​can return the safety evaluation criteria to a normal level and tighten the evaluation criteria for other drivers. For example, if the driver is excited, the AI ​​can set the safety evaluation criteria in detail to detect abnormalities early. This allows for early detection of abnormal behavior by adjusting the safety evaluation criteria based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the evaluation unit may be performed using AI or not using AI. For example, the evaluation unit can input driver facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0107] The evaluation unit can improve evaluation accuracy by referring to past evaluation data when evaluating safety. For example, the evaluation unit can improve evaluation accuracy by having the AI ​​learn safety evaluation criteria based on past evaluation data. The evaluation unit can also improve safety indicators early by having the AI ​​refer to past evaluation data. The evaluation unit can also improve the accuracy of safety predictions by having the AI ​​analyze past evaluation data. As a result, the accuracy of safety evaluation is improved by referring to past evaluation data. Some or all of the above processes in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input past evaluation data into a generating AI and have the generating AI perform the safety evaluation.

[0108] The evaluation unit can integrate vehicle maintenance data during the evaluation process to assess safety. For example, the evaluation unit can integrate vehicle engine maintenance data and brake maintenance data, and the AI ​​can evaluate safety. The evaluation unit can also integrate vehicle tire maintenance data and fuel maintenance data, and the AI ​​can evaluate safety. The evaluation unit can also integrate vehicle speed maintenance data and position maintenance data, and the AI ​​can evaluate safety. This improves the accuracy of safety evaluation by integrating vehicle maintenance data. Some or all of the above processing in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input vehicle maintenance data into a generating AI and have the generating AI perform the safety evaluation.

[0109] The evaluation unit can estimate the driver's emotions and determine the evaluation priority based on the estimated emotions. For example, if the driver is tense, the AI ​​will prioritize that driver's safety evaluation. The evaluation unit can also prioritize the safety evaluation of other drivers if the driver is relaxed. For example, if the driver is excited, the AI ​​will prioritize that driver's safety evaluation and may even evaluate it in detail. This allows important safety evaluations to be prioritized by determining the evaluation priority based on the driver's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the evaluation unit may be performed using AI or not using AI. For example, the evaluation unit can input driver facial expression data into a generative AI and have the generative AI perform emotion estimation.

[0110] The evaluation unit can improve the accuracy of safety evaluations by utilizing in-vehicle audio data. For example, the evaluation unit uses in-vehicle audio data to enable AI to evaluate safety. The evaluation unit can also use in-vehicle audio data to enable AI to detect signs of safety issues early. The evaluation unit can also analyze in-vehicle audio data to improve the accuracy of safety predictions. As a result, the accuracy of safety evaluations is improved by utilizing in-vehicle audio data. Some or all of the above processes in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input in-vehicle audio data into a generating AI and have the generating AI perform the safety evaluation.

[0111] The evaluation unit can assess safety by referring to external traffic data during the evaluation process. For example, the evaluation unit uses external traffic congestion data to enable the AI ​​to evaluate safety. The evaluation unit can also refer to external traffic accident data to enable the AI ​​to detect signs of safety issues early. The evaluation unit can also analyze external traffic data to improve the accuracy of safety predictions. This improves the accuracy of safety evaluation by referring to external traffic data. Some or all of the above processes in the evaluation unit may be performed using AI, for example, or without AI. For example, the evaluation unit can input external traffic data into a generating AI and have the generating AI perform the safety evaluation.

[0112] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0113] Ridesharing safety systems can estimate passengers' emotions and adjust in-car entertainment based on those estimates. For example, if a passenger is feeling anxious, the AI ​​can provide relaxing music or videos. If a passenger is relaxed, the AI ​​can select content to maintain that state. If a passenger is agitated, the AI ​​can provide content to alleviate that agitation. This allows for a more comfortable in-car environment by adjusting entertainment based on passengers' emotions.

[0114] The rideshare safety system can monitor the air quality inside the vehicle and detect abnormal conditions. For example, if the carbon dioxide concentration inside the vehicle becomes high, the AI ​​will prompt ventilation. The AI ​​can also issue a warning if the concentration of harmful substances inside the vehicle rises. If the oxygen concentration inside the vehicle drops, the AI ​​can instruct the driver to open the windows. This allows the system to monitor the air quality inside the vehicle and detect abnormal conditions, thereby protecting the health of passengers.

[0115] Ridesharing safety systems can estimate passengers' emotions and adjust seat comfort based on those emotions. For example, if a passenger is feeling anxious, the AI ​​can adjust the seat recline to provide a relaxing position. If the passenger is relaxed, the AI ​​can also adjust the seat temperature and massage functions to maintain that state. If the passenger is agitated, the AI ​​can use the seat's vibration function to help them relax. In this way, a comfortable ride can be provided by adjusting seat comfort based on the passenger's emotions.

[0116] The rideshare safety system can monitor the vehicle's interior lighting and detect abnormal lighting conditions. For example, if the interior is too dark, the AI ​​can automatically adjust the lighting. If the interior is too bright, the AI ​​can also adjust the lighting to provide a comfortable brightness. If the interior lighting changes suddenly, the AI ​​can detect the anomaly and take appropriate action. In this way, by monitoring the interior lighting and detecting abnormal lighting conditions, a comfortable in-vehicle environment can be maintained.

[0117] Ridesharing safety systems can estimate passengers' emotions and adjust the vehicle's temperature based on those estimates. For example, if a passenger is feeling anxious, the AI ​​can adjust the temperature to create a relaxing environment. If the passenger is relaxed, the AI ​​can adjust the temperature to maintain that state. If the passenger is agitated, the AI ​​can lower the temperature to help them calm down. This allows for a comfortable in-car environment by adjusting the temperature based on passengers' emotions.

[0118] The rideshare safety system monitors audio data inside the vehicle and can detect abnormal sound patterns. For example, if loud noises or shouting occur inside the vehicle, the AI ​​can detect the anomaly and respond immediately. If abnormal sound patterns (e.g., sounds of a fight) occur inside the vehicle, the AI ​​can detect the anomaly and issue a warning. If abnormal sounds (e.g., the sound of breaking glass) occur inside the vehicle, the AI ​​can detect the anomaly and take emergency action. This allows for a rapid response by monitoring audio data inside the vehicle and detecting abnormal sound patterns.

[0119] The rideshare safety system can estimate passengers' emotions and adjust the in-car scent based on those emotions. For example, if a passenger is feeling anxious, the AI ​​can provide a relaxing scent. If the passenger is relaxed, the AI ​​can select a scent to maintain that state. If the passenger is agitated, the AI ​​can provide a scent to calm that agitation. In this way, a comfortable in-car environment can be provided by adjusting the scent based on the passenger's emotions.

[0120] The rideshare safety system monitors in-vehicle vibration data and can detect abnormal vibrations. For example, if abnormal vibrations occur inside the vehicle, the AI ​​can detect the anomaly and take immediate action. If vibrations suddenly increase inside the vehicle, the AI ​​can detect the anomaly and issue a warning. If vibrations occur continuously inside the vehicle, the AI ​​can detect the anomaly and take appropriate action. This allows for a rapid response by monitoring in-vehicle vibration data and detecting abnormal vibrations.

[0121] The rideshare safety system can estimate passengers' emotions and adjust the in-car volume based on those emotions. For example, if a passenger is feeling anxious, the AI ​​can lower the volume to create a relaxing environment. If the passenger is relaxed, the AI ​​can adjust the volume to maintain that state. If the passenger is agitated, the AI ​​can lower the volume to help them calm down. This allows for a comfortable in-car environment by adjusting the volume based on passengers' emotions.

[0122] The rideshare safety system can monitor the seating arrangement inside the vehicle and detect abnormal arrangements. For example, if seats are not positioned correctly, the AI ​​will detect the anomaly and issue a warning. If seats move suddenly, the AI ​​can also detect the anomaly and take appropriate action. If seats are positioned in a way that threatens passenger safety, the AI ​​can detect the anomaly and take immediate action. In this way, by monitoring the seating arrangement inside the vehicle and detecting abnormal arrangements, passenger safety can be ensured.

[0123] The following briefly describes the processing flow for example form 2.

[0124] Step 1: The monitoring unit monitors the behavior inside the vehicle. The monitoring unit monitors the behavior of passengers and the driver, for example, through cameras installed inside the vehicle. The monitoring unit can use the in-vehicle cameras to detect if a passenger behaves abnormally or if an emergency occurs. The monitoring unit can also use the in-vehicle cameras to detect if a passenger engages in violent behavior or experiences a sudden illness. The monitoring unit can also use the in-vehicle cameras to detect if a passenger is involved in an accident or if a fire occurs. Step 2: The detection unit detects abnormal behavior or emergencies based on the behavior monitored by the monitoring unit. The detection unit can, for example, use AI to detect abnormal behavior or emergencies involving passengers. The detection unit can also, for example, use AI to detect violent behavior or sudden illness by passengers. The detection unit can also, for example, use AI to detect accidents or fires involving passengers. Step 3: The response unit immediately responds to abnormal behavior or emergencies detected by the detection unit. The response unit can, for example, use AI to immediately respond to abnormal behavior or emergencies involving passengers. The response unit can, for example, use AI to immediately respond to violent behavior or sudden illness by passengers. The response unit can, for example, use AI to immediately respond to accidents or fires involving passengers. Step 4: The analysis unit analyzes the driver's driving behavior in real time. The analysis unit analyzes the driver's driving behavior in real time, for example, using AI. The analysis unit can also use AI to detect, for example, when the driver applies the brakes suddenly or makes a sudden steering turn. The analysis unit can also use AI to detect, for example, when the driver accelerates suddenly or decelerates suddenly. Step 5: The evaluation unit evaluates safety based on the driving behavior analyzed by the analysis unit. The evaluation unit evaluates safety based on the driver's driving behavior, for example, using AI. The evaluation unit can also evaluate, for example, situations where the driver brakes suddenly or makes a sudden steering turn, using AI. The evaluation unit can also evaluate, for example, situations where the driver accelerates suddenly or decelerates suddenly, using AI.

[0125] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0126] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0127] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0128] Each of the multiple elements described above, including the monitoring unit, detection unit, response unit, analysis unit, and evaluation unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the monitoring unit monitors in-vehicle behavior using the camera 42 of the smart device 14. The detection unit is implemented in the specific processing unit 290 of the data processing unit 12 and uses AI to detect abnormal behavior or emergencies. The response unit is implemented in the specific processing unit 46A of the smart device 14 and responds immediately to detected abnormal behavior or emergencies. The analysis unit is implemented in the specific processing unit 290 of the data processing unit 12 and analyzes the driver's driving behavior in real time. The evaluation unit is implemented in the specific processing unit 290 of the data processing unit 12 and evaluates safety based on the analyzed driving behavior. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0129] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0130] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0131] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0132] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0133] The microphone 238 receives voice commands and other instructions from the user by receiving voice signals. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0134] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0135] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0136] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0137] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0138] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0139] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0140] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0141] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0142] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0143] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0144] Each of the multiple elements described above, including the monitoring unit, detection unit, response unit, analysis unit, and evaluation unit, is implemented, for example, in at least one of the smart glasses 214 and the data processing unit 12. For example, the monitoring unit monitors in-vehicle behavior using the camera 42 of the smart glasses 214. The detection unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12, and uses AI to detect abnormal behavior or emergencies. The response unit is implemented, for example, by the control unit 46A of the smart glasses 214, and responds immediately to detected abnormal behavior or emergencies. The analysis unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12, and analyzes the driver's driving behavior in real time. The evaluation unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12, and evaluates safety based on the analyzed driving behavior. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0145] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0146] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0147] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0148] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0149] The microphone 238 receives voice commands and other instructions from the user by receiving voice signals. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0150] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0151] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0152] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0153] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0154] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0155] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0156] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0157] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0158] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0159] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0160] Each of the multiple elements described above, including the monitoring unit, detection unit, response unit, analysis unit, and evaluation unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the monitoring unit monitors in-vehicle behavior using the camera 42 of the headset terminal 314. The detection unit is implemented in the specific processing unit 290 of the data processing unit 12 and uses AI to detect abnormal behavior or emergencies. The response unit is implemented in the specific processing unit 46A of the headset terminal 314 and responds immediately to detected abnormal behavior or emergencies. The analysis unit is implemented in the specific processing unit 290 of the data processing unit 12 and analyzes the driver's driving behavior in real time. The evaluation unit is implemented in the specific processing unit 290 of the data processing unit 12 and evaluates safety based on the analyzed driving behavior. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0161] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0162] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0163] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0164] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0165] The microphone 238 receives voice commands and other instructions from the user by receiving voice signals. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0166] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0167] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0168] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0169] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0170] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0171] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0172] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0173] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0174] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0175] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0176] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0177] Each of the multiple elements described above, including the monitoring unit, detection unit, response unit, analysis unit, and evaluation unit, is implemented in at least one of the following: the robot 414 and the data processing unit 12. For example, the monitoring unit monitors the behavior inside the vehicle using the camera 42 of the robot 414. The detection unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12, and uses AI to detect abnormal behavior or emergencies. The response unit is implemented, for example, by the control unit 46A of the robot 414, and responds immediately to the detected abnormal behavior or emergencies. The analysis unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12, and analyzes the driver's driving behavior in real time. The evaluation unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12, and evaluates safety based on the analyzed driving behavior. The correspondence between each unit and the devices and control units is not limited to the examples described above, and various modifications are possible.

[0178] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0179] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0180] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0181] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0182] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0183] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0184] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0185] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0186] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0187] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0188] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0189] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0190] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0191] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0192] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0193] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0194] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0195] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0196] (Note 1) A monitoring unit that monitors the behavior inside the vehicle, A detection unit that detects abnormal behavior or emergencies based on the behavior monitored by the aforementioned monitoring unit, A response unit that immediately responds to abnormal behavior or emergencies detected by the aforementioned detection unit, The analysis unit analyzes the driver's driving behavior in real time, The system includes an evaluation unit that evaluates safety based on the driving behavior analyzed by the analysis unit. A system characterized by the following features. (Note 2) The aforementioned monitoring unit, The behavior of passengers and the driver is monitored through cameras installed inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned monitoring unit, We verify the identities of both drivers and passengers to prevent fraudulent use. The system described in Appendix 1, characterized by the features described herein. (Note 4) The detection unit, Detects abnormal behavior and emergencies. The system described in Appendix 1, characterized by the features described herein. (Note 5) The corresponding part is, Respond immediately to unusual behavior or emergencies. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned analysis unit is Analyze the driver's driving behavior in real time. The system described in Appendix 1, characterized by the features described herein. (Note 7) The evaluation unit, Safety is evaluated based on the analyzed driving behavior. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned monitoring unit, The system estimates passenger emotions and adjusts monitoring focus based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned monitoring unit, It monitors environmental data such as temperature and humidity inside the vehicle and detects abnormal environmental changes. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned monitoring unit, During monitoring, abnormal behavior is predicted by referring to the passenger's past behavioral history. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned monitoring unit, The system estimates passengers' emotions and adjusts the frequency of monitoring based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned monitoring unit, It monitors audio data inside the vehicle and detects abnormal audio patterns. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned monitoring unit, During monitoring, the system also monitors the situation outside the vehicle and coordinates the monitoring of both the inside and outside of the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 14) The detection unit, The system estimates passengers' emotions and adjusts the criteria for detecting abnormal behavior based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The detection unit, When detecting abnormal behavior, past abnormal behavior data is referenced to improve detection accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 16) The detection unit, Upon detection, data from multiple sensors inside the vehicle is integrated to detect abnormal behavior. The system described in Appendix 1, characterized by the features described herein. (Note 17) The detection unit, The system estimates passengers' emotions and prioritizes abnormal behaviors based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 18) The detection unit, When detecting abnormal behavior, the system improves detection accuracy by taking into account the lighting conditions inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 19) The detection unit, Upon detection, the system combines audio and video data from inside the vehicle to detect abnormal behavior. The system described in Appendix 1, characterized by the features described herein. (Note 20) The corresponding part is, The system estimates the passenger's emotions and adjusts its response based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The corresponding part is, When responding to abnormal behavior or emergencies, the system selects the optimal response by referring to past response records. The system described in Appendix 1, characterized by the features described herein. (Note 22) The corresponding part is, When responding, the appropriate course of action will be selected considering the safety of other passengers on board. The system described in Appendix 1, characterized by the features described herein. (Note 23) The corresponding part is, The system estimates the passenger's emotions and determines the priority of responses based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The corresponding part is, In response to abnormal behavior or emergencies, use in-vehicle voice alerts to ensure a rapid response. The system described in Appendix 1, characterized by the features described herein. (Note 25) The corresponding part is, During a response, the response will be coordinated with external emergency services. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned analysis unit is The system estimates the driver's emotions and adjusts the analysis criteria for driving behavior based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned analysis unit is When analyzing driving behavior, past driving data is used to improve the accuracy of the analysis. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned analysis unit is During analysis, vehicle condition data is integrated to analyze driving behavior. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned analysis unit is The system estimates the driver's emotions and prioritizes analysis based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned analysis unit is Utilizing in-vehicle audio data during driving behavior analysis improves the accuracy of the analysis. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned analysis unit is During the analysis, external traffic data is referenced to analyze driving behavior. The system described in Appendix 1, characterized by the features described herein. (Note 32) The evaluation unit, The system estimates the driver's emotions and adjusts safety evaluation criteria based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 33) The evaluation unit, When evaluating safety, past evaluation data is used to improve evaluation accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 34) The evaluation unit, During the evaluation, vehicle maintenance data is integrated to assess safety. The system described in Appendix 1, characterized by the features described herein. (Note 35) The evaluation unit, The system estimates the driver's emotions and determines evaluation priorities based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 36) The evaluation unit, Utilizing in-vehicle audio data during safety evaluations improves evaluation accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 37) The evaluation unit, During the evaluation, safety is assessed by referring to external traffic data. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0197] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A monitoring unit that monitors the behavior inside the vehicle, A detection unit that detects abnormal behavior or emergencies based on the behavior monitored by the aforementioned monitoring unit, A response unit that immediately responds to abnormal behavior or emergencies detected by the aforementioned detection unit, The analysis unit analyzes the driver's driving behavior in real time, The system includes an evaluation unit that evaluates safety based on the driving behavior analyzed by the analysis unit. A system characterized by the following features.

2. The aforementioned monitoring unit, The behavior of passengers and the driver is monitored through cameras installed inside the vehicle. The system according to feature 1.

3. The aforementioned monitoring unit, We verify the identities of both drivers and passengers to prevent fraudulent use. The system according to feature 1.

4. The detection unit is Detects abnormal behavior and emergencies. The system according to feature 1.

5. The corresponding part is, Respond immediately to unusual behavior or emergencies. The system according to feature 1.

6. The aforementioned analysis unit is Analyze the driver's driving behavior in real time. The system according to feature 1.

7. The evaluation unit, Safety is evaluated based on the analyzed driving behavior. The system according to feature 1.

8. The aforementioned monitoring unit, The system estimates passenger emotions and adjusts monitoring focus based on those estimated emotions. The system according to feature 1.

9. The aforementioned monitoring unit, It monitors environmental data such as temperature and humidity inside the vehicle and detects abnormal environmental changes. The system according to feature 1.

10. The aforementioned monitoring unit, During monitoring, abnormal behavior is predicted by referring to the passenger's past behavioral history. The system according to feature 1.

Images