system

Abstract

The system according to this embodiment aims to prevent accidents in which children are left unattended in a vehicle. [Solution] The system according to the embodiment comprises a location information acquisition unit, a camera unit, a thermometer unit, a management system, and a smartphone. The location information acquisition unit acquires location information. The camera unit manages the number of people inside the vehicle based on the location information acquired by the location information acquisition unit. The thermometer unit monitors the temperature inside the vehicle based on the number of people information managed by the camera unit. The management system detects abnormalities based on the temperature information monitored by the thermometer unit. The smartphone sends an alert based on the abnormality information detected by the management system.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, monitoring for preventing accidents where children are left unattended in the vehicle is not sufficiently performed, and there is room for improvement.

[0005] The system according to the embodiment aims to prevent accidents where children are left unattended in the vehicle.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a location information acquisition unit, a camera unit, a thermometer unit, a management system, and a smartphone. The location information acquisition unit acquires location information. The camera unit manages the number of people inside the vehicle based on the location information acquired by the location information acquisition unit. The thermometer unit monitors the temperature inside the vehicle based on the number of people information managed by the camera unit. The management system detects abnormalities based on the temperature information monitored by the thermometer unit. The smartphone sends an alert based on the abnormality information detected by the management system. [Effects of the Invention]

[0007] The system according to this embodiment can prevent accidents in which children are left unattended in a vehicle. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10]This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F manages communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The in-vehicle safety management system according to an embodiment of the present invention is a system for preventing accidents in which children are left unattended in a vehicle. This system uses GPS, a vehicle camera, an in-vehicle thermometer, and in-vehicle motion detection in conjunction with a smartphone to monitor whether a person is left inside the vehicle. For example, the location information acquisition unit shares location information with the camera unit to identify the location of people inside the vehicle. Next, the camera unit performs facial recognition to manage the number of people inside the vehicle. Furthermore, the thermometer unit constantly monitors the temperature inside the vehicle and transmits information to the management system if a high temperature condition persists. The management system integrates this data and sends an alert to the smartphone if an abnormality is detected. For example, an alert is issued if location information remains inside the vehicle after the engine has been turned off, if there is a discrepancy in the number of people entering or leaving the vehicle, or if the temperature inside the vehicle remains above 40 degrees Celsius for more than 10 minutes. This mechanism makes it possible to prevent accidents in which children are left unattended in a vehicle. Thus, the in-vehicle safety management system can prevent accidents in which children are left unattended in a vehicle.

[0029] The in-vehicle safety management system according to this embodiment comprises a location information acquisition unit, a camera unit, a thermometer unit, a management system, and a smartphone. The location information acquisition unit identifies the location of people inside the vehicle. The location information acquisition unit acquires location information inside the vehicle using, for example, GPS. The location information acquisition unit can also acquire location information using Wi-Fi or Bluetooth®. Furthermore, the location information acquisition unit can also work in conjunction with in-vehicle motion detection to acquire location information when motion is detected. For example, when motion is detected inside the vehicle, the location information acquisition unit immediately acquires location information to understand the situation inside the vehicle. The camera unit manages the number of people inside the vehicle using facial recognition. The camera unit counts the number of people inside the vehicle using, for example, a facial recognition algorithm. Furthermore, the camera unit can work in conjunction with motion detection to perform facial recognition according to the type of motion. For example, when motion is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. The thermometer unit constantly monitors the temperature inside the vehicle and transmits information to the management system if a high temperature condition persists. The thermometer unit measures the temperature inside the vehicle using, for example, a temperature sensor. The thermometer unit can also monitor humidity information and integrate it with the temperature information to detect anomalies. For example, the thermometer unit monitors humidity information inside the vehicle and detects anomalies based on the combination of temperature and humidity. The management system sends an alert to the smartphone when an anomaly is detected. The management system integrates temperature information and occupancy information to detect anomalies. The management system can also analyze motion information and audio information to improve the accuracy of anomaly detection. For example, the management system analyzes motion information inside the vehicle and detects an anomaly when a specific motion is detected. The smartphone receives the alert and notifies the user. The smartphone displays the alert using, for example, a notification sound or message content. The smartphone can also estimate the user's emotions and adjust the alert display method based on the estimated emotions. For example, if the user is stressed, the smartphone displays a simple and highly visible alert. As a result, the in-vehicle safety management system according to this embodiment can prevent accidents where children are left behind in the vehicle.

[0030] The location information acquisition unit identifies the location of people inside the vehicle. For example, the unit can acquire location information inside the vehicle using GPS. GPS receives signals from satellites and can pinpoint the location of each device inside the vehicle with high accuracy. The location information acquisition unit can also acquire location information using Wi-Fi or Bluetooth. Wi-Fi determines the device's location by measuring the distance to an access point installed inside the vehicle. Bluetooth determines the location based on the communication range with beacons placed inside the vehicle. Furthermore, the location information acquisition unit can work in conjunction with motion detection inside the vehicle to acquire location information when motion is detected. For example, if a motion sensor installed inside the vehicle detects a person's movement, the location information acquisition unit immediately identifies that person's location and understands the situation inside the vehicle. This allows the location information acquisition unit to know the location of people inside the vehicle in real time, enabling rapid response in emergencies. Furthermore, the location information acquisition unit can improve the accuracy of location information by combining multiple technologies. For example, combining GPS, Wi-Fi, and Bluetooth makes it possible to acquire highly accurate location information both indoors and outdoors. This allows the location information acquisition unit to perform safety management inside the vehicle more effectively.

[0031] The camera unit manages the number of people inside the vehicle using facial recognition. For example, the camera unit counts the number of people inside the vehicle using a facial recognition algorithm. The facial recognition algorithm detects human faces from the video captured by the camera and identifies individual faces, allowing for an accurate determination of the number of people inside the vehicle. The camera unit can also work in conjunction with motion detection to perform facial recognition according to the type of motion. For example, when motion is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. This allows the camera unit to monitor the situation inside the vehicle in real time and respond quickly if an anomaly occurs. Furthermore, the camera unit can use facial recognition technology to identify specific individuals, enhancing safety management inside the vehicle. For example, by comparing the person inside the vehicle with a registered facial database, the camera unit can confirm whether they are a registered family member or friend. This enhances safety inside the vehicle and allows for early detection of abnormal situations.

[0032] The thermometer unit constantly monitors the temperature inside the vehicle and transmits information to the management system if high temperatures persist. The thermometer unit measures the temperature inside the vehicle using, for example, temperature sensors. Temperature sensors are installed in various locations inside the vehicle, allowing for real-time detection of temperature changes. The thermometer unit can also monitor humidity information and integrate it with temperature information to detect abnormalities. For example, the thermometer unit monitors humidity information inside the vehicle and detects abnormalities based on the combination of temperature and humidity. This allows the thermometer unit to comprehensively monitor the environment inside the vehicle and respond quickly if an abnormality occurs. Furthermore, when an abnormality is detected, the thermometer unit transmits information to the management system, enabling appropriate countermeasures to be taken. For example, the thermometer unit sends an alert to the management system when the temperature inside the vehicle exceeds a certain threshold, allowing the management system to take appropriate action. This enhances safety inside the vehicle and enables early detection of abnormal situations.

[0033] The management system sends alerts to smartphones when an anomaly is detected. The management system integrates information such as temperature and the number of people to detect anomalies. Temperature and the number of people are collected from various sensors inside the vehicle and sent to the management system. The management system analyzes this information and sends alerts when an anomaly occurs. The management system can also analyze motion and sound information to improve the accuracy of anomaly detection. For example, the management system analyzes motion information inside the vehicle and detects an anomaly when a specific motion is detected. This allows the management system to comprehensively monitor the situation inside the vehicle and respond quickly when an anomaly occurs. Furthermore, the management system sends alerts to smartphones when an anomaly is detected, notifying the user. This allows the management system to enhance safety inside the vehicle and detect abnormal situations early.

[0034] Smartphones receive alerts and notify the user. For example, they display alerts using notification sounds and message content. Notification sounds are set to immediately alert the user, and message content includes detailed information about the anomaly. Smartphones can also estimate the user's emotions and adjust how alerts are displayed based on that estimation. For example, if the user is feeling anxious, the smartphone displays a simple, highly visible alert. This allows the smartphone to quickly provide the user with appropriate information and assist in responding to unusual situations. Furthermore, smartphones can collect user feedback and continuously improve the way alerts are displayed and the content they contain. This allows smartphones to provide more effective information to users and enhance safety within the vehicle.

[0035] The location information acquisition unit can determine the location of people inside the vehicle. The location information acquisition unit can acquire location information inside the vehicle using, for example, GPS. The location information acquisition unit can also acquire location information using, for example, Wi-Fi or Bluetooth. The location information acquisition unit can also work in conjunction with, for example, motion detection inside the vehicle to acquire location information when motion is detected. This allows for the accurate determination of the location of people inside the vehicle. Some or all of the above-described processes in the location information acquisition unit may be performed using, for example, AI, or without AI. For example, the location information acquisition unit can input motion detection data inside the vehicle into a generating AI and have the generating AI perform location information determination from the motion detection data.

[0036] The camera unit can manage the number of people inside the vehicle through facial recognition. For example, the camera unit counts the number of people inside the vehicle using a facial recognition algorithm. The camera unit can also work in conjunction with motion detection to perform facial recognition according to the type of motion. For example, when motion is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. This allows for accurate management of the number of people inside the vehicle. Some or all of the above processing in the camera unit may be performed using AI, or not. For example, the camera unit can input motion detection data from inside the vehicle into a generating AI, and have the generating AI perform facial recognition based on the motion detection data.

[0037] The thermometer unit can continuously monitor the temperature inside the vehicle and transmit information to the management system if high temperatures persist. The thermometer unit measures the temperature inside the vehicle using a temperature sensor, for example. The thermometer unit can also monitor humidity information and integrate it with the temperature information to detect abnormalities. For example, the thermometer unit monitors the humidity information inside the vehicle and detects abnormalities based on the combination of temperature and humidity. This allows for continuous monitoring of the temperature inside the vehicle and detection of high temperatures. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without AI. For example, the thermometer unit can input humidity information inside the vehicle into a generating AI and have the generating AI perform the integration of humidity information and temperature information.

[0038] The management system can send alerts to smartphones when an anomaly is detected. The management system can, for example, integrate temperature information and occupancy information to detect anomalies. The management system can also, for example, analyze motion information and voice information to improve the accuracy of anomaly detection. The management system can, for example, analyze motion information inside the vehicle and detect an anomaly when a specific motion is detected. This allows for the rapid sending of alerts when an anomaly is detected. Some or all of the above processes in the management system may be performed using AI, for example, or without AI. For example, the management system can input motion information inside the vehicle into a generating AI and have the generating AI perform anomaly detection from the motion information.

[0039] A smartphone can receive alerts and notify the user. The smartphone can display alerts using, for example, a notification sound or message content. The smartphone can also estimate the user's emotions and adjust how the alert is displayed based on the estimated emotions. For example, if the user is stressed, the smartphone can display a simple, highly visible alert. This allows the user to be notified of the alert quickly. Some or all of the above processes in a smartphone may be performed using, for example, AI, or not using AI. For example, the smartphone can input user emotion data into a generating AI and have the generating AI adjust how the alert is displayed based on that emotion data.

[0040] The location information acquisition unit works in conjunction with in-vehicle motion detection to acquire location information when motion is detected. For example, when motion is detected in the vehicle, the location information acquisition unit immediately acquires location information to understand the situation inside the vehicle. For example, the location information acquisition unit adjusts the frequency of location information acquisition according to the type of motion to perform efficient monitoring. For example, if motion continues for a certain period of time, the location information acquisition unit periodically acquires location information to detect abnormalities early. This allows location information to be acquired when motion is detected. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without AI. For example, the location information acquisition unit can input in-vehicle motion detection data into a generating AI and cause the generating AI to acquire location information from the motion detection data.

[0041] The location information acquisition unit can determine a location by considering the seating arrangement information inside the vehicle when acquiring location information. For example, the location information acquisition unit can identify which seats inside the vehicle are occupied based on the seating arrangement information. For example, the location information acquisition unit can work in conjunction with the seating arrangement information to issue an alert if a person is in a specific seat. For example, the location information acquisition unit can improve the accuracy of the location information by considering the seating arrangement information. This allows the location to be determined while considering the seating arrangement information. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without using AI. For example, the location information acquisition unit can input seating arrangement information into a generating AI and have the generating AI perform location determination from the seating arrangement information.

[0042] The location information acquisition unit can acquire location information while considering the vehicle's driving conditions. For example, when the vehicle is stopped, the location information acquisition unit increases the frequency of location information acquisition to understand the situation inside the vehicle in detail. For example, when the vehicle is moving, the location information acquisition unit decreases the frequency of location information acquisition to conserve battery power. For example, the location information acquisition unit adjusts the timing of location information acquisition according to the vehicle's driving speed. This allows location information to be acquired while considering the vehicle's driving conditions. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without using AI. For example, the location information acquisition unit can input vehicle driving data into a generating AI and have the generating AI acquire location information from the driving data.

[0043] The location information acquisition unit can analyze in-vehicle audio information and supplement the location information when acquiring location information. For example, the location information acquisition unit analyzes in-vehicle audio information and acquires location information when a specific sound is detected. For example, the location information acquisition unit improves the accuracy of location information by coordinating with audio information. For example, the location information acquisition unit adjusts the timing of location information acquisition based on audio information. This allows for the analysis of audio information to supplement location information. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without AI. For example, the location information acquisition unit can input in-vehicle audio data into a generating AI and have the generating AI perform location information supplementation from the audio data.

[0044] The camera unit can detect movement inside the vehicle and perform facial recognition according to the type of movement. For example, when movement is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. For example, the camera unit adjusts the frequency of facial recognition according to the type of movement to perform efficient monitoring. For example, if movement continues for a certain period of time, the camera unit performs facial recognition periodically to detect abnormalities early. This enables efficient monitoring by performing facial recognition according to the type of movement. Some or all of the above processing in the camera unit may be performed using AI, for example, or without AI. For example, the camera unit can input movement data inside the vehicle into a generating AI and have the generating AI perform facial recognition from the movement data.

[0045] The camera unit can optimize the facial recognition algorithm by taking into account the lighting conditions inside the vehicle. For example, if the lighting inside the vehicle is dim, the camera unit adjusts the facial recognition algorithm to improve recognition accuracy. For example, if the lighting inside the vehicle is bright, the camera unit optimizes the facial recognition algorithm to improve processing speed. For example, the camera unit adjusts the facial recognition algorithm in real time in response to changes in lighting. This allows the facial recognition algorithm to be optimized according to lighting conditions. Some or all of the above processing in the camera unit may be performed using AI, for example, or without AI. For example, the camera unit can input lighting data inside the vehicle into a generating AI and have the generating AI optimize the facial recognition algorithm based on the lighting data.

[0046] The camera unit can analyze audio information from inside the vehicle and use it as auxiliary information for facial recognition. For example, the camera unit analyzes audio information from inside the vehicle and performs facial recognition when a specific sound is detected. For example, the camera unit improves the accuracy of facial recognition by coordinating with the audio information. For example, the camera unit adjusts the frequency of facial recognition based on the audio information. This allows the audio information to be analyzed and used as auxiliary information for facial recognition. Some or all of the above processing in the camera unit may be performed using AI, for example, or without AI. For example, the camera unit can input audio data from inside the vehicle into a generating AI and have the generating AI perform analysis of the audio data to obtain auxiliary information for facial recognition.

[0047] The camera unit can improve the accuracy of facial recognition by taking into account the temperature information inside the vehicle. For example, if the temperature inside the vehicle is high, the camera unit adjusts the facial recognition algorithm to improve recognition accuracy. For example, if the temperature inside the vehicle is low, the camera unit optimizes the facial recognition algorithm to improve processing speed. For example, the camera unit adjusts the facial recognition algorithm in real time according to changes in temperature. This makes it possible to improve the accuracy of facial recognition by taking temperature information into account. Some or all of the above processing in the camera unit may be performed using AI, for example, or without using AI. For example, the camera unit can input the temperature data inside the vehicle into a generating AI and cause the generating AI to adjust the facial recognition algorithm based on the temperature data.

[0048] The thermometer unit can also monitor humidity information inside the vehicle and integrate it with temperature information to detect abnormalities. For example, the thermometer unit monitors humidity information inside the vehicle and detects abnormalities based on the combination of temperature and humidity. For example, the thermometer unit improves the accuracy of temperature monitoring by coordinating with humidity information. For example, the thermometer unit adjusts the frequency of temperature monitoring according to changes in humidity. This allows for more accurate detection of abnormalities by monitoring humidity information together. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without AI. For example, the thermometer unit can input humidity data inside the vehicle into a generating AI and have the generating AI perform the integration of humidity data and temperature data.

[0049] The thermometer unit can improve the accuracy of temperature monitoring by taking into account the ventilation conditions inside the vehicle. For example, the thermometer unit monitors the ventilation conditions inside the vehicle and detects abnormalities based on a combination of temperature and ventilation. For example, the thermometer unit improves the accuracy of temperature monitoring by coordinating with the ventilation conditions. For example, the thermometer unit adjusts the frequency of temperature monitoring according to changes in ventilation. This allows the accuracy of temperature monitoring to be improved by taking into account the ventilation conditions. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without using AI. For example, the thermometer unit can input ventilation data inside the vehicle into a generating AI and have the generating AI perform the integration of ventilation data and temperature data.

[0050] The thermometer unit can analyze in-vehicle operation information and use it as supplementary information for temperature monitoring. For example, the thermometer unit analyzes in-vehicle operation information and performs temperature monitoring when a specific operation is detected. For example, the thermometer unit improves the accuracy of temperature monitoring by linking with operation information. For example, the thermometer unit adjusts the frequency of temperature monitoring based on operation information. This allows the operation information to be analyzed and used as supplementary information for temperature monitoring. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without AI. For example, the thermometer unit can input in-vehicle operation data into a generating AI and have the generating AI perform analysis of the operation data to perform supplementary information for temperature monitoring.

[0051] The thermometer unit can improve the accuracy of temperature monitoring by taking into account the lighting conditions inside the vehicle. For example, if the lighting inside the vehicle is dim, the thermometer unit can increase the accuracy of temperature monitoring and detect abnormalities. For example, if the lighting inside the vehicle is bright, the thermometer unit can optimize the accuracy of temperature monitoring and improve the processing speed. For example, the thermometer unit can adjust the frequency of temperature monitoring in response to changes in lighting. This allows the accuracy of temperature monitoring to be improved by taking into account the lighting conditions. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without using AI. For example, the thermometer unit can input lighting data inside the vehicle into a generating AI and cause the generating AI to improve the accuracy of temperature monitoring based on the lighting data.

[0052] The management system can analyze in-vehicle operation information and improve the accuracy of anomaly detection. For example, the management system analyzes in-vehicle operation information and detects an anomaly when a specific operation is detected. For example, the management system improves the accuracy of anomaly detection by linking with operation information. For example, the management system adjusts the frequency of anomaly detection based on operation information. This allows the accuracy of anomaly detection to be improved by analyzing operation information. Some or all of the above processes in the management system may be performed using AI, for example, or without AI. For example, the management system can input in-vehicle operation data into a generating AI and have the generating AI perform anomaly detection accuracy improvement based on the operation data.

[0053] The management system can analyze audio information from inside the vehicle and use it as supplementary information for anomaly detection. For example, the management system analyzes audio information from inside the vehicle and detects an anomaly when a specific sound is detected. For example, the management system can improve the accuracy of anomaly detection by linking with the audio information. For example, the management system can adjust the frequency of anomaly detection based on the audio information. This allows the audio information to be analyzed and used as supplementary information for anomaly detection. Some or all of the above processes in the management system may be performed using AI, for example, or without AI. For example, the management system can input audio data from inside the vehicle into a generating AI and have the generating AI perform analysis of the audio data to provide supplementary information for anomaly detection.

[0054] The management system can analyze temperature information inside the vehicle and improve the accuracy of anomaly detection. For example, the management system analyzes temperature information inside the vehicle and detects an anomaly when a specific temperature is detected. For example, the management system improves the accuracy of anomaly detection by linking with temperature information. For example, the management system adjusts the frequency of anomaly detection based on temperature information. This allows the system to improve the accuracy of anomaly detection by analyzing temperature information. Some or all of the above processes in the management system may be performed using AI, for example, or without AI. For example, the management system can input temperature data inside the vehicle into a generating AI and have the generating AI perform anomaly detection accuracy improvement based on the temperature data.

[0055] The management system can improve the accuracy of anomaly detection by taking into account the lighting conditions inside the vehicle. For example, if the lighting inside the vehicle is dim, the management system can improve the accuracy of anomaly detection and detect anomalies. For example, if the lighting inside the vehicle is bright, the management system can optimize the accuracy of anomaly detection and improve processing speed. For example, the management system can adjust the frequency of anomaly detection in response to changes in lighting. This allows the accuracy of anomaly detection to be improved by taking into account the lighting conditions. Some or all of the above processing in the management system may be performed using AI, for example, or without using AI. For example, the management system can input lighting data inside the vehicle into a generating AI and cause the generating AI to improve the accuracy of anomaly detection based on the lighting data.

[0056] A smartphone can analyze in-car activity data and optimize the content of alerts. For example, a smartphone can analyze in-car activity data and issue an alert when a specific action is detected. For example, a smartphone can optimize the content of alerts in conjunction with activity data. For example, a smartphone can adjust the timing of alert issuance based on activity data. This allows for the optimization of alert content by analyzing activity data. Some or all of the above processes in a smartphone may be performed using AI, for example, or without AI. For example, a smartphone can input in-car activity data into a generating AI and have the generating AI optimize the content of alerts based on the activity data.

[0057] A smartphone can analyze audio information from inside a vehicle and use it as supplementary information for alerts. For example, a smartphone can analyze audio information from inside a vehicle and issue an alert when a specific sound is detected. For example, a smartphone can optimize the content of an alert in conjunction with the audio information. For example, a smartphone can adjust the timing of alert issuance based on the audio information. This allows the audio information to be analyzed and used as supplementary information for alerts. Some or all of the above processing in a smartphone may be performed using AI, for example, or without AI. For example, a smartphone can input audio data from inside a vehicle into a generating AI and have the generating AI perform analysis of the audio data to generate supplementary information for alerts.

[0058] A smartphone can analyze temperature information inside a vehicle and optimize the content of alerts. For example, a smartphone can analyze temperature information inside a vehicle and send an alert when a specific temperature is detected. For example, a smartphone can optimize the content of alerts in conjunction with temperature information. For example, a smartphone can adjust the timing of alert sending based on temperature information. This allows for the optimization of alert content by analyzing temperature information. Some or all of the above processes in a smartphone may be performed using AI, for example, or without AI. For example, a smartphone can input temperature data inside the vehicle into a generating AI and have the generating AI optimize the content of alerts based on the temperature data.

[0059] A smartphone can adjust how alerts are displayed based on the lighting conditions inside the vehicle. For example, if the lighting inside the vehicle is dim, the smartphone will display a highly visible alert. If the lighting inside the vehicle is bright, the smartphone will display an alert containing detailed information. For example, the smartphone can adjust how alerts are displayed in real time in response to changes in lighting. This allows the smartphone to adjust how alerts are displayed based on lighting conditions. Some or all of the above processing in the smartphone may be performed using AI, for example, or without AI. For example, the smartphone can input lighting data inside the vehicle into a generating AI and have the generating AI adjust how alerts are displayed based on the lighting data.

[0060] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0061] The in-vehicle safety management system can also be equipped with a voice recognition unit. The voice recognition unit can analyze the sounds inside the vehicle and issue an alert when specific sounds are detected. For example, if a child's crying or a cry for help is detected, the information is immediately sent to the management system, and an alert is sent to a smartphone. The voice recognition unit can also analyze the in-vehicle audio data and detect abnormal sound patterns. For example, if abnormal sounds are continuously detected inside the vehicle, the information can be sent to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety by utilizing audio information.

[0062] The in-vehicle safety management system can also be equipped with a vibration detection unit. This unit monitors vibrations inside the vehicle and can issue an alert if abnormal vibrations are detected. For example, if vibrations from a vehicle collision or violent movements inside the vehicle are detected, the unit sends information to the management system and an alert to a smartphone. The vibration detection unit can also analyze in-vehicle vibration data and detect abnormal vibration patterns. For example, if abnormal vibrations are continuously detected inside the vehicle, the unit can send information to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety by utilizing vibration information.

[0063] The in-vehicle safety management system can also be equipped with a biometric authentication unit. This unit can acquire biometric information from people inside the vehicle and issue alerts if specific biometric information is detected. For example, if heart rate or respiratory rate shows abnormal values, it can send information to the management system and send an alert to a smartphone. The biometric authentication unit can also analyze biometric data within the vehicle and detect abnormal biometric patterns. For example, if abnormal biometric information is continuously detected inside the vehicle, it can send information to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety through the use of biometric information.

[0064] The in-vehicle safety management system can also be equipped with an air quality monitoring unit. This unit monitors the air quality inside the vehicle and can issue alerts if abnormal air quality is detected. For example, if carbon dioxide concentration or harmful gas concentration reaches abnormal levels, it can send information to the management system and send an alert to a smartphone. The air quality monitoring unit can also analyze in-vehicle air quality data and detect abnormal air quality patterns. For example, if abnormal air quality is continuously detected inside the vehicle, it can send information to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety by utilizing air quality information.

[0065] The in-vehicle safety management system can also be equipped with a light sensor. The light sensor monitors the lighting conditions inside the vehicle and can issue an alert if an abnormal lighting condition is detected. For example, if the interior suddenly becomes dark or bright, it can send information to the management system and send an alert to a smartphone. The light sensor can also analyze the lighting data inside the vehicle and detect abnormal lighting patterns. For example, if abnormal lighting conditions are continuously detected inside the vehicle, it can send information to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety by utilizing lighting information.

[0066] The following briefly describes the processing flow for example form 1.

[0067] Step 1: The location information acquisition unit identifies the location of people inside the vehicle. The location information acquisition unit acquires location information inside the vehicle using, for example, GPS. It can also acquire location information using Wi-Fi or Bluetooth. Furthermore, the location information acquisition unit can work in conjunction with motion detection inside the vehicle to acquire location information when motion is detected. For example, when motion is detected inside the vehicle, the location information acquisition unit immediately acquires location information to understand the situation inside the vehicle. Step 2: The camera unit manages the number of people inside the vehicle using facial recognition. The camera unit counts the number of people inside the vehicle using, for example, a facial recognition algorithm. The camera unit can also work in conjunction with motion detection to perform facial recognition according to the type of motion. For example, when motion is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. Step 3: The thermometer unit constantly monitors the temperature inside the vehicle and transmits information to the management system if high temperatures persist. The thermometer unit measures the temperature inside the vehicle using, for example, a temperature sensor. The thermometer unit can also monitor humidity information and integrate it with the temperature information to detect abnormalities. For example, the thermometer unit monitors the humidity information inside the vehicle and detects abnormalities based on the combination of temperature and humidity. Step 4: The management system sends an alert to the smartphone when an anomaly is detected. The management system integrates information such as temperature and the number of people to detect anomalies. The management system can also analyze motion and audio information to improve the accuracy of anomaly detection. For example, the management system analyzes motion information inside the vehicle and detects an anomaly when a specific motion is detected. Step 5: The smartphone receives the alert and notifies the user. The smartphone displays the alert using, for example, a notification sound or message content. The smartphone can also estimate the user's emotions and adjust how the alert is displayed based on the estimated emotions. For example, if the smartphone is stressed, it will display a simple and highly visible alert.

[0068] (Example of form 2) The in-vehicle safety management system according to an embodiment of the present invention is a system for preventing accidents in which children are left unattended in a vehicle. This system uses GPS, a vehicle camera, an in-vehicle thermometer, and in-vehicle motion detection in conjunction with a smartphone to monitor whether a person is left inside the vehicle. For example, the location information acquisition unit shares location information with the camera unit to identify the location of people inside the vehicle. Next, the camera unit performs facial recognition to manage the number of people inside the vehicle. Furthermore, the thermometer unit constantly monitors the temperature inside the vehicle and transmits information to the management system if a high temperature condition persists. The management system integrates this data and sends an alert to the smartphone if an abnormality is detected. For example, an alert is issued if location information remains inside the vehicle after the engine has been turned off, if there is a discrepancy in the number of people entering or leaving the vehicle, or if the temperature inside the vehicle remains above 40 degrees Celsius for more than 10 minutes. This mechanism makes it possible to prevent accidents in which children are left unattended in a vehicle. Thus, the in-vehicle safety management system can prevent accidents in which children are left unattended in a vehicle.

[0069] The in-vehicle safety management system according to this embodiment comprises a location information acquisition unit, a camera unit, a thermometer unit, a management system, and a smartphone. The location information acquisition unit identifies the location of people inside the vehicle. The location information acquisition unit acquires location information inside the vehicle using, for example, GPS. The location information acquisition unit can also acquire location information using Wi-Fi or Bluetooth. Furthermore, the location information acquisition unit can also work in conjunction with in-vehicle motion detection to acquire location information when motion is detected. For example, when motion is detected inside the vehicle, the location information acquisition unit immediately acquires location information to understand the situation inside the vehicle. The camera unit manages the number of people inside the vehicle using facial recognition. The camera unit counts the number of people inside the vehicle using, for example, a facial recognition algorithm. Furthermore, the camera unit can work in conjunction with motion detection to perform facial recognition according to the type of motion. For example, when motion is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. The thermometer unit constantly monitors the temperature inside the vehicle and transmits information to the management system if a high temperature condition persists. The thermometer unit measures the temperature inside the vehicle using, for example, a temperature sensor. The thermometer unit can also monitor humidity information and integrate it with the temperature information to detect anomalies. For example, the thermometer unit monitors humidity information inside the vehicle and detects anomalies based on the combination of temperature and humidity. The management system sends an alert to the smartphone when an anomaly is detected. The management system integrates temperature information and occupancy information to detect anomalies. The management system can also analyze motion information and audio information to improve the accuracy of anomaly detection. For example, the management system analyzes motion information inside the vehicle and detects an anomaly when a specific motion is detected. The smartphone receives the alert and notifies the user. The smartphone displays the alert using, for example, a notification sound or message content. The smartphone can also estimate the user's emotions and adjust the alert display method based on the estimated emotions. For example, if the user is stressed, the smartphone displays a simple and highly visible alert. As a result, the in-vehicle safety management system according to this embodiment can prevent accidents where children are left behind in the vehicle.

[0070] The location information acquisition unit identifies the location of people inside the vehicle. For example, the unit can acquire location information inside the vehicle using GPS. GPS receives signals from satellites and can pinpoint the location of each device inside the vehicle with high accuracy. The location information acquisition unit can also acquire location information using Wi-Fi or Bluetooth. Wi-Fi determines the device's location by measuring the distance to an access point installed inside the vehicle. Bluetooth determines the location based on the communication range with beacons placed inside the vehicle. Furthermore, the location information acquisition unit can work in conjunction with motion detection inside the vehicle to acquire location information when motion is detected. For example, if a motion sensor installed inside the vehicle detects a person's movement, the location information acquisition unit immediately identifies that person's location and understands the situation inside the vehicle. This allows the location information acquisition unit to know the location of people inside the vehicle in real time, enabling rapid response in emergencies. Furthermore, the location information acquisition unit can improve the accuracy of location information by combining multiple technologies. For example, combining GPS, Wi-Fi, and Bluetooth makes it possible to acquire highly accurate location information both indoors and outdoors. This allows the location information acquisition unit to perform safety management inside the vehicle more effectively.

[0071] The camera unit manages the number of people inside the vehicle using facial recognition. For example, the camera unit counts the number of people inside the vehicle using a facial recognition algorithm. The facial recognition algorithm detects human faces from the video captured by the camera and identifies individual faces, allowing for an accurate determination of the number of people inside the vehicle. The camera unit can also work in conjunction with motion detection to perform facial recognition according to the type of motion. For example, when motion is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. This allows the camera unit to monitor the situation inside the vehicle in real time and respond quickly if an anomaly occurs. Furthermore, the camera unit can use facial recognition technology to identify specific individuals, enhancing safety management inside the vehicle. For example, by comparing the person inside the vehicle with a registered facial database, the camera unit can confirm whether they are a registered family member or friend. This enhances safety inside the vehicle and allows for early detection of abnormal situations.

[0072] The thermometer unit constantly monitors the temperature inside the vehicle and transmits information to the management system if high temperatures persist. The thermometer unit measures the temperature inside the vehicle using, for example, temperature sensors. Temperature sensors are installed in various locations inside the vehicle, allowing for real-time detection of temperature changes. The thermometer unit can also monitor humidity information and integrate it with temperature information to detect abnormalities. For example, the thermometer unit monitors humidity information inside the vehicle and detects abnormalities based on the combination of temperature and humidity. This allows the thermometer unit to comprehensively monitor the environment inside the vehicle and respond quickly if an abnormality occurs. Furthermore, when an abnormality is detected, the thermometer unit transmits information to the management system, enabling appropriate countermeasures to be taken. For example, the thermometer unit sends an alert to the management system when the temperature inside the vehicle exceeds a certain threshold, allowing the management system to take appropriate action. This enhances safety inside the vehicle and enables early detection of abnormal situations.

[0073] The management system sends alerts to smartphones when an anomaly is detected. The management system integrates information such as temperature and the number of people to detect anomalies. Temperature and the number of people are collected from various sensors inside the vehicle and sent to the management system. The management system analyzes this information and sends alerts when an anomaly occurs. The management system can also analyze motion and sound information to improve the accuracy of anomaly detection. For example, the management system analyzes motion information inside the vehicle and detects an anomaly when a specific motion is detected. This allows the management system to comprehensively monitor the situation inside the vehicle and respond quickly when an anomaly occurs. Furthermore, the management system sends alerts to smartphones when an anomaly is detected, notifying the user. This allows the management system to enhance safety inside the vehicle and detect abnormal situations early.

[0074] Smartphones receive alerts and notify the user. For example, they display alerts using notification sounds and message content. Notification sounds are set to immediately alert the user, and message content includes detailed information about the anomaly. Smartphones can also estimate the user's emotions and adjust how alerts are displayed based on that estimation. For example, if the user is feeling anxious, the smartphone displays a simple, highly visible alert. This allows the smartphone to quickly provide the user with appropriate information and assist in responding to unusual situations. Furthermore, smartphones can collect user feedback and continuously improve the way alerts are displayed and the content they contain. This allows smartphones to provide more effective information to users and enhance safety within the vehicle.

[0075] The location information acquisition unit can determine the location of people inside the vehicle. The location information acquisition unit can acquire location information inside the vehicle using, for example, GPS. The location information acquisition unit can also acquire location information using, for example, Wi-Fi or Bluetooth. The location information acquisition unit can also work in conjunction with, for example, motion detection inside the vehicle to acquire location information when motion is detected. This allows for the accurate determination of the location of people inside the vehicle. Some or all of the above-described processes in the location information acquisition unit may be performed using, for example, AI, or without AI. For example, the location information acquisition unit can input motion detection data inside the vehicle into a generating AI and have the generating AI perform location information determination from the motion detection data.

[0076] The camera unit can manage the number of people inside the vehicle through facial recognition. For example, the camera unit counts the number of people inside the vehicle using a facial recognition algorithm. The camera unit can also work in conjunction with motion detection to perform facial recognition according to the type of motion. For example, when motion is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. This allows for accurate management of the number of people inside the vehicle. Some or all of the above processing in the camera unit may be performed using AI, or not. For example, the camera unit can input motion detection data from inside the vehicle into a generating AI, and have the generating AI perform facial recognition based on the motion detection data.

[0077] The thermometer unit can continuously monitor the temperature inside the vehicle and transmit information to the management system if high temperatures persist. The thermometer unit measures the temperature inside the vehicle using a temperature sensor, for example. The thermometer unit can also monitor humidity information and integrate it with the temperature information to detect abnormalities. For example, the thermometer unit monitors the humidity information inside the vehicle and detects abnormalities based on the combination of temperature and humidity. This allows for continuous monitoring of the temperature inside the vehicle and detection of high temperatures. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without AI. For example, the thermometer unit can input humidity information inside the vehicle into a generating AI and have the generating AI perform the integration of humidity information and temperature information.

[0078] The management system can send alerts to smartphones when an anomaly is detected. The management system can, for example, integrate temperature information and occupancy information to detect anomalies. The management system can also, for example, analyze motion information and voice information to improve the accuracy of anomaly detection. The management system can, for example, analyze motion information inside the vehicle and detect an anomaly when a specific motion is detected. This allows for the rapid sending of alerts when an anomaly is detected. Some or all of the above processes in the management system may be performed using AI, for example, or without AI. For example, the management system can input motion information inside the vehicle into a generating AI and have the generating AI perform anomaly detection from the motion information.

[0079] A smartphone can receive alerts and notify the user. The smartphone can display alerts using, for example, a notification sound or message content. The smartphone can also estimate the user's emotions and adjust how the alert is displayed based on the estimated emotions. For example, if the user is stressed, the smartphone can display a simple, highly visible alert. This allows the user to be notified of the alert quickly. Some or all of the above processes in a smartphone may be performed using, for example, AI, or not using AI. For example, the smartphone can input user emotion data into a generating AI and have the generating AI adjust how the alert is displayed based on that emotion data.

[0080] The location information acquisition unit can estimate the user's emotions and adjust the timing of location information acquisition based on the estimated emotions. For example, if the user is stressed, the location information acquisition unit will acquire location information more frequently to provide a sense of security. For example, if the user is relaxed, the location information acquisition unit will reduce the frequency of location information acquisition to conserve battery power. For example, if the user is in a hurry, the location information acquisition unit will acquire location information in real time to enable a quick response. This allows the timing of location information acquisition to be adjusted according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without AI. For example, the location information acquisition unit can input user emotion data into a generative AI and have the generative AI adjust the timing of location information acquisition based on the emotion data.

[0081] The location information acquisition unit works in conjunction with in-vehicle motion detection to acquire location information when motion is detected. For example, when motion is detected in the vehicle, the location information acquisition unit immediately acquires location information to understand the situation inside the vehicle. For example, the location information acquisition unit adjusts the frequency of location information acquisition according to the type of motion to perform efficient monitoring. For example, if motion continues for a certain period of time, the location information acquisition unit periodically acquires location information to detect abnormalities early. This allows location information to be acquired when motion is detected. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without AI. For example, the location information acquisition unit can input in-vehicle motion detection data into a generating AI and cause the generating AI to acquire location information from the motion detection data.

[0082] The location information acquisition unit can determine a location by considering the seating arrangement information inside the vehicle when acquiring location information. For example, the location information acquisition unit can identify which seats inside the vehicle are occupied based on the seating arrangement information. For example, the location information acquisition unit can work in conjunction with the seating arrangement information to issue an alert if a person is in a specific seat. For example, the location information acquisition unit can improve the accuracy of the location information by considering the seating arrangement information. This allows the location to be determined while considering the seating arrangement information. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without using AI. For example, the location information acquisition unit can input seating arrangement information into a generating AI and have the generating AI perform location determination from the seating arrangement information.

[0083] The location information acquisition unit can estimate the user's emotions and determine the priority of location information to acquire based on the estimated user emotions. For example, if the user is feeling anxious, the location information acquisition unit will prioritize acquiring important location information. For example, if the user is relaxed, the location information acquisition unit will postpone acquiring less important location information. For example, if the user is in a hurry, the location information acquisition unit will prioritize acquiring urgent location information. This allows the priority of location information to be determined according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without using AI. For example, the location information acquisition unit can input the user's emotion data into a generative AI and have the generative AI determine the priority of location information from the emotion data.

[0084] The location information acquisition unit can acquire location information while considering the vehicle's driving conditions. For example, when the vehicle is stopped, the location information acquisition unit increases the frequency of location information acquisition to understand the situation inside the vehicle in detail. For example, when the vehicle is moving, the location information acquisition unit decreases the frequency of location information acquisition to conserve battery power. For example, the location information acquisition unit adjusts the timing of location information acquisition according to the vehicle's driving speed. This allows location information to be acquired while considering the vehicle's driving conditions. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without using AI. For example, the location information acquisition unit can input vehicle driving data into a generating AI and have the generating AI acquire location information from the driving data.

[0085] The location information acquisition unit can analyze in-vehicle audio information and supplement the location information when acquiring location information. For example, the location information acquisition unit analyzes in-vehicle audio information and acquires location information when a specific sound is detected. For example, the location information acquisition unit improves the accuracy of location information by coordinating with audio information. For example, the location information acquisition unit adjusts the timing of location information acquisition based on audio information. This allows for the analysis of audio information to supplement location information. Some or all of the above processing in the location information acquisition unit may be performed using AI, for example, or without AI. For example, the location information acquisition unit can input in-vehicle audio data into a generating AI and have the generating AI perform location information supplementation from the audio data.

[0086] The camera unit can estimate the user's emotions and adjust the accuracy of facial recognition based on the estimated emotions. For example, if the user is nervous, the camera unit can increase the accuracy of facial recognition to prevent misrecognition. For example, if the user is relaxed, the camera unit can adjust the accuracy of facial recognition appropriately to improve processing speed. For example, if the user is in a hurry, the camera unit can optimize the accuracy of facial recognition for faster authentication. This allows the accuracy of facial recognition to be adjusted according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the camera unit may be performed using AI, for example, or without AI. For example, the camera unit can input user emotion data into a generative AI and have the generative AI adjust the accuracy of facial recognition based on the emotion data.

[0087] The camera unit can detect movement inside the vehicle and perform facial recognition according to the type of movement. For example, when movement is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. For example, the camera unit adjusts the frequency of facial recognition according to the type of movement to perform efficient monitoring. For example, if movement continues for a certain period of time, the camera unit performs facial recognition periodically to detect abnormalities early. This enables efficient monitoring by performing facial recognition according to the type of movement. Some or all of the above processing in the camera unit may be performed using AI, for example, or without AI. For example, the camera unit can input movement data inside the vehicle into a generating AI and have the generating AI perform facial recognition from the movement data.

[0088] The camera unit can optimize the facial recognition algorithm by taking into account the lighting conditions inside the vehicle. For example, if the lighting inside the vehicle is dim, the camera unit adjusts the facial recognition algorithm to improve recognition accuracy. For example, if the lighting inside the vehicle is bright, the camera unit optimizes the facial recognition algorithm to improve processing speed. For example, the camera unit adjusts the facial recognition algorithm in real time in response to changes in lighting. This allows the facial recognition algorithm to be optimized according to lighting conditions. Some or all of the above processing in the camera unit may be performed using AI, for example, or without AI. For example, the camera unit can input lighting data inside the vehicle into a generating AI and have the generating AI optimize the facial recognition algorithm based on the lighting data.

[0089] The camera unit can estimate the user's emotions and adjust the frequency of facial recognition based on the estimated emotions. For example, if the user is feeling anxious, the camera unit increases the frequency of facial recognition to provide a sense of security. For example, if the user is relaxed, the camera unit decreases the frequency of facial recognition to conserve battery power. For example, if the user is in a hurry, the camera unit optimizes the frequency of facial recognition for quick authentication. This allows the frequency of facial recognition to be adjusted according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the camera unit may be performed using AI, for example, or without AI. For example, the camera unit can input user emotion data into a generative AI and have the generative AI adjust the frequency of facial recognition based on the emotion data.

[0090] The camera unit can analyze audio information from inside the vehicle and use it as auxiliary information for facial recognition. For example, the camera unit analyzes audio information from inside the vehicle and performs facial recognition when a specific sound is detected. For example, the camera unit improves the accuracy of facial recognition by coordinating with the audio information. For example, the camera unit adjusts the frequency of facial recognition based on the audio information. This allows the audio information to be analyzed and used as auxiliary information for facial recognition. Some or all of the above processing in the camera unit may be performed using AI, for example, or without AI. For example, the camera unit can input audio data from inside the vehicle into a generating AI and have the generating AI perform analysis of the audio data to obtain auxiliary information for facial recognition.

[0091] The camera unit can improve the accuracy of facial recognition by taking into account the temperature information inside the vehicle. For example, if the temperature inside the vehicle is high, the camera unit adjusts the facial recognition algorithm to improve recognition accuracy. For example, if the temperature inside the vehicle is low, the camera unit optimizes the facial recognition algorithm to improve processing speed. For example, the camera unit adjusts the facial recognition algorithm in real time according to changes in temperature. This makes it possible to improve the accuracy of facial recognition by taking temperature information into account. Some or all of the above processing in the camera unit may be performed using AI, for example, or without using AI. For example, the camera unit can input the temperature data inside the vehicle into a generating AI and cause the generating AI to adjust the facial recognition algorithm based on the temperature data.

[0092] The thermometer unit can estimate the user's emotions and adjust the frequency of temperature monitoring based on the estimated emotions. For example, if the user is feeling anxious, the thermometer unit increases the frequency of temperature monitoring to provide a sense of security. For example, if the user is relaxed, the thermometer unit decreases the frequency of temperature monitoring to conserve battery power. For example, if the user is in a hurry, the thermometer unit optimizes the frequency of temperature monitoring to enable a quick response. This allows the frequency of temperature monitoring to be adjusted according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without AI. For example, the thermometer unit can input user emotion data into a generative AI and have the generative AI adjust the frequency of temperature monitoring based on the emotion data.

[0093] The thermometer unit can also monitor humidity information inside the vehicle and integrate it with temperature information to detect abnormalities. For example, the thermometer unit monitors humidity information inside the vehicle and detects abnormalities based on the combination of temperature and humidity. For example, the thermometer unit improves the accuracy of temperature monitoring by coordinating with humidity information. For example, the thermometer unit adjusts the frequency of temperature monitoring according to changes in humidity. This allows for more accurate detection of abnormalities by monitoring humidity information together. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without AI. For example, the thermometer unit can input humidity data inside the vehicle into a generating AI and have the generating AI perform the integration of humidity data and temperature data.

[0094] The thermometer unit can improve the accuracy of temperature monitoring by taking into account the ventilation conditions inside the vehicle. For example, the thermometer unit monitors the ventilation conditions inside the vehicle and detects abnormalities based on a combination of temperature and ventilation. For example, the thermometer unit improves the accuracy of temperature monitoring by coordinating with the ventilation conditions. For example, the thermometer unit adjusts the frequency of temperature monitoring according to changes in ventilation. This allows the accuracy of temperature monitoring to be improved by taking into account the ventilation conditions. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without using AI. For example, the thermometer unit can input ventilation data inside the vehicle into a generating AI and have the generating AI perform the integration of ventilation data and temperature data.

[0095] The thermometer unit can estimate the user's emotions and determine the priority of temperature monitoring based on the estimated emotions. For example, if the user is feeling anxious, the thermometer unit will prioritize monitoring important temperature information. For example, if the user is relaxed, the thermometer unit will postpone monitoring less important temperature information. For example, if the user is in a hurry, the thermometer unit will prioritize monitoring urgent temperature information. This allows the thermometer unit to determine the priority of temperature monitoring according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without AI. For example, the thermometer unit can input user emotion data into a generative AI and have the generative AI determine the priority of temperature monitoring based on the emotion data.

[0096] The thermometer unit can analyze in-vehicle operation information and use it as supplementary information for temperature monitoring. For example, the thermometer unit analyzes in-vehicle operation information and performs temperature monitoring when a specific operation is detected. For example, the thermometer unit improves the accuracy of temperature monitoring by linking with operation information. For example, the thermometer unit adjusts the frequency of temperature monitoring based on operation information. This allows the operation information to be analyzed and used as supplementary information for temperature monitoring. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without AI. For example, the thermometer unit can input in-vehicle operation data into a generating AI and have the generating AI perform analysis of the operation data to perform supplementary information for temperature monitoring.

[0097] The thermometer unit can improve the accuracy of temperature monitoring by taking into account the lighting conditions inside the vehicle. For example, if the lighting inside the vehicle is dim, the thermometer unit can increase the accuracy of temperature monitoring and detect abnormalities. For example, if the lighting inside the vehicle is bright, the thermometer unit can optimize the accuracy of temperature monitoring and improve the processing speed. For example, the thermometer unit can adjust the frequency of temperature monitoring in response to changes in lighting. This allows the accuracy of temperature monitoring to be improved by taking into account the lighting conditions. Some or all of the above processing in the thermometer unit may be performed using AI, for example, or without using AI. For example, the thermometer unit can input lighting data inside the vehicle into a generating AI and cause the generating AI to improve the accuracy of temperature monitoring based on the lighting data.

[0098] The management system can estimate the user's emotions and adjust the anomaly detection criteria based on the estimated emotions. For example, if the user is feeling anxious, the management system can tighten the anomaly detection criteria to detect anomalies earlier. For example, if the user is relaxed, the management system can loosen the anomaly detection criteria to reduce false positives. For example, if the user is in a hurry, the management system can optimize the anomaly detection criteria to enable a quick response. This allows the anomaly detection criteria to be adjusted according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the management system may be performed using AI, for example, or not using AI. For example, the management system can input user emotion data into a generative AI and have the generative AI adjust the anomaly detection criteria based on the emotion data.

[0099] The management system can analyze in-vehicle operation information and improve the accuracy of anomaly detection. For example, the management system analyzes in-vehicle operation information and detects an anomaly when a specific operation is detected. For example, the management system improves the accuracy of anomaly detection by linking with operation information. For example, the management system adjusts the frequency of anomaly detection based on operation information. This allows the accuracy of anomaly detection to be improved by analyzing operation information. Some or all of the above processes in the management system may be performed using AI, for example, or without AI. For example, the management system can input in-vehicle operation data into a generating AI and have the generating AI perform anomaly detection accuracy improvement based on the operation data.

[0100] The management system can analyze audio information from inside the vehicle and use it as supplementary information for anomaly detection. For example, the management system analyzes audio information from inside the vehicle and detects an anomaly when a specific sound is detected. For example, the management system can improve the accuracy of anomaly detection by linking with the audio information. For example, the management system can adjust the frequency of anomaly detection based on the audio information. This allows the audio information to be analyzed and used as supplementary information for anomaly detection. Some or all of the above processes in the management system may be performed using AI, for example, or without AI. For example, the management system can input audio data from inside the vehicle into a generating AI and have the generating AI perform analysis of the audio data to provide supplementary information for anomaly detection.

[0101] The management system can estimate the user's emotions and adjust the frequency of anomaly detection based on the estimated emotions. For example, if the user is feeling anxious, the management system can increase the frequency of anomaly detection to provide reassurance. For example, if the user is relaxed, the management system can decrease the frequency of anomaly detection to conserve battery power. For example, if the user is in a hurry, the management system can optimize the frequency of anomaly detection to enable a quick response. This allows the frequency of anomaly detection to be adjusted according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the management system may be performed using AI, for example, or not using AI. For example, the management system can input user emotion data into a generative AI and have the generative AI adjust the frequency of anomaly detection based on the emotion data.

[0102] The management system can analyze temperature information inside the vehicle and improve the accuracy of anomaly detection. For example, the management system analyzes temperature information inside the vehicle and detects an anomaly when a specific temperature is detected. For example, the management system improves the accuracy of anomaly detection by linking with temperature information. For example, the management system adjusts the frequency of anomaly detection based on temperature information. This allows the system to improve the accuracy of anomaly detection by analyzing temperature information. Some or all of the above processes in the management system may be performed using AI, for example, or without AI. For example, the management system can input temperature data inside the vehicle into a generating AI and have the generating AI perform anomaly detection accuracy improvement based on the temperature data.

[0103] The management system can improve the accuracy of anomaly detection by taking into account the lighting conditions inside the vehicle. For example, if the lighting inside the vehicle is dim, the management system can improve the accuracy of anomaly detection and detect anomalies. For example, if the lighting inside the vehicle is bright, the management system can optimize the accuracy of anomaly detection and improve processing speed. For example, the management system can adjust the frequency of anomaly detection in response to changes in lighting. This allows the accuracy of anomaly detection to be improved by taking into account the lighting conditions. Some or all of the above processing in the management system may be performed using AI, for example, or without using AI. For example, the management system can input lighting data inside the vehicle into a generating AI and cause the generating AI to improve the accuracy of anomaly detection based on the lighting data.

[0104] A smartphone can estimate the user's emotions and adjust how alerts are displayed based on those emotions. For example, if the user is stressed, the smartphone might display a simple, highly visible alert. If the user is relaxed, the smartphone might display an alert with more detailed information. If the user is in a hurry, the smartphone might display a concise alert. This allows the alert display method to be adjusted according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in a smartphone may be performed using AI or not. For example, a smartphone can input user emotion data into a generative AI and have the generative AI adjust how alerts are displayed based on that emotion data.

[0105] A smartphone can analyze in-car activity data and optimize the content of alerts. For example, a smartphone can analyze in-car activity data and issue an alert when a specific action is detected. For example, a smartphone can optimize the content of alerts in conjunction with activity data. For example, a smartphone can adjust the timing of alert issuance based on activity data. This allows for the optimization of alert content by analyzing activity data. Some or all of the above processes in a smartphone may be performed using AI, for example, or without AI. For example, a smartphone can input in-car activity data into a generating AI and have the generating AI optimize the content of alerts based on the activity data.

[0106] A smartphone can analyze audio information from inside a vehicle and use it as supplementary information for alerts. For example, a smartphone can analyze audio information from inside a vehicle and issue an alert when a specific sound is detected. For example, a smartphone can optimize the content of an alert in conjunction with the audio information. For example, a smartphone can adjust the timing of alert issuance based on the audio information. This allows the audio information to be analyzed and used as supplementary information for alerts. Some or all of the above processing in a smartphone may be performed using AI, for example, or without AI. For example, a smartphone can input audio data from inside a vehicle into a generating AI and have the generating AI perform analysis of the audio data to generate supplementary information for alerts.

[0107] A smartphone can estimate the user's emotions and prioritize alerts based on those emotions. For example, if the user is feeling anxious, the smartphone will prioritize important alerts. If the user is relaxed, the smartphone will postpone displaying less important alerts. If the user is in a hurry, the smartphone will prioritize displaying urgent alerts. This allows the smartphone to prioritize alerts according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in a smartphone may be performed using AI or not. For example, a smartphone can input user emotion data into a generative AI and have the generative AI determine the priority of alerts based on that emotion data.

[0108] A smartphone can analyze temperature information inside a vehicle and optimize the content of alerts. For example, a smartphone can analyze temperature information inside a vehicle and send an alert when a specific temperature is detected. For example, a smartphone can optimize the content of alerts in conjunction with temperature information. For example, a smartphone can adjust the timing of alert sending based on temperature information. This allows for the optimization of alert content by analyzing temperature information. Some or all of the above processes in a smartphone may be performed using AI, for example, or without AI. For example, a smartphone can input temperature data inside the vehicle into a generating AI and have the generating AI optimize the content of alerts based on the temperature data.

[0109] A smartphone can adjust how alerts are displayed based on the lighting conditions inside the vehicle. For example, if the lighting inside the vehicle is dim, the smartphone will display a highly visible alert. If the lighting inside the vehicle is bright, the smartphone will display an alert containing detailed information. For example, the smartphone can adjust how alerts are displayed in real time in response to changes in lighting. This allows the smartphone to adjust how alerts are displayed based on lighting conditions. Some or all of the above processing in the smartphone may be performed using AI, for example, or without AI. For example, the smartphone can input lighting data inside the vehicle into a generating AI and have the generating AI adjust how alerts are displayed based on the lighting data.

[0110] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0111] The in-vehicle safety management system can also be equipped with a voice recognition unit. The voice recognition unit can analyze the sounds inside the vehicle and issue an alert when specific sounds are detected. For example, if a child's crying or a cry for help is detected, the information is immediately sent to the management system, and an alert is sent to a smartphone. The voice recognition unit can also analyze the in-vehicle audio data and detect abnormal sound patterns. For example, if abnormal sounds are continuously detected inside the vehicle, the information can be sent to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety by utilizing audio information.

[0112] The in-vehicle safety management system can also be equipped with a vibration detection unit. This unit monitors vibrations inside the vehicle and can issue an alert if abnormal vibrations are detected. For example, if vibrations from a vehicle collision or violent movements inside the vehicle are detected, the unit sends information to the management system and an alert to a smartphone. The vibration detection unit can also analyze in-vehicle vibration data and detect abnormal vibration patterns. For example, if abnormal vibrations are continuously detected inside the vehicle, the unit can send information to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety by utilizing vibration information.

[0113] The in-vehicle safety management system can also be equipped with a biometric authentication unit. This unit can acquire biometric information from people inside the vehicle and issue alerts if specific biometric information is detected. For example, if heart rate or respiratory rate shows abnormal values, it can send information to the management system and send an alert to a smartphone. The biometric authentication unit can also analyze biometric data within the vehicle and detect abnormal biometric patterns. For example, if abnormal biometric information is continuously detected inside the vehicle, it can send information to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety through the use of biometric information.

[0114] The in-vehicle safety management system can also be equipped with an air quality monitoring unit. This unit monitors the air quality inside the vehicle and can issue alerts if abnormal air quality is detected. For example, if carbon dioxide concentration or harmful gas concentration reaches abnormal levels, it can send information to the management system and send an alert to a smartphone. The air quality monitoring unit can also analyze in-vehicle air quality data and detect abnormal air quality patterns. For example, if abnormal air quality is continuously detected inside the vehicle, it can send information to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety by utilizing air quality information.

[0115] The in-vehicle safety management system can also be equipped with a light sensor. The light sensor monitors the lighting conditions inside the vehicle and can issue an alert if an abnormal lighting condition is detected. For example, if the interior suddenly becomes dark or bright, it can send information to the management system and send an alert to a smartphone. The light sensor can also analyze the lighting data inside the vehicle and detect abnormal lighting patterns. For example, if abnormal lighting conditions are continuously detected inside the vehicle, it can send information to the management system to prompt a quick response. This allows for further improvement of in-vehicle safety by utilizing lighting information.

[0116] The management system can estimate the user's emotions and adjust the content of alerts based on those emotions. For example, if the user is feeling anxious, it can display an alert with detailed information to provide reassurance. If the user is relaxed, it can display a concise alert to reduce annoyance. If the user is in a hurry, it can display a to-the-point alert to encourage a quick response. By adjusting the content of alerts according to the user's emotions, more effective notifications become possible.

[0117] The camera unit can estimate the user's emotions and adjust the camera's field of view based on those emotions. For example, if the user is tense, a wide field of view monitors the entire interior of the vehicle to provide a sense of security. If the user is relaxed, a narrow field of view focuses on monitoring a specific area to conserve battery power. If the user is in a hurry, the field of view is optimized for rapid monitoring. By adjusting the camera's field of view according to the user's emotions, more effective monitoring becomes possible.

[0118] The thermometer unit can estimate the user's emotions and adjust the temperature alert threshold based on those emotions. For example, if the user is feeling anxious, it will issue an alert even at low temperatures to provide reassurance. If the user is relaxed, it will only issue an alert at high temperatures to reduce annoyance. If the user is in a hurry, it will optimize the temperature alert threshold to encourage a quick response. This allows for more effective temperature management by adjusting the temperature alert threshold according to the user's emotions.

[0119] The location information acquisition unit can estimate the user's emotions and adjust the accuracy of the location information based on those emotions. For example, if the user is feeling anxious, it acquires highly accurate location information to provide a sense of security. If the user is relaxed, it adjusts the accuracy appropriately to conserve battery power. If the user is in a hurry, it optimizes the accuracy of the location information to enable a quick response. By adjusting the accuracy of location information according to the user's emotions, more effective location information management becomes possible.

[0120] Smartphones can estimate the user's emotions and adjust the type of notification sound based on that estimation. For example, if the user is stressed, a calm notification sound can be used to provide reassurance. If the user is relaxed, a bright notification sound can be used to maintain that mood. If the user is in a hurry, an attention-grabbing notification sound can be used to encourage a quick response. By adjusting the type of notification sound according to the user's emotions, more effective notifications can be achieved.

[0121] The following briefly describes the processing flow for example form 2.

[0122] Step 1: The location information acquisition unit identifies the location of people inside the vehicle. The location information acquisition unit acquires location information inside the vehicle using, for example, GPS. It can also acquire location information using Wi-Fi or Bluetooth. Furthermore, the location information acquisition unit can work in conjunction with motion detection inside the vehicle to acquire location information when motion is detected. For example, when motion is detected inside the vehicle, the location information acquisition unit immediately acquires location information to understand the situation inside the vehicle. Step 2: The camera unit manages the number of people inside the vehicle using facial recognition. The camera unit counts the number of people inside the vehicle using, for example, a facial recognition algorithm. The camera unit can also work in conjunction with motion detection to perform facial recognition according to the type of motion. For example, when motion is detected inside the vehicle, the camera unit immediately performs facial recognition to confirm the number of people inside. Step 3: The thermometer unit constantly monitors the temperature inside the vehicle and transmits information to the management system if high temperatures persist. The thermometer unit measures the temperature inside the vehicle using, for example, a temperature sensor. The thermometer unit can also monitor humidity information and integrate it with the temperature information to detect abnormalities. For example, the thermometer unit monitors the humidity information inside the vehicle and detects abnormalities based on the combination of temperature and humidity. Step 4: The management system sends an alert to the smartphone when an anomaly is detected. The management system integrates information such as temperature and the number of people to detect anomalies. The management system can also analyze motion and audio information to improve the accuracy of anomaly detection. For example, the management system analyzes motion information inside the vehicle and detects an anomaly when a specific motion is detected. Step 5: The smartphone receives the alert and notifies the user. The smartphone displays the alert using, for example, a notification sound or message content. The smartphone can also estimate the user's emotions and adjust how the alert is displayed based on the estimated emotions. For example, if the smartphone is stressed, it will display a simple and highly visible alert.

[0123] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0124] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0125] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0126] Each of the multiple elements described above, including the location information acquisition unit, camera unit, thermometer unit, management system, and smartphone, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the location information acquisition unit is implemented using the GPS function, Wi-Fi, or Bluetooth of the smart device 14. The camera unit uses the camera 42 of the smart device 14 to perform facial recognition and manage the number of people inside the vehicle. The thermometer unit monitors the temperature inside the vehicle using the temperature sensor of the smart device 14. The management system integrates temperature information and person information using the identification processing unit 290 of the data processing unit 12 and detects abnormalities. The smartphone receives alerts and notifies the user using notification sounds and message content. The correspondence between each unit and the device or control unit is not limited to the example described above and can be changed in various ways.

[0127] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0128] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0129] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0130] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0131] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0132] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0133] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0134] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0135] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0136] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0137] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0138] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0139] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0140] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0141] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0142] Each of the multiple elements described above, including the location information acquisition unit, camera unit, thermometer unit, management system, and smartphone, is implemented in at least one of the smart glasses 214 and the data processing unit 12. For example, the location information acquisition unit is implemented using the GPS function, Wi-Fi, and Bluetooth of the smart glasses 214. The camera unit performs facial recognition using the camera 42 of the smart glasses 214 and manages the number of people inside the vehicle. The thermometer unit monitors the temperature inside the vehicle using the temperature sensor of the smart glasses 214. The management system integrates temperature information and person information using the identification processing unit 290 of the data processing unit 12 and detects abnormalities. The smartphone receives alerts and notifies the user using notification sounds and message content. The correspondence between each unit and the device and control unit is not limited to the example described above and can be changed in various ways.

[0143] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0144] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0145] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0146] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0147] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0148] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0149] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0150] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0151] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0152] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0153] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0154] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0155] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0156] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0157] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0158] Each of the multiple elements described above, including the location information acquisition unit, camera unit, thermometer unit, management system, and smartphone, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the location information acquisition unit is implemented using the GPS function, Wi-Fi, or Bluetooth of the headset terminal 314. The camera unit performs facial recognition using the camera 42 of the headset terminal 314 and manages the number of people inside the vehicle. The thermometer unit monitors the temperature inside the vehicle using the temperature sensor of the headset terminal 314. The management system integrates temperature information and person information using the identification processing unit 290 of the data processing unit 12 and detects abnormalities. The smartphone receives alerts and notifies the user using notification sounds and message content. The correspondence between each unit and the device or control unit is not limited to the example described above and can be changed in various ways.

[0159] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0160] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0161] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0162] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0163] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0164] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0165] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0166] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0167] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0168] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0169] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0170] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0171] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0172] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0173] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0174] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0175] Each of the multiple elements described above, including the location information acquisition unit, camera unit, thermometer unit, management system, and smartphone, is implemented in at least one of the following: the robot 414 and the data processing unit 12. For example, the location information acquisition unit is implemented using the GPS function, Wi-Fi, or Bluetooth of the robot 414. The camera unit uses the camera 42 of the robot 414 to perform facial recognition and manage the number of people inside the vehicle. The thermometer unit monitors the temperature inside the vehicle using the temperature sensor of the robot 414. The management system integrates temperature information and person information using the identification processing unit 290 of the data processing unit 12 and detects abnormalities. The smartphone receives alerts and notifies the user using notification sounds and message content. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0176] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0177] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0178] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0179] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0180] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0181] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0182] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0183] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0184] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0185] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0186] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0187] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0188] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0189] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0190] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0191] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0192] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0193] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0194] (Note 1) A location information acquisition unit that acquires location information, A camera unit manages the number of people inside the vehicle based on the location information acquired by the location information acquisition unit, A thermometer unit monitors the temperature inside the vehicle based on the number of people information managed by the aforementioned camera unit, A management system that detects abnormalities based on temperature information monitored by the thermometer unit, The system includes a smartphone that sends an alert based on abnormal information detected by the aforementioned management system. A system characterized by the following features. (Note 2) The aforementioned location information acquisition unit, Identify the location of people inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned camera unit is Facial recognition is used to manage the number of people inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 4) The thermometer section is The system constantly monitors the temperature inside the vehicle and sends information to the management system if high temperatures persist. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned management system is If an anomaly is detected, an alert will be sent to your smartphone. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned smartphone is Receive alerts and notify the user. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned location information acquisition unit, The system estimates the user's emotions and adjusts the timing of location data acquisition based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned location information acquisition unit, It works in conjunction with motion detection inside the vehicle to acquire location information when motion is detected. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned location information acquisition unit, When acquiring location information, the vehicle's seating arrangement is taken into consideration to determine the location. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned location information acquisition unit, The system estimates the user's emotions and determines the priority of location information to acquire based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned location information acquisition unit, When acquiring location information, the vehicle's driving conditions were taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned location information acquisition unit, When acquiring location information, the system analyzes audio information from inside the vehicle to supplement the location information. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned camera unit is It estimates the user's emotions and adjusts the accuracy of facial recognition based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned camera unit is The system detects movement inside the vehicle and performs facial recognition according to the type of movement. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned camera unit is The facial recognition algorithm is optimized considering the lighting conditions inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned camera unit is The system estimates the user's emotions and adjusts the frequency of facial recognition based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned camera unit is The system analyzes audio information from inside the vehicle and uses it as supplementary information for facial recognition. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned camera unit is Improving facial recognition accuracy by taking into account the temperature information inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 19) The thermometer section is The system estimates the user's emotions and adjusts the frequency of temperature monitoring based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 20) The thermometer section is The system also monitors humidity levels inside the vehicle and integrates them with temperature information to detect abnormalities. The system described in Appendix 1, characterized by the features described herein. (Note 21) The thermometer section is Improve the accuracy of temperature monitoring by taking into account the ventilation conditions inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 22) The thermometer section is The system estimates the user's emotions and determines the priority of temperature monitoring based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 23) The thermometer section is The system analyzes operational data inside the vehicle and uses it as supplementary information for temperature monitoring. The system described in Appendix 1, characterized by the features described herein. (Note 24) The thermometer section is Improve the accuracy of temperature monitoring by taking into account the lighting conditions inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned management system is The system estimates the user's emotions and adjusts the anomaly detection criteria based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned management system is Analyzing in-vehicle operation data improves the accuracy of anomaly detection. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned management system is The system analyzes audio information from inside the vehicle and uses it as supplementary information for detecting anomalies. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned management system is It estimates the user's emotions and adjusts the frequency of anomaly detection based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned management system is Analyzing in-vehicle temperature information improves the accuracy of anomaly detection. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned management system is Improve the accuracy of anomaly detection by taking into account the lighting conditions inside the vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned smartphone is It estimates the user's emotions and adjusts how alerts are displayed based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 32) The aforementioned smartphone is Analyze in-vehicle operation data and optimize the content of alerts. The system described in Appendix 1, characterized by the features described herein. (Note 33) The aforementioned smartphone is The system analyzes audio information from inside the vehicle and uses it as supplementary information for alerts. The system described in Appendix 1, characterized by the features described herein. (Note 34) The aforementioned smartphone is It estimates the user's emotions and determines the priority of alerts based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 35) The aforementioned smartphone is Analyze the temperature information inside the vehicle and optimize the content of the alerts. The system described in Appendix 1, characterized by the features described herein. (Note 36) The aforementioned smartphone is The way alerts are displayed will be adjusted to take into account the lighting conditions inside the vehicle. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0195] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A location information acquisition unit that acquires location information, A camera unit manages the number of people inside the vehicle based on the location information acquired by the location information acquisition unit, A thermometer unit monitors the temperature inside the vehicle based on the number of people information managed by the aforementioned camera unit, A management system that detects abnormalities based on temperature information monitored by the thermometer unit, The system includes a smartphone that sends an alert based on abnormal information detected by the aforementioned management system. A system characterized by the following features.

2. The aforementioned location information acquisition unit, Identify the location of people inside the vehicle. The system according to feature 1.

3. The aforementioned camera unit is Facial recognition is used to manage the number of people inside the vehicle. The system according to feature 1.

4. The thermometer section is The system constantly monitors the temperature inside the vehicle and sends information to the management system if high temperatures persist. The system according to feature 1.

5. The aforementioned management system is If an anomaly is detected, an alert will be sent to your smartphone. The system according to feature 1.

6. The aforementioned smartphone is Receive alerts and notify the user. The system according to feature 1.

7. The aforementioned location information acquisition unit, The system estimates the user's emotions and adjusts the timing of location data acquisition based on those emotions. The system according to feature 1.

8. The aforementioned location information acquisition unit, It works in conjunction with motion detection inside the vehicle to acquire location information when motion is detected. The system according to feature 1.

9. The aforementioned location information acquisition unit, When acquiring location information, the vehicle's seating arrangement is taken into consideration to determine the location. The system according to feature 1.

10. The aforementioned location information acquisition unit, The system estimates the user's emotions and determines the priority of location information to acquire based on the estimated emotions. The system according to feature 1.

Images