System

Abstract

An object of a system according to an embodiment is to prevent an accident caused by a person remaining in a vehicle.SOLUTION: A system according to an embodiment includes a position information acquisition unit, a video acquisition unit, an image analysis unit, a temperature monitoring unit, an abnormality detection unit, and a warning provision unit. The position information acquisition unit acquires position information of the vehicle. The image acquisition unit acquires an image inside the vehicle. The image analysis unit analyzes the image acquired by the image acquisition unit to detect whether there is a person in the vehicle. The temperature monitoring unit measures a temperature inside the vehicle. The abnormality detection unit analyzes the temperature data measured by the temperature monitoring unit to detect an abnormal temperature rise. The warning provision unit provides a warning based on information obtained by the position information acquisition unit, the image analysis unit, and the abnormality detection unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology does not adequately monitor vehicles to prevent accidents caused by people being left inside, and there is room for improvement.

[0005] The system according to the embodiment aims to prevent accidents caused by people being left in a vehicle. [Means for solving the problem]

[0006] The system according to the embodiment includes a location information acquisition unit, a video acquisition unit, an image analysis unit, a temperature monitoring unit, an abnormality detection unit, and a warning provision unit. The location information acquisition unit acquires location information of the vehicle. The video acquisition unit acquires video of the interior of the vehicle. The image analysis unit analyzes the video acquired by the video acquisition unit to detect whether there is anyone in the vehicle. The temperature monitoring unit measures the temperature inside the vehicle. The abnormality detection unit analyzes the temperature data measured by the temperature monitoring unit to detect an abnormal temperature rise. The warning provision unit provides a warning based on the information obtained by the location information acquisition unit, the image analysis unit, and the abnormality detection unit. [Effects of the Invention]

[0007] The system according to the embodiment can prevent accidents caused by people being left in the vehicle. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) An in-vehicle safety management system according to an embodiment of the present invention is a system for preventing accidents in which children are left unattended in a vehicle. The in-vehicle safety management system acquires vehicle location information, acquires video of the vehicle interior, analyzes the video to detect whether a person is inside the vehicle, measures the temperature inside the vehicle, detects an abnormal temperature rise, and provides a warning based on this information. For example, the in-vehicle safety management system acquires vehicle location information. Next, the in-vehicle safety management system acquires video of the vehicle interior, and an image analysis unit analyzes the video to detect whether a person is inside the vehicle. Furthermore, the in-vehicle safety management system measures the temperature inside the vehicle, and an abnormality detection unit analyzes the data to detect an abnormal temperature rise. This information is transmitted to a warning providing unit. If the driver leaves the vehicle while a child is inside the vehicle, the warning providing unit sends a warning to a smartphone. For example, if the driver moves a certain distance away from the vehicle, a warning message stating "A child has been left in the vehicle" is displayed on the smartphone. This allows the driver to immediately return to the vehicle and safely protect the child. In this way, the in-vehicle safety management system can prevent accidents in which children are left unattended in the vehicle. For example, if a child is left in a car on a hot summer day, the temperature inside the car can rise sharply, potentially putting their life at risk. This system monitors the temperature inside the car and sends an alert if an abnormal temperature rise is detected, ensuring the safety of the child. The system also constantly monitors whether anyone is left in the car, ensuring the safety of not only children, but also pets and the elderly. For example, if a pet is left in the car, a similar alert is sent, ensuring the pet's safety. In this way, the in-car safety management system is an effective means of preventing accidents involving children being left in cars, and by providing prompt warnings to drivers, it can ensure the safety of children.

[0029] An in-vehicle safety management system according to an embodiment includes a location information acquisition unit, a video acquisition unit, an image analysis unit, a temperature monitoring unit, an abnormality detection unit, and a warning provision unit. The location information acquisition unit acquires vehicle location information. The location information acquisition unit can acquire location information using, for example, GPS, Wi-Fi, a cell tower, etc. The video acquisition unit acquires video of the vehicle interior. The video acquisition unit can acquire video by setting, for example, the type of camera, resolution, frame rate, etc. The image analysis unit analyzes the video acquired by the video acquisition unit to detect whether a person is inside the vehicle. The image analysis unit can analyze the video by, for example, an image processing algorithm or a machine learning model. The temperature monitoring unit measures the temperature inside the vehicle. The temperature monitoring unit can measure the temperature by setting, for example, the type of temperature sensor, measurement range, accuracy, etc. The abnormality detection unit analyzes the temperature data measured by the temperature monitoring unit to detect an abnormal temperature rise. The abnormality detection unit can detect an abnormal temperature rise based on, for example, a temperature threshold value, a temperature rise rate, etc. The warning providing unit provides a warning based on the information obtained by the location information acquiring unit, the image analyzing unit, and the abnormality detecting unit. The warning providing unit can provide a warning by, for example, an audio warning, a visual warning, a smartphone notification, etc. As a result, the in-vehicle safety management system according to the embodiment is a system for preventing accidents in which children are left unattended in a vehicle.

[0030] The warning unit may send a warning message to the smartphone when the driver moves a certain distance away from the vehicle. The certain distance may be, for example, a distance in meters or GPS accuracy, but is not limited to these examples. The warning message may be, for example, a text message or a voice message, but is not limited to these examples. For example, the warning unit may send a warning message to the smartphone when the driver moves more than 10 meters away from the vehicle. The warning unit may also send a voice message when the driver moves more than 20 meters away from the vehicle. The warning unit may also provide a visual warning when the driver moves more than 30 meters away from the vehicle. This ensures the safety of children by sending a warning when the driver moves away from the vehicle.

[0031] The video acquisition unit acquires video of the interior of the vehicle, and the image analysis unit analyzes the video to detect whether a person is inside the vehicle. The video acquisition unit can acquire video by setting, for example, the type of camera, resolution, frame rate, etc. For example, the video acquisition unit acquires video of the interior of the vehicle using a high-resolution camera. The video acquisition unit can also acquire video of the entire interior of the vehicle using a wide-angle lens. The video acquisition unit can also acquire video even in dark places using an infrared camera. The image analysis unit can analyze the video using, for example, an image processing algorithm or a machine learning model. For example, the image analysis unit can detect whether a person is inside the vehicle using a face recognition algorithm. The image analysis unit can also detect movement inside the vehicle using a motion detection algorithm. The image analysis unit can also detect objects inside the vehicle using an object recognition algorithm. This makes it possible to accurately detect whether a person is inside the vehicle.

[0032] The temperature monitoring unit measures the temperature inside the vehicle, and the anomaly detection unit analyzes the measured data to detect an abnormal temperature rise. The temperature monitoring unit can measure the temperature by setting, for example, the type of temperature sensor, measurement range, accuracy, etc. For example, the temperature monitoring unit measures the temperature inside the vehicle using a high-precision temperature sensor. The temperature monitoring unit can also measure the temperature of various parts inside the vehicle using multiple temperature sensors. The temperature monitoring unit can also acquire temperature data in real time. The anomaly detection unit can detect an abnormal temperature rise based on, for example, a temperature threshold or a temperature rise rate. For example, the anomaly detection unit detects an abnormality when the temperature exceeds a certain threshold. The anomaly detection unit can also detect an abnormality when the temperature rises rapidly. The anomaly detection unit can also detect an abnormality by analyzing the temperature fluctuation pattern. This allows the system to detect an increase in the temperature inside the vehicle and issue a warning of an abnormal temperature rise.

[0033] The warning providing unit can also send a warning when a pet or an elderly person is left in the vehicle. Pets include, but are not limited to, for example, dogs and cats. Elderly people can be detected, for example, using face recognition or an age estimation algorithm. For example, the warning providing unit sends a warning when a pet is left in the vehicle. The warning providing unit can also send a warning when an elderly person is left in the vehicle. The warning providing unit can also send a voice warning when a pet or an elderly person is left in the vehicle. This can also ensure the safety of pets and elderly people.

[0034] The warning providing unit can also send a warning when an abnormal temperature rise is detected. Examples of abnormal temperature rise include, but are not limited to, a temperature threshold and a temperature rise rate. For example, the warning providing unit sends a warning when the temperature exceeds a certain threshold. The warning providing unit can also send a warning when the temperature rises rapidly. The warning providing unit can also send a warning when an abnormality is detected by analyzing a temperature fluctuation pattern. In this way, by sending a warning even when an abnormal temperature rise is detected, the safety of children can be ensured.

[0035] When acquiring vehicle position information, the position information acquisition unit can select the optimal acquisition timing by taking into consideration surrounding traffic conditions. Traffic conditions include, for example, but are not limited to, traffic sensors and real-time data. The optimal acquisition timing includes, for example, but are not limited to, traffic volume and traffic light timing. For example, when traffic congestion occurs, the position information acquisition unit increases the frequency of acquiring position information to provide real-time position information. Furthermore, when traffic is smooth, the position information acquisition unit can also reduce the frequency of acquiring position information to reduce the load on the system. Furthermore, when a traffic accident occurs, the position information acquisition unit can increase the frequency of acquiring position information to enable a quick response. Thus, by selecting the timing of acquiring position information by taking into consideration surrounding traffic conditions, more accurate position information can be provided.

[0036] When acquiring position information, the position information acquisition unit can improve the accuracy of the position information by taking into account the speed and direction of the vehicle. The speed includes, for example, but is not limited to, a speed sensor or GPS data. The direction includes, for example, but is not limited to, a compass or a gyro sensor. For example, when the vehicle is moving at high speed, the position information acquisition unit can increase the frequency of acquiring the position information to provide highly accurate position information. Furthermore, when the vehicle is moving at low speed, the position information acquisition unit can also reduce the frequency of acquiring the position information to reduce the load on the system. Furthermore, when the vehicle changes direction frequently, the position information acquisition unit can also increase the frequency of acquiring the position information to provide accurate position information. Thus, by improving the accuracy of the position information by taking into account the speed and direction of the vehicle, more accurate position information can be provided.

[0037] The position information acquisition unit can simultaneously acquire altitude information of the vehicle when acquiring the position information, and provide three-dimensional position information. Altitude information includes, for example, but is not limited to, data from a barometric sensor or GPS. For example, when the vehicle is traveling in a mountainous area, the position information acquisition unit can acquire altitude information and provide three-dimensional position information. Furthermore, when the vehicle is traveling between high-rise buildings, the position information acquisition unit can acquire altitude information and provide accurate position information. Furthermore, when the vehicle is crossing a bridge, the position information acquisition unit can acquire altitude information and provide three-dimensional position information. Thus, by simultaneously acquiring altitude information, three-dimensional position information can be provided.

[0038] When acquiring the location information, the location information acquisition unit can propose an optimal route taking into account the remaining fuel level of the vehicle. The remaining fuel level includes, for example, but is not limited to, fuel sensor and fuel efficiency data. The optimal route includes, for example, but is not limited to, distance, time, and fuel efficiency. For example, when fuel is low, the location information acquisition unit can propose the shortest route to save fuel. Furthermore, when fuel is sufficient, the location information acquisition unit can also propose a scenic route to increase driver satisfaction. Furthermore, when fuel is moderate, the location information acquisition unit can also propose a route with a moderate distance to enable balanced driving. Thus, by proposing an optimal route taking into account the remaining fuel level, fuel can be saved and driver satisfaction can be increased.

[0039] When acquiring location information, the location information acquisition unit can select an optimal acquisition method by taking into consideration vehicle maintenance information. Maintenance information includes, for example, vehicle diagnostic data and maintenance history, but is not limited to these examples. The optimal acquisition method includes, for example, data type and acquisition frequency, but is not limited to these examples. For example, when vehicle maintenance is required, the location information acquisition unit can suggest the shortest route to enable prompt maintenance. Furthermore, when vehicle maintenance is not required, the location information acquisition unit can suggest a scenic route to increase driver satisfaction. Furthermore, when vehicle maintenance is moderate, the location information acquisition unit can suggest a route with a moderate distance to enable balanced driving. As a result, by selecting an optimal acquisition method by taking into consideration the maintenance information, vehicle maintenance can be performed efficiently.

[0040] When acquiring position information, the position information acquisition unit can adjust the frequency of acquiring position information taking into account the number of vehicle occupants. The number of occupants can be, for example, determined by seat sensors or camera recognition, but is not limited to these examples. The acquisition frequency can be, for example, determined by seconds or minutes, but is not limited to these examples. For example, when the number of occupants is large, the position information acquisition unit can increase the frequency of acquiring position information to provide accurate position information. Furthermore, when the number of occupants is small, the position information acquisition unit can reduce the frequency of acquiring position information to reduce the load on the system. Furthermore, when the number of occupants is medium, the position information acquisition unit can set the frequency of acquiring position information to a medium level to provide appropriate information. In this way, by adjusting the frequency of acquiring position information taking into account the number of occupants, more appropriate information can be provided.

[0041] When acquiring video, the video acquisition unit can select optimal camera settings taking into account the lighting conditions inside the vehicle. Examples of lighting conditions include, but are not limited to, an illuminance sensor and camera settings. Examples of optimal camera settings include, but are not limited to, exposure and white balance. For example, when the inside of the vehicle is dark, the video acquisition unit increases the sensitivity of the camera to acquire bright video. Furthermore, when the inside of the vehicle is bright, the video acquisition unit can also decrease the sensitivity of the camera to acquire appropriate video. Furthermore, when the lighting inside the vehicle fluctuates, the video acquisition unit can automatically adjust the sensitivity of the camera to acquire optimal video. In this way, more appropriate video can be acquired by selecting optimal camera settings taking into account the lighting conditions.

[0042] The video acquisition unit can simultaneously acquire audio information from within the vehicle when acquiring video, and perform an integrated analysis of the video and audio. Audio information includes, for example, but is not limited to, the type of microphone and an audio analysis algorithm. The integrated analysis includes, for example, but is not limited to, a data synchronization method and an analysis algorithm. The video acquisition unit, for example, acquires the audio of conversations within the vehicle and integrates it with the video for analysis. The video acquisition unit can also acquire environmental sounds within the vehicle and integrate it with the video for analysis. The video acquisition unit can also acquire audio instructions from within the vehicle and integrate it with the video for analysis. This allows for more detailed analysis by simultaneously acquiring audio information.

[0043] When acquiring video, the video acquisition unit can adjust the video acquisition method taking into account the air quality inside the vehicle. Examples of air quality include, but are not limited to, an air quality sensor and measurement items. Examples of acquisition methods include, but are not limited to, camera settings and shooting angles. For example, if the air quality inside the vehicle is poor, the video acquisition unit adjusts the camera filter to acquire clear video. Furthermore, if the air quality inside the vehicle is good, the video acquisition unit can also acquire video with normal camera settings. Furthermore, if the air quality inside the vehicle fluctuates, the video acquisition unit can automatically adjust the camera filter to acquire optimal video. Thus, by adjusting the video acquisition method taking into account the air quality, clearer video can be acquired.

[0044] When acquiring video, the video acquisition unit can select the optimal camera angle taking into account the seat arrangement inside the vehicle. Examples of seat arrangement include, but are not limited to, seat sensors and camera recognition. Examples of the optimal camera angle include, but are not limited to, the shooting angle and camera position. For example, when the seats are full, the video acquisition unit can use a wide-angle lens to acquire the entire video. Furthermore, when seats are empty, the video acquisition unit can also use a standard lens to acquire appropriate video. Furthermore, when the seat arrangement changes, the video acquisition unit can automatically adjust the camera angle to acquire optimal video. In this way, more appropriate video can be acquired by selecting the optimal camera angle taking into account the seat arrangement.

[0045] When acquiring video, the video acquisition unit can adjust the video acquisition method taking into account temperature information inside the vehicle. Temperature information includes, for example, but is not limited to, a temperature sensor, a measurement range, and accuracy. Acquisition methods include, for example, but are not limited to, camera settings and shooting angles. For example, when the temperature inside the vehicle is high, the video acquisition unit can adjust the camera settings to acquire clear video. Furthermore, when the temperature inside the vehicle is low, the video acquisition unit can also adjust the camera settings to acquire appropriate video. Furthermore, when the temperature inside the vehicle fluctuates, the video acquisition unit can automatically adjust the camera settings to acquire optimal video. In this way, by adjusting the video acquisition method taking into account temperature information, more appropriate video can be acquired.

[0046] When acquiring video, the video acquisition unit can adjust the video acquisition method taking into account humidity information inside the vehicle. Humidity information includes, for example, but is not limited to, a humidity sensor, a measurement range, and accuracy. Acquisition methods include, for example, but are not limited to, camera settings and shooting angles. For example, when the humidity inside the vehicle is high, the video acquisition unit can adjust the camera settings to acquire clear video. Furthermore, when the humidity inside the vehicle is low, the video acquisition unit can also adjust the camera settings to acquire appropriate video. Furthermore, when the humidity inside the vehicle fluctuates, the video acquisition unit can automatically adjust the camera settings to acquire optimal video. In this way, by adjusting the video acquisition method taking into account humidity information, more appropriate video can be acquired.

[0047] During image analysis, the image analysis unit can perform object recognition inside the vehicle to detect the presence or absence of dangerous objects. Object recognition includes, but is not limited to, examples of image processing algorithms and machine learning models. Dangerous objects include, but are not limited to, examples of weapons and flammable materials. For example, the image analysis unit performs object recognition and issues a warning if a dangerous object such as a knife is present inside the vehicle. The image analysis unit can also perform object recognition and issue a warning if a firearm is present inside the vehicle. The image analysis unit can also perform object recognition and issue a warning if a suspicious object is present inside the vehicle. Thus, by performing object recognition, dangerous objects inside the vehicle can be detected and a warning can be issued.

[0048] The image analysis unit performs motion detection inside the vehicle during image analysis, and can detect abnormal motion. Examples of motion detection include, but are not limited to, a motion sensor and an image analysis algorithm. Examples of abnormal motion include, but are not limited to, a pattern or speed of motion. For example, the image analysis unit issues a warning when abnormal motion is detected inside the vehicle. The image analysis unit can also issue a warning when suspicious motion is detected inside the vehicle. The image analysis unit can also issue a warning when sudden motion is detected inside the vehicle. Thus, by performing motion detection, abnormal motion inside the vehicle can be detected and a warning can be issued.

[0049] During image analysis, the image analysis unit can perform facial recognition inside the vehicle to identify occupants. Examples of facial recognition include, but are not limited to, image processing algorithms and machine learning models. Examples of occupant identification include, but are not limited to, facial recognition and ID authentication. For example, the image analysis unit can recognize the faces of people inside the vehicle and check whether they match registered occupants. The image analysis unit can also recognize the faces of people inside the vehicle and issue a warning if a suspicious person is present. The image analysis unit can also recognize the faces of people inside the vehicle and warn drivers to be extra cautious if children or elderly people are present. Thus, facial recognition can be used to identify occupants and detect suspicious people or people requiring special attention.

[0050] The image analysis unit can improve the accuracy of the analysis by integrating in-vehicle audio information during image analysis. Audio information includes, for example, but is not limited to, the type of microphone and the audio analysis algorithm. The integrated analysis includes, for example, but is not limited to, the data synchronization method and the analysis algorithm. The image analysis unit can, for example, integrate in-vehicle conversation audio into the analysis to improve the accuracy. The image analysis unit can also integrate in-vehicle environmental sounds into the analysis to improve the accuracy. The image analysis unit can also integrate in-vehicle audio instructions into the analysis to improve the accuracy. In this way, by integrating audio information, the analysis accuracy can be improved.

[0051] The image analysis unit can integrate interior temperature information during image analysis to improve analysis accuracy. Temperature information includes, for example, but is not limited to, temperature sensors, measurement ranges, and accuracy. Integrated analysis includes, for example, but is not limited to, data synchronization methods and analysis algorithms. For example, the image analysis unit can integrate interior temperature information into the analysis to detect abnormal temperature increases. The image analysis unit can also integrate interior temperature information into the analysis to evaluate passenger comfort. The image analysis unit can also integrate interior temperature information into the analysis to perform appropriate temperature management. By integrating temperature information, analysis accuracy can be improved.

[0052] The image analysis unit can improve the accuracy of the analysis by integrating humidity information about the vehicle interior during image analysis. Humidity information includes, for example, but is not limited to, humidity sensors, measurement ranges, and accuracy. Integrated analysis includes, for example, but is not limited to, data synchronization methods and analysis algorithms. For example, the image analysis unit can integrate humidity information about the vehicle interior into the analysis to detect abnormal humidity fluctuations. The image analysis unit can also integrate humidity information about the vehicle interior into the analysis to evaluate passenger comfort. The image analysis unit can also integrate humidity information about the vehicle interior into the analysis to perform appropriate humidity management. By integrating humidity information, the accuracy of the analysis can be improved.

[0053] During temperature monitoring, the temperature monitoring unit can individually monitor the temperature of each seat in the vehicle. Examples of temperature monitoring for each seat include, but are not limited to, a seat sensor and a temperature sensor. Examples of individual monitoring include, but are not limited to, the placement of sensors and the method of collecting data. For example, the temperature monitoring unit can individually monitor the temperature of each seat to detect an abnormal temperature rise. The temperature monitoring unit can also individually monitor the temperature of each seat to evaluate the comfort of the occupants. The temperature monitoring unit can also individually monitor the temperature of each seat to perform appropriate temperature management. This allows for more detailed temperature management by individually monitoring the temperature of each seat.

[0054] When monitoring the temperature, the temperature monitoring unit can analyze the temperature data taking into account the air quality inside the vehicle. The air quality includes, for example, an air quality sensor and measurement items, but is not limited to these examples. The temperature data analysis includes, for example, data preprocessing and analysis algorithms, but is not limited to these examples. For example, if the air quality inside the vehicle is poor, the temperature monitoring unit analyzes the temperature data and takes appropriate measures. Furthermore, if the air quality inside the vehicle is good, the temperature monitoring unit can analyze the temperature data and maintain a comfortable environment. Furthermore, if the air quality inside the vehicle fluctuates, the temperature monitoring unit can analyze the temperature data and take appropriate measures. Thus, analyzing the temperature data taking into account the air quality enables more appropriate temperature management.

[0055] During temperature monitoring, the temperature monitoring unit can analyze temperature data by integrating humidity information about the vehicle interior. The humidity information includes, for example, but is not limited to, humidity sensors, measurement ranges, and accuracy. The temperature data analysis includes, for example, but is not limited to, data preprocessing and analysis algorithms. For example, the temperature monitoring unit can analyze temperature data by integrating humidity information about the vehicle interior to detect abnormal temperature increases. The temperature monitoring unit can also analyze temperature data by integrating humidity information about the vehicle interior to evaluate passenger comfort. The temperature monitoring unit can also analyze temperature data by integrating humidity information about the vehicle interior to perform appropriate temperature management. As a result, more appropriate temperature management is possible by integrating humidity information and analyzing temperature data.

[0056] When monitoring the temperature, the temperature monitoring unit can analyze the temperature data taking into account the lighting conditions inside the vehicle. The lighting conditions include, for example, but are not limited to, an illuminance sensor and camera settings. The temperature data analysis includes, for example, but is not limited to, data preprocessing and analysis algorithms. For example, when the vehicle interior is dark, the temperature monitoring unit analyzes the temperature data and takes appropriate action. Furthermore, when the vehicle interior is bright, the temperature monitoring unit can also analyze the temperature data to maintain a comfortable environment. Furthermore, when the lighting conditions inside the vehicle fluctuate, the temperature monitoring unit can analyze the temperature data and take appropriate action. Thus, analyzing the temperature data taking into account the lighting conditions enables more appropriate temperature management.

[0057] During temperature monitoring, the temperature monitoring unit can analyze temperature data by integrating voice information from inside the vehicle. Voice information includes, for example, but is not limited to, the type of microphone and a voice analysis algorithm. Temperature data analysis includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the temperature monitoring unit can analyze temperature data by integrating voice information from inside the vehicle to detect an abnormal temperature rise. The temperature monitoring unit can also evaluate the comfort of occupants by integrating voice information from inside the vehicle to analyze temperature data and perform appropriate temperature management. As a result, more appropriate temperature management is possible by integrating voice information and analyzing temperature data.

[0058] During temperature monitoring, the temperature monitoring unit can analyze temperature data by integrating vibration information within the vehicle interior. The vibration information includes, for example, but is not limited to, a vibration sensor, a measurement range, and accuracy. The temperature data analysis includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the temperature monitoring unit can analyze temperature data by integrating vibration information within the vehicle interior to detect an abnormal temperature rise. The temperature monitoring unit can also analyze temperature data by integrating vibration information within the vehicle interior to evaluate the comfort of the occupants. The temperature monitoring unit can also analyze temperature data by integrating vibration information within the vehicle interior to perform appropriate temperature management. As a result, more appropriate temperature management is possible by integrating vibration information and analyzing temperature data.

[0059] When an abnormality is detected, the anomaly detection unit can perform object recognition inside the vehicle to detect the presence or absence of a dangerous object. Object recognition includes, but is not limited to, for example, an image processing algorithm or a machine learning model. Dangerous objects include, but are not limited to, for example, weapons and flammable materials. For example, the anomaly detection unit performs object recognition and issues a warning if a dangerous object such as a knife is present inside the vehicle. The anomaly detection unit can also perform object recognition and issue a warning if a firearm is present inside the vehicle. The anomaly detection unit can also perform object recognition and issue a warning if a suspicious object is present inside the vehicle. In this way, by performing object recognition, dangerous objects inside the vehicle can be detected and a warning can be issued.

[0060] When an abnormality is detected, the anomaly detection unit detects motion inside the vehicle and can detect abnormal motion. Examples of motion detection include, but are not limited to, a motion sensor and an image analysis algorithm. Examples of abnormal motion include, but are not limited to, a pattern or speed of motion. For example, the anomaly detection unit issues a warning when abnormal motion is detected inside the vehicle. The anomaly detection unit can also issue a warning when suspicious motion is detected inside the vehicle. The anomaly detection unit can also issue a warning when sudden motion is detected inside the vehicle. In this way, by performing motion detection, abnormal motion inside the vehicle can be detected and a warning can be issued.

[0061] When an abnormality is detected, the anomaly detection unit can perform facial recognition inside the vehicle to identify the occupant. Examples of facial recognition include, but are not limited to, image processing algorithms and machine learning models. Examples of occupant identification include, but are not limited to, facial recognition and ID authentication. For example, the anomaly detection unit can recognize the face of a person inside the vehicle and check whether it matches a registered occupant. The anomaly detection unit can also recognize the face of a person inside the vehicle and issue a warning if a suspicious person is present. The anomaly detection unit can also recognize the face of a person inside the vehicle and warn the driver to be extra careful if a child or elderly person is present. In this way, facial recognition can be used to identify the occupant and detect suspicious people or people who require special attention.

[0062] When detecting an abnormality, the anomaly detection unit can improve detection accuracy by integrating in-vehicle voice information. Voice information includes, for example, but is not limited to, the type of microphone and a voice analysis algorithm. Detection accuracy includes, for example, but is not limited to, data preprocessing and an analysis algorithm. The anomaly detection unit can, for example, integrate in-vehicle conversation voices into the analysis to improve accuracy. The anomaly detection unit can also integrate in-vehicle environmental sounds into the analysis to improve accuracy. The anomaly detection unit can also integrate in-vehicle voice instructions into the analysis to improve accuracy. In this way, by integrating voice information, detection accuracy can be improved.

[0063] When detecting an abnormality, the anomaly detection unit can improve detection accuracy by integrating temperature information inside the vehicle. The temperature information includes, for example, but is not limited to, a temperature sensor, a measurement range, and accuracy. The detection accuracy includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the anomaly detection unit integrates the temperature information inside the vehicle into its analysis to detect an abnormal temperature rise. The anomaly detection unit can also integrate the temperature information inside the vehicle into its analysis to evaluate the comfort of occupants. The anomaly detection unit can also integrate the temperature information inside the vehicle into its analysis to perform appropriate temperature management. In this way, by integrating the temperature information, detection accuracy can be improved.

[0064] When detecting an abnormality, the anomaly detection unit can improve detection accuracy by integrating humidity information inside the vehicle. Humidity information includes, for example, but is not limited to, humidity sensors, measurement ranges, and accuracy. Detection accuracy includes, for example, but is not limited to, data preprocessing and analysis algorithms. For example, the anomaly detection unit integrates humidity information inside the vehicle into its analysis to detect abnormal humidity fluctuations. The anomaly detection unit can also integrate humidity information inside the vehicle into its analysis to evaluate passenger comfort. The anomaly detection unit can also integrate humidity information inside the vehicle into its analysis to perform appropriate humidity management. In this way, by integrating humidity information, detection accuracy can be improved.

[0065] When providing a warning, the warning providing unit can select the optimal warning means by taking into consideration vehicle location information. Location information includes, but is not limited to, GPS, Wi-Fi, cell towers, etc. Warning means include, but are not limited to, audio warnings, visual warnings, smartphone notifications, etc. For example, the warning providing unit can prioritize audio warnings when the vehicle is on a highway. Furthermore, the warning providing unit can prioritize visual warnings when the vehicle is in an urban area. Furthermore, the warning providing unit can provide both audio and visual warnings when the vehicle is in a parking lot. This allows for more appropriate warnings by selecting the optimal warning means by taking into consideration location information.

[0066] When providing a warning, the warning providing unit can improve the accuracy of the warning by integrating in-vehicle voice information. The voice information includes, for example, but is not limited to, the type of microphone and a voice analysis algorithm. The accuracy of the warning includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the warning providing unit analyzes in-vehicle conversation voices and provides a warning at an appropriate time. The warning providing unit can also analyze in-vehicle environmental sounds and provide a warning at an appropriate volume. The warning providing unit can also analyze in-vehicle voice instructions and provide a warning with appropriate content. In this way, the accuracy of the warning can be improved by integrating the voice information.

[0067] When providing a warning, the warning providing unit can improve the accuracy of the warning by integrating temperature information inside the vehicle. The temperature information includes, for example, but is not limited to, a temperature sensor, a measurement range, and accuracy. The accuracy of the warning includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the warning providing unit analyzes the temperature information inside the vehicle and provides a warning when an abnormal temperature rise is detected. The warning providing unit can also analyze the temperature information inside the vehicle and evaluate the comfort of the occupants to provide a warning. The warning providing unit can also analyze the temperature information inside the vehicle and provide a warning for appropriate temperature management. In this way, the accuracy of the warning can be improved by integrating the temperature information.

[0068] When providing a warning, the warning providing unit may adjust the warning presentation method taking into account the lighting conditions inside the vehicle. Examples of lighting conditions include, but are not limited to, an illuminance sensor or camera settings. Examples of warning presentation methods include, but are not limited to, an audio warning, a visual warning, or a smartphone notification. For example, when the vehicle interior is dark, the warning providing unit may provide an emphasized visual warning. Furthermore, when the vehicle interior is bright, the warning providing unit may prioritize providing an audio warning. Furthermore, when the lighting conditions inside the vehicle fluctuate, the warning providing unit may provide both a visual and an audio warning. In this way, by adjusting the warning presentation method taking into account the lighting conditions, more appropriate warnings can be provided.

[0069] When providing a warning, the warning providing unit can improve the accuracy of the warning by integrating vibration information inside the vehicle. The vibration information includes, for example, but is not limited to, a vibration sensor, a measurement range, and accuracy. The accuracy of the warning includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the warning providing unit analyzes the vibration information inside the vehicle and provides a warning at an appropriate timing. The warning providing unit can also analyze the vibration information inside the vehicle and provide a warning with appropriate intensity. The warning providing unit can also analyze the vibration information inside the vehicle and provide a warning with appropriate content. In this way, the accuracy of the warning can be improved by integrating the vibration information.

[0070] When providing a warning, the warning providing unit may integrate humidity information about the vehicle interior to improve the accuracy of the warning. The humidity information may include, for example, but is not limited to, a humidity sensor, a measurement range, and accuracy. The accuracy of the warning may include, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the warning providing unit may analyze the humidity information about the vehicle interior and provide a warning when an abnormal humidity fluctuation is detected. The warning providing unit may also analyze the humidity information about the vehicle interior, evaluate the comfort of the occupants, and provide a warning. The warning providing unit may also analyze the humidity information about the vehicle interior and provide a warning for appropriate humidity management. In this way, the accuracy of the warning can be improved by integrating the humidity information.

[0071] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0072] The in-vehicle safety management system may further include a voice recognition unit. The voice recognition unit can analyze voices inside the vehicle and detect specific keywords or phrases. For example, if a child cries "help," the voice recognition unit detects the voice and notifies the warning provision unit. The voice recognition unit can also analyze conversations inside the vehicle and detect abnormal situations. For example, it can issue a warning if a child is crying or a pet is barking. This allows for the use of voice recognition to detect abnormalities inside the vehicle and respond quickly.

[0073] The in-vehicle safety management system may further include a vibration detection unit. The vibration detection unit can detect vehicle vibrations and analyze abnormal vibrations. For example, when the vehicle brakes suddenly or collides, the vibration detection unit detects the vibrations and notifies the warning provision unit. The vibration detection unit can also issue a warning when the vehicle is in an unstable state. This allows vehicle abnormalities to be detected using vibrations and a prompt response to be made.

[0074] The in-vehicle safety management system may further include a biometric authentication unit. The biometric authentication unit can acquire biometric information of the occupants in the vehicle and perform authentication. For example, fingerprint authentication or facial authentication can be used to confirm whether the person in the vehicle is a registered person. The biometric authentication unit can also acquire biometric information such as heart rate and body temperature and detect abnormalities. For example, it can issue a warning if the occupant's heart rate suddenly increases or if their body temperature is abnormally high. This allows the system to detect abnormalities in the vehicle using biometric information and respond quickly.

[0075] The vehicle interior safety management system may further include an air quality monitoring unit. The air quality monitoring unit can monitor the air quality inside the vehicle and detect abnormalities. For example, it measures the concentration of carbon dioxide or harmful substances and issues a warning if an abnormal value is detected. The air quality monitoring unit can also monitor the ventilation status inside the vehicle and prompt ventilation as necessary. This makes it possible to detect abnormalities inside the vehicle using air quality and respond quickly.

[0076] The vehicle interior safety management system may further include an optical sensor unit. The optical sensor unit can monitor the lighting conditions inside the vehicle and detect abnormalities. For example, it issues a warning if the vehicle interior is too dark or too bright. The optical sensor unit can also automatically adjust the lighting inside the vehicle. This allows the system to detect abnormalities inside the vehicle using the lighting conditions and respond quickly.

[0077] The processing flow of the first embodiment will be briefly explained below.

[0078] Step 1: The location information acquisition unit acquires location information of the vehicle. The location information acquisition unit can acquire location information using, for example, GPS, Wi-Fi, cell towers, etc. Step 2: The image acquisition unit acquires an image of the inside of the vehicle. The image acquisition unit can acquire an image by setting, for example, the type of camera, resolution, frame rate, etc. Step 3: The image analysis unit analyzes the video captured by the video capture unit to detect whether a person is present inside the vehicle. The image analysis unit can analyze the video using, for example, an image processing algorithm or a machine learning model. Step 4: The temperature monitoring unit measures the temperature inside the vehicle. The temperature monitoring unit can measure the temperature by setting, for example, the type of temperature sensor, the measurement range, the accuracy, etc. Step 5: The abnormality detection unit analyzes the temperature data measured by the temperature monitoring unit to detect an abnormal temperature rise. The abnormality detection unit can detect an abnormal temperature rise based on, for example, a temperature threshold value or a rate of rise. Step 6: The warning providing unit provides a warning based on the information obtained by the location information obtaining unit, the image analyzing unit, and the anomaly detecting unit. The warning providing unit can provide a warning by, for example, an audio warning, a visual warning, a smartphone notification, or the like.

[0079] (Example 2) An in-vehicle safety management system according to an embodiment of the present invention is a system for preventing accidents in which children are left unattended in a vehicle. The in-vehicle safety management system acquires vehicle location information, acquires video of the vehicle interior, analyzes the video to detect whether a person is inside the vehicle, measures the temperature inside the vehicle, detects an abnormal temperature rise, and provides a warning based on this information. For example, the in-vehicle safety management system acquires vehicle location information. Next, the in-vehicle safety management system acquires video of the vehicle interior, and an image analysis unit analyzes the video to detect whether a person is inside the vehicle. Furthermore, the in-vehicle safety management system measures the temperature inside the vehicle, and an abnormality detection unit analyzes the data to detect an abnormal temperature rise. This information is transmitted to a warning providing unit. If the driver leaves the vehicle while a child is inside the vehicle, the warning providing unit sends a warning to a smartphone. For example, if the driver moves a certain distance away from the vehicle, a warning message stating "A child has been left in the vehicle" is displayed on the smartphone. This allows the driver to immediately return to the vehicle and safely protect the child. In this way, the in-vehicle safety management system can prevent accidents in which children are left unattended in the vehicle. For example, if a child is left in a car on a hot summer day, the temperature inside the car can rise sharply, potentially putting their life at risk. This system monitors the temperature inside the car and sends an alert if an abnormal temperature rise is detected, ensuring the safety of the child. The system also constantly monitors whether anyone is left in the car, ensuring the safety of not only children, but also pets and the elderly. For example, if a pet is left in the car, a similar alert is sent, ensuring the pet's safety. In this way, the in-car safety management system is an effective means of preventing accidents involving children being left in cars, and by providing prompt warnings to drivers, it can ensure the safety of children.

[0080] An in-vehicle safety management system according to an embodiment includes a location information acquisition unit, a video acquisition unit, an image analysis unit, a temperature monitoring unit, an abnormality detection unit, and a warning provision unit. The location information acquisition unit acquires vehicle location information. The location information acquisition unit can acquire location information using, for example, GPS, Wi-Fi, a cell tower, etc. The video acquisition unit acquires video of the vehicle interior. The video acquisition unit can acquire video by setting, for example, the type of camera, resolution, frame rate, etc. The image analysis unit analyzes the video acquired by the video acquisition unit to detect whether a person is inside the vehicle. The image analysis unit can analyze the video by, for example, an image processing algorithm or a machine learning model. The temperature monitoring unit measures the temperature inside the vehicle. The temperature monitoring unit can measure the temperature by setting, for example, the type of temperature sensor, measurement range, accuracy, etc. The abnormality detection unit analyzes the temperature data measured by the temperature monitoring unit to detect an abnormal temperature rise. The abnormality detection unit can detect an abnormal temperature rise based on, for example, a temperature threshold value, a temperature rise rate, etc. The warning providing unit provides a warning based on the information obtained by the location information acquiring unit, the image analyzing unit, and the abnormality detecting unit. The warning providing unit can provide a warning by, for example, an audio warning, a visual warning, a smartphone notification, etc. As a result, the in-vehicle safety management system according to the embodiment is a system for preventing accidents in which children are left unattended in a vehicle.

[0081] The warning unit may send a warning message to the smartphone when the driver moves a certain distance away from the vehicle. The certain distance may be, for example, a distance in meters or GPS accuracy, but is not limited to these examples. The warning message may be, for example, a text message or a voice message, but is not limited to these examples. For example, the warning unit may send a warning message to the smartphone when the driver moves more than 10 meters away from the vehicle. The warning unit may also send a voice message when the driver moves more than 20 meters away from the vehicle. The warning unit may also provide a visual warning when the driver moves more than 30 meters away from the vehicle. This ensures the safety of children by sending a warning when the driver moves away from the vehicle.

[0082] The video acquisition unit acquires video of the interior of the vehicle, and the image analysis unit analyzes the video to detect whether a person is inside the vehicle. The video acquisition unit can acquire video by setting, for example, the type of camera, resolution, frame rate, etc. For example, the video acquisition unit acquires video of the interior of the vehicle using a high-resolution camera. The video acquisition unit can also acquire video of the entire interior of the vehicle using a wide-angle lens. The video acquisition unit can also acquire video even in dark places using an infrared camera. The image analysis unit can analyze the video using, for example, an image processing algorithm or a machine learning model. For example, the image analysis unit can detect whether a person is inside the vehicle using a face recognition algorithm. The image analysis unit can also detect movement inside the vehicle using a motion detection algorithm. The image analysis unit can also detect objects inside the vehicle using an object recognition algorithm. This makes it possible to accurately detect whether a person is inside the vehicle.

[0083] The temperature monitoring unit measures the temperature inside the vehicle, and the anomaly detection unit analyzes the measured data to detect an abnormal temperature rise. The temperature monitoring unit can measure the temperature by setting, for example, the type of temperature sensor, measurement range, accuracy, etc. For example, the temperature monitoring unit measures the temperature inside the vehicle using a high-precision temperature sensor. The temperature monitoring unit can also measure the temperature of various parts inside the vehicle using multiple temperature sensors. The temperature monitoring unit can also acquire temperature data in real time. The anomaly detection unit can detect an abnormal temperature rise based on, for example, a temperature threshold or a temperature rise rate. For example, the anomaly detection unit detects an abnormality when the temperature exceeds a certain threshold. The anomaly detection unit can also detect an abnormality when the temperature rises rapidly. The anomaly detection unit can also detect an abnormality by analyzing the temperature fluctuation pattern. This allows the system to detect an increase in the temperature inside the vehicle and issue a warning of an abnormal temperature rise.

[0084] The warning providing unit can also send a warning when a pet or an elderly person is left in the vehicle. Pets include, but are not limited to, for example, dogs and cats. Elderly people can be detected, for example, using face recognition or an age estimation algorithm. For example, the warning providing unit sends a warning when a pet is left in the vehicle. The warning providing unit can also send a warning when an elderly person is left in the vehicle. The warning providing unit can also send a voice warning when a pet or an elderly person is left in the vehicle. This can also ensure the safety of pets and elderly people.

[0085] The warning providing unit can also send a warning when an abnormal temperature rise is detected. Examples of abnormal temperature rise include, but are not limited to, a temperature threshold and a temperature rise rate. For example, the warning providing unit sends a warning when the temperature exceeds a certain threshold. The warning providing unit can also send a warning when the temperature rises rapidly. The warning providing unit can also send a warning when an abnormality is detected by analyzing a temperature fluctuation pattern. In this way, by sending a warning even when an abnormal temperature rise is detected, the safety of children can be ensured.

[0086] The location information acquisition unit can estimate the driver's emotions and adjust the frequency of location information acquisition based on the estimated driver's emotions. The emotions can be estimated using, for example, facial expression recognition or voice analysis. The acquisition frequency can be, for example, on a second-by-second or minute-by-minute basis, but is not limited to these examples. For example, if the driver is anxious, the location information acquisition unit can increase the frequency of location information acquisition to provide more accurate location information. Furthermore, if the driver is relaxed, the location information acquisition unit can reduce the frequency of location information acquisition to reduce the load on the system. Furthermore, if the driver is tired, the location information acquisition unit can set the frequency of location information acquisition to a medium level to provide appropriate information. This allows for more appropriate information to be provided by adjusting the frequency of location information acquisition according to the driver's emotions.

[0087] When acquiring vehicle position information, the position information acquisition unit can select the optimal acquisition timing by taking into consideration surrounding traffic conditions. Traffic conditions include, for example, but are not limited to, traffic sensors and real-time data. The optimal acquisition timing includes, for example, but are not limited to, traffic volume and traffic light timing. For example, when traffic congestion occurs, the position information acquisition unit increases the frequency of acquiring position information to provide real-time position information. Furthermore, when traffic is smooth, the position information acquisition unit can also reduce the frequency of acquiring position information to reduce the load on the system. Furthermore, when a traffic accident occurs, the position information acquisition unit can increase the frequency of acquiring position information to enable a quick response. Thus, by selecting the timing of acquiring position information by taking into consideration surrounding traffic conditions, more accurate position information can be provided.

[0088] When acquiring position information, the position information acquisition unit can improve the accuracy of the position information by taking into account the speed and direction of the vehicle. The speed includes, for example, but is not limited to, a speed sensor or GPS data. The direction includes, for example, but is not limited to, a compass or a gyro sensor. For example, when the vehicle is moving at high speed, the position information acquisition unit can increase the frequency of acquiring the position information to provide highly accurate position information. Furthermore, when the vehicle is moving at low speed, the position information acquisition unit can also reduce the frequency of acquiring the position information to reduce the load on the system. Furthermore, when the vehicle changes direction frequently, the position information acquisition unit can also increase the frequency of acquiring the position information to provide accurate position information. Thus, by improving the accuracy of the position information by taking into account the speed and direction of the vehicle, more accurate position information can be provided.

[0089] The position information acquisition unit can simultaneously acquire altitude information of the vehicle when acquiring the position information, and provide three-dimensional position information. Altitude information includes, for example, but is not limited to, data from a barometric sensor or GPS. For example, when the vehicle is traveling in a mountainous area, the position information acquisition unit can acquire altitude information and provide three-dimensional position information. Furthermore, when the vehicle is traveling between high-rise buildings, the position information acquisition unit can acquire altitude information and provide accurate position information. Furthermore, when the vehicle is crossing a bridge, the position information acquisition unit can acquire altitude information and provide three-dimensional position information. Thus, by simultaneously acquiring altitude information, three-dimensional position information can be provided.

[0090] The location information acquisition unit can estimate the driver's emotions and adjust the acquisition range of location information based on the estimated driver's emotions. The emotions can be estimated using, for example, facial expression recognition or voice analysis. The acquisition range can include, for example, a geographical range and accuracy, but is not limited to these examples. For example, if the driver is anxious, the location information acquisition unit can acquire location information over a wide range to enable a quick response. Furthermore, if the driver is relaxed, the location information acquisition unit can acquire location information over a narrow range to reduce the load on the system. Furthermore, if the driver is tired, the location information acquisition unit can acquire location information over a medium range to provide appropriate information. This allows for more appropriate information to be provided by adjusting the acquisition range of location information according to the driver's emotions.

[0091] When acquiring the location information, the location information acquisition unit can propose an optimal route taking into account the remaining fuel level of the vehicle. The remaining fuel level includes, for example, but is not limited to, fuel sensor and fuel efficiency data. The optimal route includes, for example, but is not limited to, distance, time, and fuel efficiency. For example, when fuel is low, the location information acquisition unit can propose the shortest route to save fuel. Furthermore, when fuel is sufficient, the location information acquisition unit can also propose a scenic route to increase driver satisfaction. Furthermore, when fuel is moderate, the location information acquisition unit can also propose a route with a moderate distance to enable balanced driving. Thus, by proposing an optimal route taking into account the remaining fuel level, fuel can be saved and driver satisfaction can be increased.

[0092] When acquiring location information, the location information acquisition unit can select an optimal acquisition method by taking into consideration vehicle maintenance information. Maintenance information includes, for example, vehicle diagnostic data and maintenance history, but is not limited to these examples. The optimal acquisition method includes, for example, data type and acquisition frequency, but is not limited to these examples. For example, when vehicle maintenance is required, the location information acquisition unit can suggest the shortest route to enable prompt maintenance. Furthermore, when vehicle maintenance is not required, the location information acquisition unit can suggest a scenic route to increase driver satisfaction. Furthermore, when vehicle maintenance is moderate, the location information acquisition unit can suggest a route with a moderate distance to enable balanced driving. As a result, by selecting an optimal acquisition method by taking into consideration the maintenance information, vehicle maintenance can be performed efficiently.

[0093] When acquiring position information, the position information acquisition unit can adjust the frequency of acquiring position information taking into account the number of vehicle occupants. The number of occupants can be, for example, determined by seat sensors or camera recognition, but is not limited to these examples. The acquisition frequency can be, for example, determined by seconds or minutes, but is not limited to these examples. For example, when the number of occupants is large, the position information acquisition unit can increase the frequency of acquiring position information to provide accurate position information. Furthermore, when the number of occupants is small, the position information acquisition unit can reduce the frequency of acquiring position information to reduce the load on the system. Furthermore, when the number of occupants is medium, the position information acquisition unit can set the frequency of acquiring position information to a medium level to provide appropriate information. In this way, by adjusting the frequency of acquiring position information taking into account the number of occupants, more appropriate information can be provided.

[0094] The video acquisition unit can estimate the driver's emotions and adjust the video acquisition frequency based on the estimated driver's emotions. The emotions can be estimated using, for example, facial expression recognition or voice analysis. The acquisition frequency can be, for example, on a second-by-second or minute-by-minute basis, but is not limited to these examples. For example, if the driver is anxious, the video acquisition unit can increase the video acquisition frequency to provide more accurate video information. Furthermore, if the driver is relaxed, the video acquisition unit can reduce the video acquisition frequency to reduce the load on the system. Furthermore, if the driver is tired, the video acquisition unit can set the video acquisition frequency to a medium level to provide appropriate information. This allows for more appropriate information to be provided by adjusting the video acquisition frequency according to the driver's emotions.

[0095] When acquiring video, the video acquisition unit can select optimal camera settings taking into account the lighting conditions inside the vehicle. Examples of lighting conditions include, but are not limited to, an illuminance sensor and camera settings. Examples of optimal camera settings include, but are not limited to, exposure and white balance. For example, when the inside of the vehicle is dark, the video acquisition unit increases the sensitivity of the camera to acquire bright video. Furthermore, when the inside of the vehicle is bright, the video acquisition unit can also decrease the sensitivity of the camera to acquire appropriate video. Furthermore, when the lighting inside the vehicle fluctuates, the video acquisition unit can automatically adjust the sensitivity of the camera to acquire optimal video. In this way, more appropriate video can be acquired by selecting optimal camera settings taking into account the lighting conditions.

[0096] The video acquisition unit can simultaneously acquire audio information from within the vehicle when acquiring video, and perform an integrated analysis of the video and audio. Audio information includes, for example, but is not limited to, the type of microphone and an audio analysis algorithm. The integrated analysis includes, for example, but is not limited to, a data synchronization method and an analysis algorithm. The video acquisition unit, for example, acquires the audio of conversations within the vehicle and integrates it with the video for analysis. The video acquisition unit can also acquire environmental sounds within the vehicle and integrate it with the video for analysis. The video acquisition unit can also acquire audio instructions from within the vehicle and integrate it with the video for analysis. This allows for more detailed analysis by simultaneously acquiring audio information.

[0097] When acquiring video, the video acquisition unit can adjust the video acquisition method taking into account the air quality inside the vehicle. Examples of air quality include, but are not limited to, an air quality sensor and measurement items. Examples of acquisition methods include, but are not limited to, camera settings and shooting angles. For example, if the air quality inside the vehicle is poor, the video acquisition unit adjusts the camera filter to acquire clear video. Furthermore, if the air quality inside the vehicle is good, the video acquisition unit can also acquire video with normal camera settings. Furthermore, if the air quality inside the vehicle fluctuates, the video acquisition unit can automatically adjust the camera filter to acquire optimal video. Thus, by adjusting the video acquisition method taking into account the air quality, clearer video can be acquired.

[0098] The image acquisition unit can estimate the driver's emotion and adjust the image resolution based on the estimated driver's emotion. The emotion can be estimated using, for example, facial expression recognition or voice analysis. The resolution can include, but is not limited to, the number of pixels and bit rate. For example, if the driver is anxious, the image acquisition unit can increase the image resolution to provide more detailed image information. Furthermore, if the driver is relaxed, the image acquisition unit can reduce the image resolution to reduce the load on the system. Furthermore, if the driver is tired, the image acquisition unit can set the image resolution to a medium level to provide appropriate information. This allows for more appropriate information to be provided by adjusting the image resolution according to the driver's emotion.

[0099] When acquiring video, the video acquisition unit can select the optimal camera angle taking into account the seat arrangement inside the vehicle. Examples of seat arrangement include, but are not limited to, seat sensors and camera recognition. Examples of the optimal camera angle include, but are not limited to, the shooting angle and camera position. For example, when the seats are full, the video acquisition unit can use a wide-angle lens to acquire the entire video. Furthermore, when seats are empty, the video acquisition unit can also use a standard lens to acquire appropriate video. Furthermore, when the seat arrangement changes, the video acquisition unit can automatically adjust the camera angle to acquire optimal video. In this way, more appropriate video can be acquired by selecting the optimal camera angle taking into account the seat arrangement.

[0100] When acquiring video, the video acquisition unit can adjust the video acquisition method taking into account temperature information inside the vehicle. Temperature information includes, for example, but is not limited to, a temperature sensor, a measurement range, and accuracy. Acquisition methods include, for example, but are not limited to, camera settings and shooting angles. For example, when the temperature inside the vehicle is high, the video acquisition unit can adjust the camera settings to acquire clear video. Furthermore, when the temperature inside the vehicle is low, the video acquisition unit can also adjust the camera settings to acquire appropriate video. Furthermore, when the temperature inside the vehicle fluctuates, the video acquisition unit can automatically adjust the camera settings to acquire optimal video. In this way, by adjusting the video acquisition method taking into account temperature information, more appropriate video can be acquired.

[0101] When acquiring video, the video acquisition unit can adjust the video acquisition method taking into account humidity information inside the vehicle. Humidity information includes, for example, but is not limited to, a humidity sensor, a measurement range, and accuracy. Acquisition methods include, for example, but are not limited to, camera settings and shooting angles. For example, when the humidity inside the vehicle is high, the video acquisition unit can adjust the camera settings to acquire clear video. Furthermore, when the humidity inside the vehicle is low, the video acquisition unit can also adjust the camera settings to acquire appropriate video. Furthermore, when the humidity inside the vehicle fluctuates, the video acquisition unit can automatically adjust the camera settings to acquire optimal video. In this way, by adjusting the video acquisition method taking into account humidity information, more appropriate video can be acquired.

[0102] The image analysis unit can estimate the driver's emotions and adjust the image analysis algorithm based on the estimated driver's emotions. The emotions can be estimated using, for example, facial expression recognition or voice analysis. The algorithm can include, for example, parameter setting and model selection, but is not limited to these examples. For example, if the driver is impatient, the image analysis unit can use an algorithm that performs a quick analysis. Furthermore, if the driver is relaxed, the image analysis unit can use an algorithm that performs a detailed analysis. Furthermore, if the driver is tired, the image analysis unit can use an algorithm that performs a balanced analysis. This allows for more appropriate analysis by adjusting the image analysis algorithm according to the driver's emotions.

[0103] During image analysis, the image analysis unit can perform object recognition inside the vehicle to detect the presence or absence of dangerous objects. Object recognition includes, but is not limited to, examples of image processing algorithms and machine learning models. Dangerous objects include, but are not limited to, examples of weapons and flammable materials. For example, the image analysis unit performs object recognition and issues a warning if a dangerous object such as a knife is present inside the vehicle. The image analysis unit can also perform object recognition and issue a warning if a firearm is present inside the vehicle. The image analysis unit can also perform object recognition and issue a warning if a suspicious object is present inside the vehicle. Thus, by performing object recognition, dangerous objects inside the vehicle can be detected and a warning can be issued.

[0104] The image analysis unit performs motion detection inside the vehicle during image analysis, and can detect abnormal motion. Examples of motion detection include, but are not limited to, a motion sensor and an image analysis algorithm. Examples of abnormal motion include, but are not limited to, a pattern or speed of motion. For example, the image analysis unit issues a warning when abnormal motion is detected inside the vehicle. The image analysis unit can also issue a warning when suspicious motion is detected inside the vehicle. The image analysis unit can also issue a warning when sudden motion is detected inside the vehicle. Thus, by performing motion detection, abnormal motion inside the vehicle can be detected and a warning can be issued.

[0105] During image analysis, the image analysis unit can perform facial recognition inside the vehicle to identify occupants. Examples of facial recognition include, but are not limited to, image processing algorithms and machine learning models. Examples of occupant identification include, but are not limited to, facial recognition and ID authentication. For example, the image analysis unit can recognize the faces of people inside the vehicle and check whether they match registered occupants. The image analysis unit can also recognize the faces of people inside the vehicle and issue a warning if a suspicious person is present. The image analysis unit can also recognize the faces of people inside the vehicle and warn drivers to be extra cautious if children or elderly people are present. Thus, facial recognition can be used to identify occupants and detect suspicious people or people requiring special attention.

[0106] The image analysis unit can estimate the driver's emotions and determine the priority of image analysis based on the estimated driver's emotions. Emotions can be estimated using, for example, facial expression recognition or voice analysis. Priorities include, but are not limited to, importance and urgency. For example, if the driver is anxious, the image analysis unit can prioritize important analysis. Furthermore, if the driver is relaxed, the image analysis unit can prioritize detailed analysis. Furthermore, if the driver is tired, the image analysis unit can prioritize balanced analysis. This allows for more appropriate analysis by determining the priority of image analysis according to the driver's emotions.

[0107] The image analysis unit can improve the accuracy of the analysis by integrating in-vehicle audio information during image analysis. Audio information includes, for example, but is not limited to, the type of microphone and the audio analysis algorithm. The integrated analysis includes, for example, but is not limited to, the data synchronization method and the analysis algorithm. The image analysis unit can, for example, integrate in-vehicle conversation audio into the analysis to improve the accuracy. The image analysis unit can also integrate in-vehicle environmental sounds into the analysis to improve the accuracy. The image analysis unit can also integrate in-vehicle audio instructions into the analysis to improve the accuracy. In this way, by integrating audio information, the analysis accuracy can be improved.

[0108] The image analysis unit can integrate interior temperature information during image analysis to improve analysis accuracy. Temperature information includes, for example, but is not limited to, temperature sensors, measurement ranges, and accuracy. Integrated analysis includes, for example, but is not limited to, data synchronization methods and analysis algorithms. For example, the image analysis unit can integrate interior temperature information into the analysis to detect abnormal temperature increases. The image analysis unit can also integrate interior temperature information into the analysis to evaluate passenger comfort. The image analysis unit can also integrate interior temperature information into the analysis to perform appropriate temperature management. By integrating temperature information, analysis accuracy can be improved.

[0109] The image analysis unit can improve the accuracy of the analysis by integrating humidity information about the vehicle interior during image analysis. Humidity information includes, for example, but is not limited to, humidity sensors, measurement ranges, and accuracy. Integrated analysis includes, for example, but is not limited to, data synchronization methods and analysis algorithms. For example, the image analysis unit can integrate humidity information about the vehicle interior into the analysis to detect abnormal humidity fluctuations. The image analysis unit can also integrate humidity information about the vehicle interior into the analysis to evaluate passenger comfort. The image analysis unit can also integrate humidity information about the vehicle interior into the analysis to perform appropriate humidity management. By integrating humidity information, the accuracy of the analysis can be improved.

[0110] The temperature monitoring unit can estimate the driver's emotions and adjust the frequency of temperature monitoring based on the estimated driver's emotions. The emotions can be estimated using, for example, facial expression recognition or voice analysis. The monitoring frequency can be, for example, every second or every minute, but is not limited to these examples. For example, if the driver is anxious, the temperature monitoring unit can increase the frequency of temperature monitoring to enable a prompt response. Furthermore, if the driver is relaxed, the temperature monitoring unit can reduce the frequency of temperature monitoring to reduce the load on the system. Furthermore, if the driver is tired, the temperature monitoring unit can set the frequency of temperature monitoring to a medium level and provide appropriate information. This allows for more appropriate information to be provided by adjusting the frequency of temperature monitoring according to the driver's emotions.

[0111] During temperature monitoring, the temperature monitoring unit can individually monitor the temperature of each seat in the vehicle. Examples of temperature monitoring for each seat include, but are not limited to, a seat sensor and a temperature sensor. Examples of individual monitoring include, but are not limited to, the placement of sensors and the method of collecting data. For example, the temperature monitoring unit can individually monitor the temperature of each seat to detect an abnormal temperature rise. The temperature monitoring unit can also individually monitor the temperature of each seat to evaluate the comfort of the occupants. The temperature monitoring unit can also individually monitor the temperature of each seat to perform appropriate temperature management. This allows for more detailed temperature management by individually monitoring the temperature of each seat.

[0112] When monitoring the temperature, the temperature monitoring unit can analyze the temperature data taking into account the air quality inside the vehicle. The air quality includes, for example, an air quality sensor and measurement items, but is not limited to these examples. The temperature data analysis includes, for example, data preprocessing and analysis algorithms, but is not limited to these examples. For example, if the air quality inside the vehicle is poor, the temperature monitoring unit analyzes the temperature data and takes appropriate measures. Furthermore, if the air quality inside the vehicle is good, the temperature monitoring unit can analyze the temperature data and maintain a comfortable environment. Furthermore, if the air quality inside the vehicle fluctuates, the temperature monitoring unit can analyze the temperature data and take appropriate measures. Thus, analyzing the temperature data taking into account the air quality enables more appropriate temperature management.

[0113] During temperature monitoring, the temperature monitoring unit can analyze temperature data by integrating humidity information about the vehicle interior. The humidity information includes, for example, but is not limited to, humidity sensors, measurement ranges, and accuracy. The temperature data analysis includes, for example, but is not limited to, data preprocessing and analysis algorithms. For example, the temperature monitoring unit can analyze temperature data by integrating humidity information about the vehicle interior to detect abnormal temperature increases. The temperature monitoring unit can also analyze temperature data by integrating humidity information about the vehicle interior to evaluate passenger comfort. The temperature monitoring unit can also analyze temperature data by integrating humidity information about the vehicle interior to perform appropriate temperature management. As a result, more appropriate temperature management is possible by integrating humidity information and analyzing temperature data.

[0114] The temperature monitoring unit can estimate the driver's emotions and determine the priority of temperature monitoring based on the estimated driver's emotions. Emotions can be estimated using, for example, facial expression recognition or voice analysis. Priorities include, but are not limited to, importance and urgency. For example, if the driver is anxious, the temperature monitoring unit can prioritize important temperature monitoring. Furthermore, if the driver is relaxed, the temperature monitoring unit can prioritize detailed temperature monitoring. Furthermore, if the driver is tired, the temperature monitoring unit can prioritize balanced temperature monitoring. This allows for more appropriate information to be provided by determining the priority of temperature monitoring according to the driver's emotions.

[0115] When monitoring the temperature, the temperature monitoring unit can analyze the temperature data taking into account the lighting conditions inside the vehicle. The lighting conditions include, for example, but are not limited to, an illuminance sensor and camera settings. The temperature data analysis includes, for example, but is not limited to, data preprocessing and analysis algorithms. For example, when the vehicle interior is dark, the temperature monitoring unit analyzes the temperature data and takes appropriate action. Furthermore, when the vehicle interior is bright, the temperature monitoring unit can also analyze the temperature data to maintain a comfortable environment. Furthermore, when the lighting conditions inside the vehicle fluctuate, the temperature monitoring unit can analyze the temperature data and take appropriate action. Thus, analyzing the temperature data taking into account the lighting conditions enables more appropriate temperature management.

[0116] During temperature monitoring, the temperature monitoring unit can analyze temperature data by integrating voice information from inside the vehicle. Voice information includes, for example, but is not limited to, the type of microphone and a voice analysis algorithm. Temperature data analysis includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the temperature monitoring unit can analyze temperature data by integrating voice information from inside the vehicle to detect an abnormal temperature rise. The temperature monitoring unit can also evaluate the comfort of occupants by integrating voice information from inside the vehicle to analyze temperature data and perform appropriate temperature management. As a result, more appropriate temperature management is possible by integrating voice information and analyzing temperature data.

[0117] During temperature monitoring, the temperature monitoring unit can analyze temperature data by integrating vibration information within the vehicle interior. The vibration information includes, for example, but is not limited to, a vibration sensor, a measurement range, and accuracy. The temperature data analysis includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the temperature monitoring unit can analyze temperature data by integrating vibration information within the vehicle interior to detect an abnormal temperature rise. The temperature monitoring unit can also analyze temperature data by integrating vibration information within the vehicle interior to evaluate the comfort of the occupants. The temperature monitoring unit can also analyze temperature data by integrating vibration information within the vehicle interior to perform appropriate temperature management. As a result, more appropriate temperature management is possible by integrating vibration information and analyzing temperature data.

[0118] The anomaly detection unit can estimate the driver's emotions and adjust the anomaly detection algorithm based on the estimated driver's emotions. The emotions can be estimated using, for example, facial expression recognition or voice analysis. The algorithm can include, for example, parameter setting and model selection, but is not limited to these examples. For example, if the driver is impatient, the anomaly detection unit can use an algorithm that performs quick anomaly detection. Furthermore, if the driver is relaxed, the anomaly detection unit can use an algorithm that performs detailed anomaly detection. Furthermore, if the driver is tired, the anomaly detection unit can use an algorithm that performs balanced anomaly detection. This allows for more appropriate anomaly detection by adjusting the anomaly detection algorithm according to the driver's emotions.

[0119] When an abnormality is detected, the anomaly detection unit can perform object recognition inside the vehicle to detect the presence or absence of a dangerous object. Object recognition includes, but is not limited to, for example, an image processing algorithm or a machine learning model. Dangerous objects include, but are not limited to, for example, weapons and flammable materials. For example, the anomaly detection unit performs object recognition and issues a warning if a dangerous object such as a knife is present inside the vehicle. The anomaly detection unit can also perform object recognition and issue a warning if a firearm is present inside the vehicle. The anomaly detection unit can also perform object recognition and issue a warning if a suspicious object is present inside the vehicle. In this way, by performing object recognition, dangerous objects inside the vehicle can be detected and a warning can be issued.

[0120] When an abnormality is detected, the anomaly detection unit detects motion inside the vehicle and can detect abnormal motion. Examples of motion detection include, but are not limited to, a motion sensor and an image analysis algorithm. Examples of abnormal motion include, but are not limited to, a pattern or speed of motion. For example, the anomaly detection unit issues a warning when abnormal motion is detected inside the vehicle. The anomaly detection unit can also issue a warning when suspicious motion is detected inside the vehicle. The anomaly detection unit can also issue a warning when sudden motion is detected inside the vehicle. In this way, by performing motion detection, abnormal motion inside the vehicle can be detected and a warning can be issued.

[0121] When an abnormality is detected, the anomaly detection unit can perform facial recognition inside the vehicle to identify the occupant. Examples of facial recognition include, but are not limited to, image processing algorithms and machine learning models. Examples of occupant identification include, but are not limited to, facial recognition and ID authentication. For example, the anomaly detection unit can recognize the face of a person inside the vehicle and check whether it matches a registered occupant. The anomaly detection unit can also recognize the face of a person inside the vehicle and issue a warning if a suspicious person is present. The anomaly detection unit can also recognize the face of a person inside the vehicle and warn the driver to be extra careful if a child or elderly person is present. In this way, facial recognition can be used to identify the occupant and detect suspicious people or people who require special attention.

[0122] The anomaly detection unit can estimate the driver's emotions and determine the priority of anomaly detection based on the estimated driver's emotions. The emotions can be estimated using, for example, facial expression recognition or voice analysis. The priority can include, for example, importance and urgency, but is not limited to these examples. For example, if the driver is impatient, the anomaly detection unit can prioritize important anomaly detection. Furthermore, if the driver is relaxed, the anomaly detection unit can prioritize detailed anomaly detection. Furthermore, if the driver is tired, the anomaly detection unit can prioritize balanced anomaly detection. Thus, by determining the priority of anomaly detection according to the driver's emotions, more appropriate anomaly detection becomes possible.

[0123] When detecting an abnormality, the anomaly detection unit can improve detection accuracy by integrating in-vehicle voice information. Voice information includes, for example, but is not limited to, the type of microphone and a voice analysis algorithm. Detection accuracy includes, for example, but is not limited to, data preprocessing and an analysis algorithm. The anomaly detection unit can, for example, integrate in-vehicle conversation voices into the analysis to improve accuracy. The anomaly detection unit can also integrate in-vehicle environmental sounds into the analysis to improve accuracy. The anomaly detection unit can also integrate in-vehicle voice instructions into the analysis to improve accuracy. In this way, by integrating voice information, detection accuracy can be improved.

[0124] When detecting an abnormality, the anomaly detection unit can improve detection accuracy by integrating temperature information inside the vehicle. The temperature information includes, for example, but is not limited to, a temperature sensor, a measurement range, and accuracy. The detection accuracy includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the anomaly detection unit integrates the temperature information inside the vehicle into its analysis to detect an abnormal temperature rise. The anomaly detection unit can also integrate the temperature information inside the vehicle into its analysis to evaluate the comfort of occupants. The anomaly detection unit can also integrate the temperature information inside the vehicle into its analysis to perform appropriate temperature management. In this way, by integrating the temperature information, detection accuracy can be improved.

[0125] When detecting an abnormality, the anomaly detection unit can improve detection accuracy by integrating humidity information inside the vehicle. Humidity information includes, for example, but is not limited to, humidity sensors, measurement ranges, and accuracy. Detection accuracy includes, for example, but is not limited to, data preprocessing and analysis algorithms. For example, the anomaly detection unit integrates humidity information inside the vehicle into its analysis to detect abnormal humidity fluctuations. The anomaly detection unit can also integrate humidity information inside the vehicle into its analysis to evaluate passenger comfort. The anomaly detection unit can also integrate humidity information inside the vehicle into its analysis to perform appropriate humidity management. In this way, by integrating humidity information, detection accuracy can be improved.

[0126] The warning providing unit may estimate the driver's emotion and adjust the warning expression method based on the estimated driver's emotion. The emotion may be estimated using, for example, facial expression recognition or voice analysis. The warning expression method may include, but is not limited to, an audio warning, a visual warning, or a smartphone notification. For example, the warning providing unit may provide a concise and quick warning if the driver is impatient. The warning providing unit may also provide a detailed warning if the driver is relaxed. The warning providing unit may also provide a balanced warning if the driver is tired. This allows for more appropriate warnings by adjusting the warning expression method according to the driver's emotion.

[0127] When providing a warning, the warning providing unit can select the optimal warning means by taking into consideration vehicle location information. Location information includes, but is not limited to, GPS, Wi-Fi, cell towers, etc. Warning means include, but are not limited to, audio warnings, visual warnings, smartphone notifications, etc. For example, the warning providing unit can prioritize audio warnings when the vehicle is on a highway. Furthermore, the warning providing unit can prioritize visual warnings when the vehicle is in an urban area. Furthermore, the warning providing unit can provide both audio and visual warnings when the vehicle is in a parking lot. This allows for more appropriate warnings by selecting the optimal warning means by taking into consideration location information.

[0128] When providing a warning, the warning providing unit can improve the accuracy of the warning by integrating in-vehicle voice information. The voice information includes, for example, but is not limited to, the type of microphone and a voice analysis algorithm. The accuracy of the warning includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the warning providing unit analyzes in-vehicle conversation voices and provides a warning at an appropriate time. The warning providing unit can also analyze in-vehicle environmental sounds and provide a warning at an appropriate volume. The warning providing unit can also analyze in-vehicle voice instructions and provide a warning with appropriate content. In this way, the accuracy of the warning can be improved by integrating the voice information.

[0129] When providing a warning, the warning providing unit can improve the accuracy of the warning by integrating temperature information inside the vehicle. The temperature information includes, for example, but is not limited to, a temperature sensor, a measurement range, and accuracy. The accuracy of the warning includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the warning providing unit analyzes the temperature information inside the vehicle and provides a warning when an abnormal temperature rise is detected. The warning providing unit can also analyze the temperature information inside the vehicle and evaluate the comfort of the occupants to provide a warning. The warning providing unit can also analyze the temperature information inside the vehicle and provide a warning for appropriate temperature management. In this way, the accuracy of the warning can be improved by integrating the temperature information.

[0130] The warning providing unit may estimate the driver's emotion and determine the priority of warnings based on the estimated driver's emotion. The emotion may be estimated using, for example, facial expression recognition or voice analysis. The priority may include, for example, importance or urgency, but is not limited to these examples. For example, the warning providing unit may prioritize providing an important warning when the driver is in a hurry. Furthermore, the warning providing unit may prioritize providing a detailed warning when the driver is relaxed. Furthermore, the warning providing unit may prioritize providing a balanced warning when the driver is tired. In this way, by determining the priority of warnings according to the driver's emotion, more appropriate warnings can be provided.

[0131] When providing a warning, the warning providing unit may adjust the warning presentation method taking into account the lighting conditions inside the vehicle. Examples of lighting conditions include, but are not limited to, an illuminance sensor or camera settings. Examples of warning presentation methods include, but are not limited to, an audio warning, a visual warning, or a smartphone notification. For example, when the vehicle interior is dark, the warning providing unit may provide an emphasized visual warning. Furthermore, when the vehicle interior is bright, the warning providing unit may prioritize providing an audio warning. Furthermore, when the lighting conditions inside the vehicle fluctuate, the warning providing unit may provide both a visual and an audio warning. In this way, by adjusting the warning presentation method taking into account the lighting conditions, more appropriate warnings can be provided.

[0132] When providing a warning, the warning providing unit can improve the accuracy of the warning by integrating vibration information inside the vehicle. The vibration information includes, for example, but is not limited to, a vibration sensor, a measurement range, and accuracy. The accuracy of the warning includes, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the warning providing unit analyzes the vibration information inside the vehicle and provides a warning at an appropriate timing. The warning providing unit can also analyze the vibration information inside the vehicle and provide a warning with appropriate intensity. The warning providing unit can also analyze the vibration information inside the vehicle and provide a warning with appropriate content. In this way, the accuracy of the warning can be improved by integrating the vibration information.

[0133] When providing a warning, the warning providing unit may integrate humidity information about the vehicle interior to improve the accuracy of the warning. The humidity information may include, for example, but is not limited to, a humidity sensor, a measurement range, and accuracy. The accuracy of the warning may include, for example, but is not limited to, data preprocessing and an analysis algorithm. For example, the warning providing unit may analyze the humidity information about the vehicle interior and provide a warning when an abnormal humidity fluctuation is detected. The warning providing unit may also analyze the humidity information about the vehicle interior, evaluate the comfort of the occupants, and provide a warning. The warning providing unit may also analyze the humidity information about the vehicle interior and provide a warning for appropriate humidity management. In this way, the accuracy of the warning can be improved by integrating the humidity information. === Hard Collateral 1-1 === Each of the multiple elements, including the location information acquisition unit, video acquisition unit, image analysis unit, temperature monitoring unit, abnormality detection unit, and warning provision unit, described above, is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the location information acquisition unit acquires vehicle location information using the GPS function of the smart device 14. The video acquisition unit acquires video of the interior of the vehicle using the camera 42 of the smart device 14. The image analysis unit analyzes the video using the specific processing unit 290 of the data processing device 12 to detect whether a person is inside the vehicle. The temperature monitoring unit measures the temperature inside the vehicle using the temperature sensor of the smart device 14. The abnormality detection unit analyzes the temperature data using the specific processing unit 290 of the data processing device 12 to detect an abnormal temperature rise. The warning provision unit provides a warning via the control unit 46A of the smart device 14. === Hard Collateral 1-2 === Each of the multiple elements, including the location information acquisition unit, video acquisition unit, image analysis unit, temperature monitoring unit, abnormality detection unit, and warning provision unit, described above, is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the location information acquisition unit acquires vehicle location information using the GPS function of the smart glasses 214. The video acquisition unit acquires video of the interior of the vehicle using the camera 42 of the smart glasses 214. The image analysis unit analyzes the video using the specific processing unit 290 of the data processing device 12 to detect whether a person is inside the vehicle. The temperature monitoring unit measures the temperature inside the vehicle using the temperature sensor of the smart glasses 214. The abnormality detection unit analyzes the temperature data using the specific processing unit 290 of the data processing device 12 to detect an abnormal temperature rise. The warning provision unit provides a warning via the control unit 46A of the smart glasses 214. === Hard Collateral 1-3 === Each of the multiple elements including the above-mentioned location information acquisition unit, video acquisition unit, image analysis unit, temperature monitoring unit, abnormality detection unit, and warning provision unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the location information acquisition unit acquires vehicle location information using the GPS function of the headset type terminal 314. The video acquisition unit acquires video of the interior of the vehicle using the camera 42 of the headset type terminal 314. The image analysis unit analyzes the video using the specific processing unit 290 of the data processing device 12 to detect whether a person is inside the vehicle. The temperature monitoring unit measures the temperature inside the vehicle using the temperature sensor of the headset type terminal 314. The abnormality detection unit analyzes the temperature data using the specific processing unit 290 of the data processing device 12 to detect an abnormal temperature rise. The warning provision unit provides a warning using the control unit 46A of the headset type terminal 314. === Hard Collateral 1-4 === Each of the multiple elements including the above-mentioned location information acquisition unit, video acquisition unit, image analysis unit, temperature monitoring unit, abnormality detection unit, and warning provision unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the location information acquisition unit acquires vehicle location information using the GPS function of the robot 414. The video acquisition unit acquires video of the interior of the vehicle using the camera 42 of the robot 414. The image analysis unit analyzes the video using the specific processing unit 290 of the data processing device 12 to detect whether a person is inside the vehicle. The temperature monitoring unit measures the temperature inside the vehicle using the temperature sensor of the robot 414. The abnormality detection unit analyzes the temperature data using the specific processing unit 290 of the data processing device 12 to detect an abnormal temperature rise. The warning provision unit provides a warning via the control unit 46A of the robot 414.

[0134] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0135] The in-vehicle safety management system may further include a voice recognition unit. The voice recognition unit can analyze voices inside the vehicle and detect specific keywords or phrases. For example, if a child cries "help," the voice recognition unit detects the voice and notifies the warning provision unit. The voice recognition unit can also analyze conversations inside the vehicle and detect abnormal situations. For example, it can issue a warning if a child is crying or a pet is barking. This allows for the use of voice recognition to detect abnormalities inside the vehicle and respond quickly.

[0136] The in-vehicle safety management system may further include a vibration detection unit. The vibration detection unit can detect vehicle vibrations and analyze abnormal vibrations. For example, when the vehicle brakes suddenly or collides, the vibration detection unit detects the vibrations and notifies the warning provision unit. The vibration detection unit can also issue a warning when the vehicle is in an unstable state. This allows vehicle abnormalities to be detected using vibrations and a prompt response to be made.

[0137] The in-vehicle safety management system may further include a biometric authentication unit. The biometric authentication unit can acquire biometric information of the occupants in the vehicle and perform authentication. For example, fingerprint authentication or facial authentication can be used to confirm whether the person in the vehicle is a registered person. The biometric authentication unit can also acquire biometric information such as heart rate and body temperature and detect abnormalities. For example, it can issue a warning if the occupant's heart rate suddenly increases or if their body temperature is abnormally high. This allows the system to detect abnormalities in the vehicle using biometric information and respond quickly.

[0138] The vehicle interior safety management system may further include an air quality monitoring unit. The air quality monitoring unit can monitor the air quality inside the vehicle and detect abnormalities. For example, it measures the concentration of carbon dioxide or harmful substances and issues a warning if an abnormal value is detected. The air quality monitoring unit can also monitor the ventilation status inside the vehicle and prompt ventilation as necessary. This makes it possible to detect abnormalities inside the vehicle using air quality and respond quickly.

[0139] The vehicle interior safety management system may further include an optical sensor unit. The optical sensor unit can monitor the lighting conditions inside the vehicle and detect abnormalities. For example, it issues a warning if the vehicle interior is too dark or too bright. The optical sensor unit can also automatically adjust the lighting inside the vehicle. This allows the system to detect abnormalities inside the vehicle using the lighting conditions and respond quickly.

[0140] The in-vehicle safety management system may further include an emotion estimation unit. The emotion estimation unit can analyze the facial expressions and voices of the occupants in the vehicle to estimate their emotions. For example, if a child is feeling anxious or scared, the emotion estimation unit detects that emotion and notifies the warning provision unit. The emotion estimation unit can also monitor the stress levels of the occupants and issue a warning if abnormal stress is detected. This makes it possible to detect abnormalities in the vehicle using emotions and respond quickly.

[0141] The in-vehicle safety management system may further include an audio feedback unit. The audio feedback unit may estimate the emotions of the occupants and provide audio feedback according to the emotions. For example, if a child feels anxious, the audio feedback unit may provide an audio message to reassure the occupants. Furthermore, if the occupants are relaxed, the audio feedback unit may play music to help them maintain their relaxation. This allows the occupants' comfort to be improved by providing audio feedback according to their emotions.

[0142] The in-vehicle safety management system may further include an emotion database unit. The emotion database unit accumulates emotional data of occupants and can infer emotions based on past data. For example, it may record situations in which a child felt anxious in the past and issue a warning if a similar situation occurs. The emotion database unit may also analyze the emotional patterns of occupants and detect abnormal emotional changes. This allows emotions to be inferred using past data and a prompt response can be made.

[0143] The in-vehicle safety management system may further include an emotion prediction unit. The emotion prediction unit can predict future emotions based on the current emotions of the occupants. For example, if a child is gradually beginning to feel anxious, a warning can be issued before the anxiety escalates. The emotion prediction unit can also monitor changes in the emotions of the occupants in real time and issue a warning if an abnormal change is predicted. This makes it possible to detect abnormalities in the vehicle using emotion prediction and respond quickly.

[0144] The in-vehicle safety management system may further include an emotion feedback unit. The emotion feedback unit may estimate the emotion of the occupant and provide feedback according to the emotion. For example, if a child feels anxious, the emotion feedback unit may provide visual feedback to reassure the occupant. Furthermore, if the occupant is relaxed, the emotion feedback unit may set the environment to maintain relaxation. In this way, by providing feedback according to the emotion, the comfort of the occupant can be improved.

[0145] The processing flow of the second embodiment will be briefly explained below.

[0146] Step 1: The location information acquisition unit acquires location information of the vehicle. The location information acquisition unit can acquire location information using, for example, GPS, Wi-Fi, cell towers, etc. Step 2: The image acquisition unit acquires an image of the inside of the vehicle. The image acquisition unit can acquire an image by setting, for example, the type of camera, resolution, frame rate, etc. Step 3: The image analysis unit analyzes the video captured by the video capture unit to detect whether a person is present inside the vehicle. The image analysis unit can analyze the video using, for example, an image processing algorithm or a machine learning model. Step 4: The temperature monitoring unit measures the temperature inside the vehicle. The temperature monitoring unit can measure the temperature by setting, for example, the type of temperature sensor, the measurement range, the accuracy, etc. Step 5: The abnormality detection unit analyzes the temperature data measured by the temperature monitoring unit to detect an abnormal temperature rise. The abnormality detection unit can detect an abnormal temperature rise based on, for example, a temperature threshold value or a rate of rise. Step 6: The warning providing unit provides a warning based on the information obtained by the location information obtaining unit, the image analyzing unit, and the anomaly detecting unit. The warning providing unit can provide a warning by, for example, an audio warning, a visual warning, a smartphone notification, or the like.

[0147] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0148] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0149] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0150] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0151] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0152] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0153] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0154] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0155] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0156] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0157] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0158] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0159] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0160] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0161] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0162] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0163] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0164] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0165] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0166] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0167] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0168] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0169] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0170] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0171] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0172] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0173] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0174] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0175] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0176] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0177] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0178] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0179] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0180] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0181] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0182] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0183] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0184] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0185] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0186] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0187] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0188] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0189] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0190] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0191] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0192] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0193] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0194] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0195] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0196] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0197] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0198] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0199] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0200] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0201] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0202] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0203] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0204] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0205] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0206] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0207] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0208] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0209] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0210] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0211] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0212] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0213] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0214] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0215] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0216] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0217] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0218] [Explanation of symbols]

[0219] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a location information acquisition unit that acquires location information of a vehicle; an image acquisition unit that acquires an image of the inside of the vehicle; an image analysis unit that analyzes the image acquired by the image acquisition unit to detect whether or not there is a person inside the vehicle; a temperature monitoring unit for measuring the temperature inside the vehicle; an abnormality detection unit that analyzes the temperature data measured by the temperature monitoring unit and detects an abnormal temperature rise; a warning providing unit that provides a warning based on the information obtained by the position information obtaining unit, the image analyzing unit, and the abnormality detecting unit. A system characterized by:

2. The warning providing unit Sends a warning message to a driver's smartphone if the driver moves a certain distance away from the vehicle 2. The system of claim 1.

3. The image acquisition unit Capture footage from inside the car The image analysis unit analyzes the video to detect whether there is a person inside the vehicle.

2. The system of claim 1.

4. The temperature monitoring unit Measure the temperature inside the car The abnormality detection unit analyzes the data and detects an abnormal temperature rise.

2. The system of claim 1.

5. The warning providing unit It also sends alerts if pets or elderly people are left in the car.

2. The system of claim 1.

6. The warning providing unit It also sends an alert if an abnormal temperature rise is detected.

2. The system of claim 1.

7. The location information acquisition unit Estimate the driver's emotions and adjust the frequency of location information acquisition based on the estimated driver's emotions.

2. The system of claim 1.

8. The location information acquisition unit When acquiring vehicle location information, the optimal acquisition timing is selected taking into account surrounding traffic conditions.

2. The system of claim 1.

Images