system

Abstract

We provide the system. [Solution] The system includes multiple radio wave transceivers as a means of detecting people, and a means of acquiring radio wave reflection information. A means for analyzing acquired reflection information to estimate the presence of an object within the surveillance space and its biological signals, A communication means for transmitting the analyzed data to an information processing device that receives it, Based on the received data, a means of analyzing the situation in the monitored space using artificial intelligence and evaluating potential risks, A means of sending a warning to a registered recipient when the potential risk exceeds a set threshold, A system that includes this.

Description

Technical Field

[0001] The technology of this disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Accidents caused by leaving people in a vehicle are a problem that can particularly lead to serious consequences for children. Conventional camera-based monitoring systems have visual blind spots and cannot always accurately detect people. In addition, there is a lack of a system that can evaluate physical risks in real time due to changes in heart rate and vehicle interior environment and prompt appropriate actions. As a result, a new method for preventing risks caused by leaving people unattended is required.

Means for Solving the Problems

[0005] This invention provides a means for detecting the presence of a person by placing multiple radio wave transceivers inside a vehicle and using the reflection of radio waves. It also includes a communication means for estimating heart rate using the same radio waves and transmitting the data to a server. The server analyzes the received data using artificial intelligence and evaluates the danger of the in-vehicle environment. If the evaluation result is determined to be dangerous, the system provides a means for immediately sending a notification to registered contacts, enabling a rapid response. Furthermore, by having a function to determine the environmental conditions by measuring the temperature inside the vehicle and by including specific danger information in the notification content, it is effective in preventing accidents caused by people being left unattended.

[0006] "Person detection means" refers to a device or method for confirming the presence of a person inside a vehicle using radio waves.

[0007] A "radio wave transceiver" is a device for transmitting and receiving radio waves, and is a means of collecting environmental information by utilizing the reflection of radio waves within a vehicle.

[0008] "Reflection data" is information generated based on signals that are reflected back from an object, and by analyzing this data, it is possible to understand the presence and dynamic characteristics of a person.

[0009] "Analysis means" refers to a process or device that performs processing to estimate the presence and heart rate of a person based on acquired data.

[0010] A "server device" is a computer device that receives data transmitted from a vehicle and processes and evaluates information based on that data.

[0011] "Communication methods" is a general term for protocols and infrastructure used to transmit or receive data to or from a specific device.

[0012] Artificial intelligence is a computer system that uses large amounts of data to execute machine learning algorithms and analyze and judge environmental conditions.

[0013] A "risk assessment method" is a process of quantifying or qualitatively determining the degree of risk based on the environment and human data inside the vehicle.

[0014] "Notification means" refers to a method or device for transmitting warnings or information to designated recipients under specific conditions.

[0015] "Vehicle interior temperature measurement" is a process for quantitatively obtaining the temperature inside a vehicle and evaluating environmental conditions based on that data. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the language used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system aimed at preventing accidents caused by people being left unattended inside vehicles. This system is implemented using a radio wave transceiver and server device installed inside the vehicle, and a user communication terminal equipped with a notification function.

[0038] Person detection using devices

[0039] The terminal uses a radio transceiver installed inside the vehicle to transmit radio waves into the vehicle's interior space and acquires the reflected data. This data forms the basis for determining whether a person is present. By using wireless technology such as Wi-Fi, it is possible to overcome visual limitations and detect people without blind spots.

[0040] Heart rate estimation by the device

[0041] The device further analyzes subtle movements from reflective data to estimate a person's heart rate. This kind of biometric information is particularly important for the early detection of health risks in hot environments.

[0042] Server-based data analysis

[0043] The analyzed data is sent to a server. The server uses artificial intelligence algorithms to analyze the data in real time and assess the risk based on the in-vehicle environment and biometric information. For example, if the in-vehicle temperature exceeds a set safety range or if there is an abnormality in the heart rate, the system immediately recognizes the risk.

[0044] Notification and alert features

[0045] When the server detects a threat, it automatically sends a notification containing specific threat information to registered contacts. The user's communication device receives this notification and issues a visual or audible alert, allowing the user to respond quickly.

[0046] Specific example

[0047] The device automatically begins scanning when the vehicle is parked. For example, if a child is left unattended in the car, the server detects a rise in temperature and a change in heart rate inside the vehicle, and as a result sends an alert to the driver's smartphone. Upon receiving the alert, the user can quickly return to the car and move the child to a safe place.

[0048] In this way, the present invention provides a practical and effective solution for ensuring the safety of people inside a vehicle and preventing unfortunate accidents caused by being left behind.

[0049] The following describes the processing flow.

[0050] Step 1:

[0051] The terminal begins transmitting radio waves when it detects that the vehicle's engine has stopped and the doors have been locked. It receives the reflected radio waves and acquires environmental information data from inside the vehicle.

[0052] Step 2:

[0053] The device analyzes the acquired reflection data to identify the presence of a person. It also estimates the heart rate of a person inside the vehicle by detecting subtle movements. This analyzed data is temporarily stored in internal memory.

[0054] Step 3:

[0055] The device transmits the presence data and heart rate data of the analyzed person to a server via its built-in communication device. Communication takes place periodically, and the data is updated in real time.

[0056] Step 4:

[0057] The server processes the received data in real time and uses artificial intelligence to assess the risks of the in-car environment. Specifically, it determines whether the heart rate is abnormal and whether the in-car temperature has reached a dangerous level.

[0058] Step 5:

[0059] If the server determines that the set danger threshold has been reached, it will generate a notification containing information about the dangers inside the vehicle for the registered contacts.

[0060] Step 6:

[0061] The server sends the generated notification to the user's communication device. This notification contains details of the risk and urges the user to take immediate action.

[0062] Step 7:

[0063] The user receives a notification on their communication device, and an alert is issued. Upon acknowledging the alert, the user immediately returns to their vehicle and begins the safety check procedure.

[0064] (Example 1)

[0065] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0066] Conventional vehicle safety systems have struggled to reliably detect the presence of people and precisely evaluate their biological activity. Furthermore, there is a need to quickly assess risks based on environmental conditions and issue appropriate warnings. Therefore, improvements are needed to ensure the safety of people inside vehicles, particularly to prevent heatstroke and other health risks.

[0067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0068] In this invention, the server includes means for transmitting radio waves to detect the presence of a person and acquiring reflected data, means for analyzing the acquired reflected data to estimate the presence and biological activity of a person, and means for sending a warning to a registered communication destination when the assessed risk exceeds a certain standard. This enables high-precision safety of a person inside a vehicle and allows for rapid risk assessment and warning issuance.

[0069] "Radio waves that detect the presence of a person" are wireless signals that are transmitted and received to identify a person inside a vehicle.

[0070] "Reflection data" refers to a collection of information acquired after radio waves strike a person or object and are reflected back.

[0071] A "means for estimating biological activity" refers to a system that analyzes reflection data to calculate biological values ​​related to a person, such as heart rate.

[0072] A "central processing unit" is a processing unit that receives, analyzes, and evaluates all data transmitted from a vehicle.

[0073] A "machine learning model" is an algorithm that uses large amounts of data to recognize specific patterns and analyze information.

[0074] "Means of risk assessment" refers to the process of determining whether the conditions inside a vehicle are safe or dangerous by using biometric activity and environmental data.

[0075] "A means of issuing warnings" refers to a function that automatically sends alerts to registered recipients when a risk that threatens a person's safety is detected.

[0076] "Environmental parameters" are data used to monitor physical conditions that may affect safety, such as temperature and humidity inside a vehicle.

[0077] A "communication destination" is a contact that has been pre-configured to receive a warning in the event of a risk.

[0078] This invention is a system for ensuring the safety of people inside vehicles and preventing accidents caused by being left behind. This system is implemented as follows.

[0079] The terminal uses a radio wave transceiver installed inside the vehicle to transmit radio waves throughout the vehicle. These radio waves reflect off people and objects, and the data is acquired by the terminal. This reflected data is important basic information for detecting the presence of people. By utilizing wireless technologies such as Wi-Fi, the terminal can eliminate blind spots inside the vehicle and detect people with high accuracy without relying on vision.

[0080] Furthermore, the device analyzes reflective data from subtle movements to estimate a person's heart rate. Heart rate, a form of biometric information, is particularly important for quickly identifying health risks, especially in high-temperature environments. This estimation utilizes advanced signal processing technology.

[0081] The analyzed data is sent to the server in real time. The server uses machine learning models to analyze the environmental conditions in detail from the received data and assess the risks. For example, the server has the ability to automatically recognize dangers such as when the temperature inside the car exceeds a safe range or when the heart rate shows an abnormality.

[0082] When a danger is detected, the server sends a notification containing the danger information to registered contacts. The user's communication device receives this notification and is presented with an alert visually or audibly. This allows the user to respond quickly and ensure the safety of anyone left inside the vehicle.

[0083] As a concrete example, the device has a function that automatically starts scanning when a vehicle is parked. For instance, if a child is left unattended in the car, the server will detect a rise in the interior temperature or a change in heart rate and send an alert to the driver's mobile phone. After receiving the alert, the user can quickly return to the vehicle and move the child to a safe place.

[0084] An example of an input prompt for a generated AI model is, "Explain how the system works to prevent people from being left behind, based on data acquired from sensors installed inside the vehicle."

[0085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0086] Step 1:

[0087] The terminal transmits radio waves using a radio transceiver installed inside the vehicle. Input requires setting the radio wave frequency and intensity. These radio waves fill the vehicle and reflect off people and objects. Specifically, the terminal continuously transmits radio waves at regular intervals to constantly obtain the latest reflection data.

[0088] Step 2:

[0089] The device acquires reflected radio wave data. The input is the signal data of the reflected radio waves. This data is analyzed and used as basic information to determine the presence of a person. The device applies a machine learning algorithm to perform the specific operation of detecting the presence or absence of a person based on the reflected data. The output is the result of the determination of whether or not a person is present.

[0090] Step 3:

[0091] The device analyzes minute movements in reflected data to estimate a person's heart rate. The input is reflected data that reflects minute movements. The device performs signal processing and aims to capture periodic changes corresponding to the heart rate. The output is the estimated heart rate value.

[0092] Step 4:

[0093] Biometric information and person presence data analyzed from the terminal are transmitted to the server in real time. The input is the analyzed data from the terminal. The server receives this and immediately begins the next processing.

[0094] Step 5:

[0095] The server uses machine learning models to analyze the received data and assess the risks based on the in-vehicle environmental conditions and biometric information. Inputs include analyzed biometric data and environmental data (e.g., temperature). The server uses multiple algorithms to calculate the risk under specific conditions and determine the level of danger. The output is a result regarding the presence and degree of danger.

[0096] Step 6:

[0097] The server sends an alert to registered contacts if it determines that the risk exceeds a certain threshold. The input is the risk assessment result from step 5. Specifically, the server uses an automated message generation function to generate an appropriate alert message and sends it over the communication network. The output is the alert notification sent to the recipient.

[0098] Step 7:

[0099] The user's communication terminal receives a warning notification from the server and issues an alert. The input is the warning message received from the server. The user can then check the audio or visual alert on their terminal and take the necessary action. The output is the alert received by the user.

[0100] (Application Example 1)

[0101] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0102] Traditional store and facility management methods have made it difficult to detect the risks associated with unattended individuals or sudden illnesses after closing hours or during unattended periods. Leaving this problem unaddressed increases the risk of accidents in unattended spaces, making it difficult to ensure safety. Therefore, there is a need for a system that ensures safety within spaces even during unattended hours and allows for rapid response.

[0103] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0104] In this invention, the server includes means for acquiring radio wave reflection information within a monitored space using a radio wave transceiver as a means for detecting people, means for estimating and analyzing biological signals from the acquired reflection information, and communication means for evaluating potential risks based on the analyzed information and transmitting it to a recipient along with a warning. This enables rapid detection of the presence of people or the risk of sudden illness within a monitored space during unmanned hours, and allows for immediate response.

[0105] A "person detection system" is a mechanism that uses radio wave transceivers to acquire reflected radio wave information and detect the presence of an object within a monitored space.

[0106] A "radio wave transceiver" is a device that transmits radio waves and receives information obtained from their reflection. This technology allows for the identification of objects in space using this information.

[0107] "Reflection information" refers to data obtained when radio waves strike objects within a monitored space and are reflected back. This data forms the basis for understanding the situation and presence of objects within that space.

[0108] "Biosignals" are indicators of the presence of life, such as human heart rate and respiration, and are data from which information can be estimated from subtle movements.

[0109] An "information processing device" is a system that receives acquired data, analyzes its contents using artificial intelligence, evaluates the situation in a monitored space, and determines whether or not there are potential risks.

[0110] "Potential hazards" refer to risks such as accidents or sudden illnesses within the monitored space, and when these exceed a certain threshold, safety measures become necessary.

[0111] "Communication method" refers to a method for sending warnings and details based on analysis results to registered recipients.

[0112] To implement this invention, it is necessary to install a radio wave transceiver that serves as a means for detecting people in the surveillance space. The radio wave transceiver transmits radio waves and monitors the reflected information. This reflected information is important for determining the presence of objects or people in the space, and the obtained information is transmitted to an information processing device.

[0113] The server functions as an information processing device, analyzing subtle movements from received reflected information to estimate biological signals. This process utilizes AI algorithms and data analysis using software such as TENSORFLOW® and PyTorch. Based on the analysis, it assesses potential hazards in the space, and if the level is determined to exceed a certain threshold, it sends detailed information, including a warning, to registered recipients.

[0114] The user's communication device receives warnings sent from the server and provides notifications visually or audibly. This allows the user to immediately recognize the situation and take appropriate action.

[0115] As a concrete example, in an unmanned store, if an employee loses consciousness after closing time, a radio transmitter / receiver will detect their presence, and the server will analyze any abnormalities related to their heart rate. Another example of a prompt message is, "If a person is detected in the store after closing time, how will an alert be quickly sent to the administrator?"

[0116] Through these processes, the invention provides a system that enhances security in surveillance spaces and enables rapid risk response.

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The terminal activates a radio transceiver and transmits radio waves within the monitored space. The input is the environment within the space, and the output is the reflected radio wave information. The terminal acquires this reflected information and collects it as data.

[0120] Step 2:

[0121] The server analyzes the reflected information received from the terminal. Here, the input is radio wave reflected information, and the output is the estimated result of biological signals based on subtle movements. The server uses an AI algorithm to estimate the presence of a person and their biological signals (e.g., heart rate) from this data.

[0122] Step 3:

[0123] The server assesses potential risks based on analyzed biosignals and on-site data. Inputs are estimated biosignals and environmental data such as temperature, while output is the risk level. The server performs this assessment against criteria set by an AI model to determine if the potential risk exceeds the threshold.

[0124] Step 4:

[0125] If the server determines, based on its assessment, that the potential risk exceeds the threshold, it will send a warning to the registered recipients. The input is the risk assessment result, and the output is the warning notification. The server generates the message and sends it to the user's communication terminal.

[0126] Step 5:

[0127] The user's communication terminal receives a warning notification from the server and displays an alert to the user visually or audibly. The input is the warning notification data, and the output is the notification to the user. The user reviews this information and takes appropriate action depending on the situation.

[0128] This system enables early detection of risks and the implementation of safety measures even in unmanned monitoring spaces.

[0129] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0130] This invention is a system that comprehensively evaluates the conditions inside a vehicle, including human emotional information, to improve the safety of people inside the vehicle. It incorporates an emotion engine using emotion recognition technology and integrates information from various sensors installed inside the vehicle to perform a comprehensive risk assessment.

[0131] Detection of people and environment using devices

[0132] The terminal uses multiple radio transceivers within the vehicle to acquire reflected radio waves and determine whether a person is present inside. In addition, it estimates heart rate and monitors the person's biological state in real time. This data is collected along with temperature data from temperature sensors installed in the vehicle and updated in real time across the entire system.

[0133] Introducing an emotional engine

[0134] The emotion engine analyzes the voices of people inside the vehicle and estimates their emotions through speech recognition technology. This emotion information is transmitted to a server along with the people's biometric and environmental data. The emotional state is used as an important evaluation metric, especially when negative emotions such as stress or discomfort are detected.

[0135] Server-based data analysis and notification

[0136] After receiving various data, the server uses artificial intelligence to comprehensively analyze the conditions inside the vehicle. This analysis takes into account the temperature inside the vehicle, heart rate, and emotional information, and an overall risk level is assessed. If the risk level is determined to be high, the server generates a message notifying the user that immediate action is required.

[0137] Notification and response process

[0138] The generated notification is sent to the user's communication device to inform them that immediate action is required. The notification includes information about the specific danger detected, such as high temperature, abnormal heart rate, or negative emotional state, allowing the user to take appropriate action promptly by reviewing the notification.

[0139] Specific example

[0140] For example, if a toddler is left inside a car, the emotional engine detects the toddler's stress level from their crying, and the temperature sensor records a rapid rise in the car's interior temperature. When the server recognizes danger based on this information, it sends a notification to the user's terminal, prompting immediate action to ensure the toddler's safety. In this way, it is a system that uses a combination of information to ensure a person's safety.

[0141] The following describes the processing flow.

[0142] Step 1:

[0143] The device detects when the vehicle is parked and the engine is turned off, and activates a radio transmitter. It transmits radio waves into the vehicle and receives the reflected data to confirm the presence of a person inside.

[0144] Step 2:

[0145] The device analyzes subtle movements and biometric information of a person from the acquired radio wave reflection data to estimate their heart rate. Additionally, a built-in temperature sensor measures the temperature inside the vehicle and updates this information in real time.

[0146] Step 3:

[0147] The device uses in-car microphones to collect audio and an emotion engine to determine the person's emotional state from the audio. This emotion data is used to detect the person's stress level and negative states.

[0148] Step 4:

[0149] The device combines data on the person's presence, heart rate, temperature, and emotions, and sends it to the server as a data packet.

[0150] Step 5:

[0151] The server analyzes the received data and uses artificial intelligence algorithms to assess the overall risk of the in-vehicle environment. This assessment is based on the risk associated with variations and combinations of each data point.

[0152] Step 6:

[0153] If the server determines there is a risk, it will immediately generate and send a notification to registered contacts containing specific information about the dangers inside the vehicle. The notification will detail any abnormalities in emotional state, temperature, and heart rate.

[0154] Step 7:

[0155] Upon receiving a notification, users are expected to take immediate action to check the safety of the vehicle's interior and take appropriate measures. For example, they may be required to return to the vehicle and check its safety if necessary.

[0156] (Example 2)

[0157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] In recent years, there has been a growing demand for improved safety within vehicles. However, conventional systems struggle to comprehensively evaluate occupants' biometric information and emotional states, making it difficult to provide timely danger alerts. Therefore, there is a need to develop systems that can instantly detect dangers, particularly those arising from temperature changes and emotional state shifts within vehicles, and ensure safety.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0160] In this invention, the server includes means for using multiple radio wave transmitting and receiving devices as a means for detecting people, means for collecting and analyzing voice data to estimate emotional states, and means for comprehensively evaluating the situation inside the vehicle and assessing the degree of danger using a generative AI model. This makes it possible to analyze the biometric information and emotional states of occupants in real time and to promptly notify the user when danger is recognized.

[0161] A "radio wave transmitting and receiving device" is a device that transmits radio waves and detects the surrounding environment through their reflection or reception.

[0162] "Reflection data" is information that captures the characteristics of radio waves when they hit an object and are reflected, and it is data that can be used to estimate position and distance.

[0163] "Biometric information" refers to data that indicates a person's physical condition, such as heart rate and body temperature.

[0164] "Voice analysis means" refers to technologies and devices for processing, identifying, and inferring emotional states from acquired voice data.

[0165] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to analyze data and make judgments about a situation.

[0166] "Means of assessing risk" refer to methods or devices used to evaluate a situation based on collected data and determine the degree of risk.

[0167] A "notification means" is a device or technology used to convey alerts or messages to users or related systems based on the information obtained.

[0168] This invention is a system designed to enhance occupant safety, accurately capturing and comprehensively evaluating human emotional and biological information within a vehicle. The specific technical configuration and functions are described below.

[0169] In this system, the terminal uses multiple radio wave transceivers to acquire reflective data inside the vehicle and estimates the presence of people and heart rate information from it. In addition, the terminal collects audio inside the vehicle via a microphone and uses audio analysis to infer emotional information. This information is transmitted to a server via a communication device.

[0170] Based on the received data, the server runs a generated AI model to comprehensively evaluate the situation inside the vehicle and assess the level of risk. In this process, biometric information such as interior temperature and heart rate, as well as emotional information obtained through voice analysis, are important factors. In particular, when a risk is recognized, the server quickly sends a notification to the user's communication terminal to prompt immediate action.

[0171] As a concrete example, consider a case where a toddler is left inside a car. The device collects the toddler's cries and detects a high level of stress through emotion analysis. In addition, a temperature sensor records the rapid rise in temperature inside the car. The server analyzes this information, and when a high level of danger is recognized, a notification is sent to the user. An example of a prompt used at this time would be an instruction given to the generating AI model such as, "Assess the danger based on the temperature and emotion data inside the car and send a notification."

[0172] This system significantly contributes to crew safety and enables a rapid response, especially in highly urgent situations.

[0173] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0174] Step 1:

[0175] Person detection and biometric information acquisition using a device

[0176] The terminal acquires reflected data from radio wave transceivers installed inside the vehicle. By analyzing this reflected data, the terminal determines whether a person is present inside the vehicle. Furthermore, it uses a biometric monitoring sensor to measure the person's heart rate. The input is reflected radio wave data, and the output is information about the presence of a person and their heart rate.

[0177] Step 2:

[0178] Acquisition of environmental and emotional information

[0179] The device collects audio data using a microphone installed inside the vehicle. This audio data is analyzed by an audio analysis device within the device to estimate emotional information. It also measures the vehicle's interior temperature using a temperature sensor and acquires this information as environmental data. The inputs are audio data and temperature information, and the outputs are emotional information and temperature data.

[0180] Step 3:

[0181] Sending data

[0182] The terminal activates communication means to send acquired biometric, emotional, and environmental information to the server. The data is encrypted using a security protocol and transmitted securely. Various information acquired by the terminal is the input, and data to be transferred to the server is generated as the output.

[0183] Step 4:

[0184] Server-based data analysis and risk assessment.

[0185] The server runs a generative AI model to process the received data. The model integrates biometric, emotional, and environmental information to comprehensively assess the conditions inside the vehicle. The inputs are the received biometric, emotional, and environmental information, and the output is a risk assessment result.

[0186] Step 5:

[0187] Generating a danger notification message

[0188] If the server determines from the analysis results that the risk level is high, it generates a notification message informing the user that action is required. This message includes specific details about the risk. The input is the risk assessment result, and the output is a warning message.

[0189] Step 6:

[0190] Notification and response to users

[0191] The server sends the generated notification to the user's communication terminal. Upon receiving the notification on the terminal, the user promptly checks the situation and takes necessary actions. The input includes the warning message sent to the user, and the output includes the notification displayed on the user's terminal.

[0192] (Application Example 2)

[0193] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0194] To ensure the safety of people inside vehicles, it is necessary to comprehensively evaluate biometric information such as heart rate and emotional state, along with the state of the in-vehicle environment, and to quickly detect and respond to any abnormalities. However, conventional vehicle monitoring systems have focused only on individual elements, resulting in insufficient overall risk assessment. In particular, with the spread of autonomous vehicles, there is a need to automatically monitor passenger safety and take appropriate measures as needed.

[0195] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0196] In this invention, the server includes means for analyzing the biometric data and emotional state of a person inside the vehicle to monitor safety, means for comprehensively analyzing temperature and other in-vehicle environmental information to assess risks, and means for quickly sending notifications to registered destinations when an anomaly is detected. This makes it possible to comprehensively assess the safety of a person, quickly identify hazards inside the vehicle, and take countermeasures.

[0197] "Person detection means" refers to a technical means for identifying the presence of a person inside a vehicle and acquiring their biometric data.

[0198] A "radio wave transceiver" is a device that transmits and receives radio waves inside a vehicle and acquires reflected information.

[0199] "Reflection information" refers to data obtained when radio waves are reflected off objects or people inside a vehicle.

[0200] "Biometric data" refers to information such as a person's heart rate and other health indicators.

[0201] "Communication means" refers to the technical means used to transfer analyzed information.

[0202] An "information processing device" is a device that analyzes the content of received data and comprehensively evaluates the environmental conditions inside a vehicle.

[0203] "Intelligent processing" is a technology that uses artificial intelligence to analyze data and evaluate the conditions inside a vehicle.

[0204] "Risk assessment" is a process of evaluating the safety of a location based on various data from inside the vehicle.

[0205] "Destination" refers to the recipient to whom a notification is sent when a threat is detected.

[0206] This system, designed to ensure the safety of individuals, employs multiple radio transceivers installed inside the vehicle to monitor the interior conditions in real time. Terminals transmit radio waves to acquire reflected information, which is then analyzed to estimate the presence of individuals inside the vehicle and their biometric data. This data is then transmitted to an information processing device (server) via communication means.

[0207] After receiving data, the server analyzes biometric data such as the person's heart rate and emotional state, and uses intelligent processing to evaluate the in-vehicle environment. Specifically, it uses an artificial intelligence framework (e.g., TensorFlow or PyTorch) to analyze the data and assess the risk. If the risk is determined to be high, it quickly sends a notification to the registered connection destination. This notification includes the situation and specific details of the risk that require action.

[0208] For example, if a passenger is feeling uncomfortable or stressed in the vehicle, or if their heart rate suddenly increases, the system will immediately analyze the information and send a notification to the passenger to alert them. It can also take measures to protect the passenger's health, such as recommending that they take an appropriate break.

[0209] Examples of prompts for the generative AI model associated with this system are as follows:

[0210] "I want to design a real-time safety monitoring system for autonomous vehicles. Please propose an approach to build a system that analyzes passenger biometric data and emotional information and sends notifications when an anomaly is detected."

[0211] In this way, the invention effectively monitors safety within the vehicle and enables a rapid response.

[0212] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0213] Step 1:

[0214] The device transmits radio waves using a radio transceiver inside the vehicle and acquires reflected information from people and objects inside the vehicle. The input is a radio signal, and the output is the acquired reflected information. By transmitting and receiving radio waves, data is collected to sense the physical presence inside the vehicle.

[0215] Step 2:

[0216] The system analyzes reflective data acquired by the terminal to estimate the location and biometric data (such as heart rate) of a person inside the vehicle. The input is reflective data, and the output is the estimated person's data. A signal processing algorithm is used to convert the reflective data into person identification and heart rate estimation.

[0217] Step 3:

[0218] The terminal transmits the analysis results to the server via a communication method. The input is data on the estimated person, and the output is the transmission of data to the server. A communication protocol is used to reliably transfer the data.

[0219] Step 4:

[0220] Based on the data received by the server, an artificial intelligence framework is used to analyze the data and evaluate the environmental conditions inside the vehicle. The input consists of biometric and environmental data from inside the vehicle, and the output is the evaluation result of the environmental conditions. An AI model is used to recognize patterns in the data and determine potential hazards.

[0221] Step 5:

[0222] Based on the evaluation results, the server sends a notification to registered connections if the risk level exceeds a certain threshold. The input is the evaluation result, and the output is the notification sent. The warning is sent to the appropriate person using email or a messaging service.

[0223] Step 6:

[0224] The system checks notifications received by the user and takes appropriate action as needed. The input is the notification, and the output is the user's response. Based on the notification, the user takes action to ensure their own safety or the safety of a third party.

[0225] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0226] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0227] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0228] [Second Embodiment]

[0229] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0230] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0231] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0232] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0233] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0234] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0235] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0236] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0237] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0238] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0239] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0240] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0241] This invention is a system aimed at preventing accidents caused by people being left unattended inside vehicles. This system is implemented using a radio wave transceiver and server device installed inside the vehicle, and a user communication terminal equipped with a notification function.

[0242] Person detection using devices

[0243] The terminal uses a radio transceiver installed inside the vehicle to transmit radio waves into the vehicle's interior space and acquires the reflected data. This data forms the basis for determining whether a person is present. By using wireless technology such as Wi-Fi, it is possible to overcome visual limitations and detect people without blind spots.

[0244] Heart rate estimation by the device

[0245] The device further analyzes subtle movements from reflective data to estimate a person's heart rate. This kind of biometric information is particularly important for the early detection of health risks in hot environments.

[0246] Server-based data analysis

[0247] The analyzed data is sent to a server. The server uses artificial intelligence algorithms to analyze the data in real time and assess the risk based on the in-vehicle environment and biometric information. For example, if the in-vehicle temperature exceeds a set safety range or if there is an abnormality in the heart rate, the system immediately recognizes the risk.

[0248] Notification and alert features

[0249] When the server detects a threat, it automatically sends a notification containing specific threat information to registered contacts. The user's communication device receives this notification and issues a visual or audible alert, allowing the user to respond quickly.

[0250] Specific example

[0251] The device automatically begins scanning when the vehicle is parked. For example, if a child is left unattended in the car, the server detects a rise in temperature and a change in heart rate inside the vehicle, and as a result sends an alert to the driver's smartphone. Upon receiving the alert, the user can quickly return to the car and move the child to a safe place.

[0252] In this way, the present invention provides a practical and effective solution for ensuring the safety of people inside a vehicle and preventing unfortunate accidents caused by being left behind.

[0253] The following describes the processing flow.

[0254] Step 1:

[0255] The terminal begins transmitting radio waves when it detects that the vehicle's engine has stopped and the doors have been locked. It receives the reflected radio waves and acquires environmental information data from inside the vehicle.

[0256] Step 2:

[0257] The device analyzes the acquired reflection data to identify the presence of a person. It also estimates the heart rate of a person inside the vehicle by detecting subtle movements. This analyzed data is temporarily stored in internal memory.

[0258] Step 3:

[0259] The device transmits the presence data and heart rate data of the analyzed person to a server via its built-in communication device. Communication takes place periodically, and the data is updated in real time.

[0260] Step 4:

[0261] The server processes the received data in real time and uses artificial intelligence to assess the risks of the in-car environment. Specifically, it determines whether the heart rate is abnormal and whether the in-car temperature has reached a dangerous level.

[0262] Step 5:

[0263] If the server determines that the set danger threshold has been reached, it will generate a notification containing information about the dangers inside the vehicle for the registered contacts.

[0264] Step 6:

[0265] The server sends the generated notification to the user's communication device. This notification contains details of the risk and urges the user to take immediate action.

[0266] Step 7:

[0267] The user receives a notification on their communication device, and an alert is issued. Upon acknowledging the alert, the user immediately returns to their vehicle and begins the safety check procedure.

[0268] (Example 1)

[0269] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0270] Conventional vehicle safety systems have struggled to reliably detect the presence of people and precisely evaluate their biological activity. Furthermore, there is a need to quickly assess risks based on environmental conditions and issue appropriate warnings. Therefore, improvements are needed to ensure the safety of people inside vehicles, particularly to prevent heatstroke and other health risks.

[0271] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0272] In this invention, the server includes means for transmitting radio waves to detect the presence of a person and acquiring reflected data, means for analyzing the acquired reflected data to estimate the presence and biological activity of a person, and means for sending a warning to a registered communication destination when the assessed risk exceeds a certain standard. This enables high-precision safety of a person inside a vehicle and allows for rapid risk assessment and warning issuance.

[0273] "Radio waves that detect the presence of a person" are wireless signals that are transmitted and received to identify a person inside a vehicle.

[0274] "Reflection data" refers to a collection of information acquired after radio waves strike a person or object and are reflected back.

[0275] A "means for estimating biological activity" refers to a system that analyzes reflection data to calculate biological values ​​related to a person, such as heart rate.

[0276] A "central processing unit" is a processing unit that receives, analyzes, and evaluates all data transmitted from a vehicle.

[0277] A "machine learning model" is an algorithm that uses large amounts of data to recognize specific patterns and analyze information.

[0278] "Means of risk assessment" refers to the process of determining whether the conditions inside a vehicle are safe or dangerous by using biometric activity and environmental data.

[0279] "A means of issuing warnings" refers to a function that automatically sends alerts to registered recipients when a risk that threatens a person's safety is detected.

[0280] "Environmental parameters" are data used to monitor physical conditions that may affect safety, such as temperature and humidity inside a vehicle.

[0281] A "communication destination" is a contact that has been pre-configured to receive a warning in the event of a risk.

[0282] This invention is a system for ensuring the safety of people inside vehicles and preventing accidents caused by being left behind. This system is implemented as follows.

[0283] The terminal uses a radio transceiver installed inside the vehicle to transmit radio waves throughout the vehicle interior. These radio waves hit people and objects and are reflected, and the data is acquired by the terminal. This reflected data becomes important basic information for detecting the presence of people. By leveraging wireless technologies such as WiFi, the terminal can eliminate blind spots inside the vehicle and detect people with high precision without relying on vision.

[0284] Furthermore, the terminal analyzes the reflected data due to minute movements and estimates the human heart rate. Biometric information such as the heart rate is particularly important for quickly identifying health risks in a high-temperature environment. Advanced signal processing techniques are used for this estimation.

[0285] The analyzed data is transmitted to the server in real time. The server uses a machine learning model to analyze the received data in detail for the environmental situation and evaluate the risk. For example, when the temperature inside the vehicle exceeds the safe range or the heart rate shows an abnormality, the server has a function to automatically recognize the danger.

[0286] When a danger is detected, the server sends a notification containing the danger information to the registered contact. The user's communication terminal receives this notification and presents an alert visually or audibly. Thereby, the user can respond promptly and ensure the safety of the people left inside the vehicle.

[0287] As a specific example, the terminal has a function to automatically start scanning when the vehicle is parked. For example, when a child is left inside the vehicle, the server detects an increase in the vehicle interior temperature or a variation in the heart rate and sends an alert to the driver's mobile phone. After receiving the alert, the user can rush back to the vehicle and evacuate the child to a safe place.

[0288] An example of an input prompt sentence for the generative AI model is "Based on the data acquired by the sensors installed inside the vehicle, explain the operation of the system to prevent people from being left behind."

[0289] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0290] Step 1:

[0291] The terminal transmits radio waves using a radio transceiver installed inside the vehicle. Input requires setting the radio wave frequency and intensity. These radio waves fill the vehicle and reflect off people and objects. Specifically, the terminal continuously transmits radio waves at regular intervals to constantly obtain the latest reflection data.

[0292] Step 2:

[0293] The device acquires reflected radio wave data. The input is the signal data of the reflected radio waves. This data is analyzed and used as basic information to determine the presence of a person. The device applies a machine learning algorithm to perform the specific operation of detecting the presence or absence of a person based on the reflected data. The output is the result of the determination of whether or not a person is present.

[0294] Step 3:

[0295] The device analyzes minute movements in reflected data to estimate a person's heart rate. The input is reflected data that reflects minute movements. The device performs signal processing and aims to capture periodic changes corresponding to the heart rate. The output is the estimated heart rate value.

[0296] Step 4:

[0297] Biometric information and person presence data analyzed from the terminal are transmitted to the server in real time. The input is the analyzed data from the terminal. The server receives this and immediately begins the next processing.

[0298] Step 5:

[0299] The server uses machine learning models to analyze the received data and assess the risks based on the in-vehicle environmental conditions and biometric information. Inputs include analyzed biometric data and environmental data (e.g., temperature). The server uses multiple algorithms to calculate the risk under specific conditions and determine the level of danger. The output is a result regarding the presence and degree of danger.

[0300] Step 6:

[0301] The server sends an alert to registered contacts if it determines that the risk exceeds a certain threshold. The input is the risk assessment result from step 5. Specifically, the server uses an automated message generation function to generate an appropriate alert message and sends it over the communication network. The output is the alert notification sent to the recipient.

[0302] Step 7:

[0303] The user's communication terminal receives a warning notification from the server and issues an alert. The input is the warning message received from the server. The user can then check the audio or visual alert on their terminal and take the necessary action. The output is the alert received by the user.

[0304] (Application Example 1)

[0305] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0306] Traditional store and facility management methods have made it difficult to detect the risks associated with unattended individuals or sudden illnesses after closing hours or during unattended periods. Leaving this problem unaddressed increases the risk of accidents in unattended spaces, making it difficult to ensure safety. Therefore, there is a need for a system that ensures safety within spaces even during unattended hours and allows for rapid response.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0308] In this invention, the server includes means for acquiring radio wave reflection information in a monitoring space using a radio wave transceiver as a person detection means, means for estimating and analyzing a biological signal from the acquired reflection information, and communication means for evaluating potential risks based on the analyzed information and transmitting it to a notification destination together with warning content. Thereby, the presence of a person and the risk of sudden illness in the monitoring space can be quickly detected during unmanned hours, enabling immediate response.

[0309] The "person detection means" is a mechanism for acquiring radio wave reflection information using a radio wave transceiver to detect the presence of a target in the monitoring space.

[0310] The "radio wave transceiver" is a device that transmits radio waves and receives information obtained by their reflection, and is a technology that can identify a target in space using this information.

[0311] The "reflection information" is data obtained when radio waves hit an object in the monitoring space and are reflected, and serves as a basis for grasping the situation in the space and the presence of a target.

[0312] The "biological signal" is an indicator showing the existence of life, such as a human heart rate and respiration, and is data from which the information can be estimated from minute movements.

[0313] The "information processing device" is a system for receiving acquired data, analyzing its content using artificial intelligence, evaluating the situation of the monitoring space, and determining the presence or absence of potential risks.

[0314] The "potential risk" is a risk such as an accident or sudden illness in the monitoring space, and when this exceeds a certain standard, a state where safety assurance is required.

[0315] "Communication method" refers to a method for sending warnings and details based on analysis results to registered recipients.

[0316] To implement this invention, it is necessary to install a radio wave transceiver that serves as a means for detecting people in the surveillance space. The radio wave transceiver transmits radio waves and monitors the reflected information. This reflected information is important for determining the presence of objects or people in the space, and the obtained information is transmitted to an information processing device.

[0317] The server functions as an information processing unit, analyzing subtle movements from received reflection information to estimate biological signals. It utilizes AI algorithms and analyzes data using software such as TensorFlow and PyTorch. Based on the analysis, it assesses potential risks in the space, and if the level is determined to exceed a certain threshold, it sends detailed information, including a warning, to registered recipients.

[0318] The user's communication device receives warnings sent from the server and provides notifications visually or audibly. This allows the user to immediately recognize the situation and take appropriate action.

[0319] As a concrete example, in an unmanned store, if an employee loses consciousness after closing time, a radio transmitter / receiver will detect their presence, and the server will analyze any abnormalities related to their heart rate. Another example of a prompt message is, "If a person is detected in the store after closing time, how will an alert be quickly sent to the administrator?"

[0320] Through these processes, the invention provides a system that enhances security in surveillance spaces and enables rapid risk response.

[0321] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0322] Step 1:

[0323] The terminal activates a radio transceiver and transmits radio waves within the monitored space. The input is the environment within the space, and the output is the reflected radio wave information. The terminal acquires this reflected information and collects it as data.

[0324] Step 2:

[0325] The server analyzes the reflected information received from the terminal. Here, the input is radio wave reflected information, and the output is the estimated result of biological signals based on subtle movements. The server uses an AI algorithm to estimate the presence of a person and their biological signals (e.g., heart rate) from this data.

[0326] Step 3:

[0327] The server assesses potential risks based on analyzed biosignals and on-site data. Inputs are estimated biosignals and environmental data such as temperature, while output is the risk level. The server performs this assessment against criteria set by an AI model to determine if the potential risk exceeds the threshold.

[0328] Step 4:

[0329] If the server determines, based on its assessment, that the potential risk exceeds the threshold, it will send a warning to the registered recipients. The input is the risk assessment result, and the output is the warning notification. The server generates the message and sends it to the user's communication terminal.

[0330] Step 5:

[0331] The user's communication terminal receives a warning notification from the server and displays an alert to the user visually or audibly. The input is the warning notification data, and the output is the notification to the user. The user reviews this information and takes appropriate action depending on the situation.

[0332] This system enables early detection of risks and the implementation of safety measures even in unmanned monitoring spaces.

[0333] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0334] This invention is a system that comprehensively evaluates the conditions inside a vehicle, including human emotional information, to improve the safety of people inside the vehicle. It incorporates an emotion engine using emotion recognition technology and integrates information from various sensors installed inside the vehicle to perform a comprehensive risk assessment.

[0335] Detection of people and environment using devices

[0336] The terminal uses multiple radio transceivers within the vehicle to acquire reflected radio waves and determine whether a person is present inside. In addition, it estimates heart rate and monitors the person's biological state in real time. This data is collected along with temperature data from temperature sensors installed in the vehicle and updated in real time across the entire system.

[0337] Introducing an emotional engine

[0338] The emotion engine analyzes the voices of people inside the vehicle and estimates their emotions through speech recognition technology. This emotion information is transmitted to a server along with the people's biometric and environmental data. The emotional state is used as an important evaluation metric, especially when negative emotions such as stress or discomfort are detected.

[0339] Server-based data analysis and notification

[0340] After receiving various data, the server uses artificial intelligence to comprehensively analyze the conditions inside the vehicle. This analysis takes into account the temperature inside the vehicle, heart rate, and emotional information, and an overall risk level is assessed. If the risk level is determined to be high, the server generates a message notifying the user that immediate action is required.

[0341] Notification and response process

[0342] The generated notification is sent to the user's communication device to inform them that immediate action is required. The notification includes information about the specific danger detected, such as high temperature, abnormal heart rate, or negative emotional state, allowing the user to take appropriate action promptly by reviewing the notification.

[0343] Specific example

[0344] For example, if a toddler is left inside a car, the emotional engine detects the toddler's stress level from their crying, and the temperature sensor records a rapid rise in the car's interior temperature. When the server recognizes danger based on this information, it sends a notification to the user's terminal, prompting immediate action to ensure the toddler's safety. In this way, it is a system that uses a combination of information to ensure a person's safety.

[0345] The following describes the processing flow.

[0346] Step 1:

[0347] The device detects when the vehicle is parked and the engine is turned off, and activates a radio transmitter. It transmits radio waves into the vehicle and receives the reflected data to confirm the presence of a person inside.

[0348] Step 2:

[0349] The device analyzes subtle movements and biometric information of a person from the acquired radio wave reflection data to estimate their heart rate. Additionally, a built-in temperature sensor measures the temperature inside the vehicle and updates this information in real time.

[0350] Step 3:

[0351] The device uses in-car microphones to collect audio and an emotion engine to determine the person's emotional state from the audio. This emotion data is used to detect the person's stress level and negative states.

[0352] Step 4:

[0353] The device combines data on the person's presence, heart rate, temperature, and emotions, and sends it to the server as a data packet.

[0354] Step 5:

[0355] The server analyzes the received data and uses artificial intelligence algorithms to assess the overall risk of the in-vehicle environment. This assessment is based on the risk associated with variations and combinations of each data point.

[0356] Step 6:

[0357] If the server determines there is a risk, it will immediately generate and send a notification to registered contacts containing specific information about the dangers inside the vehicle. The notification will detail any abnormalities in emotional state, temperature, and heart rate.

[0358] Step 7:

[0359] Upon receiving a notification, users are expected to take immediate action to check the safety of the vehicle's interior and take appropriate measures. For example, they may be required to return to the vehicle and check its safety if necessary.

[0360] (Example 2)

[0361] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0362] In recent years, there has been a growing demand for improved safety within vehicles. However, conventional systems struggle to comprehensively evaluate occupants' biometric information and emotional states, making it difficult to provide timely danger alerts. Therefore, there is a need to develop systems that can instantly detect dangers, particularly those arising from temperature changes and emotional state shifts within vehicles, and ensure safety.

[0363] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0364] In this invention, the server includes means for using multiple radio wave transmitting and receiving devices as a means for detecting people, means for collecting and analyzing voice data to estimate emotional states, and means for comprehensively evaluating the situation inside the vehicle and assessing the degree of danger using a generative AI model. This makes it possible to analyze the biometric information and emotional states of occupants in real time and to promptly notify the user when danger is recognized.

[0365] A "radio wave transmitting and receiving device" is a device that transmits radio waves and detects the surrounding environment through their reflection or reception.

[0366] "Reflection data" is information that captures the characteristics of radio waves when they hit an object and are reflected, and it is data that can be used to estimate position and distance.

[0367] "Biometric information" refers to data that indicates a person's physical condition, such as heart rate and body temperature.

[0368] "Voice analysis means" refers to technologies and devices for processing, identifying, and inferring emotional states from acquired voice data.

[0369] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to analyze data and make judgments about a situation.

[0370] "Means of assessing risk" refer to methods or devices used to evaluate a situation based on collected data and determine the degree of risk.

[0371] A "notification means" is a device or technology used to convey alerts or messages to users or related systems based on the information obtained.

[0372] This invention is a system designed to enhance occupant safety, accurately capturing and comprehensively evaluating human emotional and biological information within a vehicle. The specific technical configuration and functions are described below.

[0373] In this system, the terminal uses multiple radio wave transceivers to acquire reflective data inside the vehicle and estimates the presence of people and heart rate information from it. In addition, the terminal collects audio inside the vehicle via a microphone and uses audio analysis to infer emotional information. This information is transmitted to a server via a communication device.

[0374] Based on the received data, the server runs a generated AI model to comprehensively evaluate the situation inside the vehicle and assess the level of risk. In this process, biometric information such as interior temperature and heart rate, as well as emotional information obtained through voice analysis, are important factors. In particular, when a risk is recognized, the server quickly sends a notification to the user's communication terminal to prompt immediate action.

[0375] As a concrete example, consider a case where a toddler is left inside a car. The device collects the toddler's cries and detects a high level of stress through emotion analysis. In addition, a temperature sensor records the rapid rise in temperature inside the car. The server analyzes this information, and when a high level of danger is recognized, a notification is sent to the user. An example of a prompt used at this time would be an instruction given to the generating AI model such as, "Assess the danger based on the temperature and emotion data inside the car and send a notification."

[0376] This system significantly contributes to crew safety and enables a rapid response, especially in highly urgent situations.

[0377] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0378] Step 1:

[0379] Person detection and biometric information acquisition using a device

[0380] The terminal acquires reflected data from radio wave transceivers installed inside the vehicle. By analyzing this reflected data, the terminal determines whether a person is present inside the vehicle. Furthermore, it uses a biometric monitoring sensor to measure the person's heart rate. The input is reflected radio wave data, and the output is information about the presence of a person and their heart rate.

[0381] Step 2:

[0382] Acquisition of environmental and emotional information

[0383] The device collects audio data using a microphone installed inside the vehicle. This audio data is analyzed by an audio analysis device within the device to estimate emotional information. It also measures the vehicle's interior temperature using a temperature sensor and acquires this information as environmental data. The inputs are audio data and temperature information, and the outputs are emotional information and temperature data.

[0384] Step 3:

[0385] Sending data

[0386] The terminal activates communication means to send acquired biometric, emotional, and environmental information to the server. The data is encrypted using a security protocol and transmitted securely. Various information acquired by the terminal is the input, and data to be transferred to the server is generated as the output.

[0387] Step 4:

[0388] Server-based data analysis and risk assessment.

[0389] The server runs a generative AI model to process the received data. The model integrates biometric, emotional, and environmental information to comprehensively assess the conditions inside the vehicle. The inputs are the received biometric, emotional, and environmental information, and the output is a risk assessment result.

[0390] Step 5:

[0391] Generating a danger notification message

[0392] If the server determines from the analysis results that the risk level is high, it generates a notification message informing the user that action is required. This message includes specific details about the risk. The input is the risk assessment result, and the output is a warning message.

[0393] Step 6:

[0394] Notification and response to users

[0395] The server sends the generated notification to the user's communication terminal. Upon receiving the notification on the terminal, the user promptly checks the situation and takes necessary actions. The input includes the warning message sent to the user, and the output includes the notification displayed on the user's terminal.

[0396] (Application Example 2)

[0397] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0398] To ensure the safety of people inside vehicles, it is necessary to comprehensively evaluate biometric information such as heart rate and emotional state, along with the state of the in-vehicle environment, and to quickly detect and respond to any abnormalities. However, conventional vehicle monitoring systems have focused only on individual elements, resulting in insufficient overall risk assessment. In particular, with the spread of autonomous vehicles, there is a need to automatically monitor passenger safety and take appropriate measures as needed.

[0399] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0400] In this invention, the server includes means for analyzing the biometric data and emotional state of a person inside the vehicle to monitor safety, means for comprehensively analyzing temperature and other in-vehicle environmental information to assess risks, and means for quickly sending notifications to registered destinations when an anomaly is detected. This makes it possible to comprehensively assess the safety of a person, quickly identify hazards inside the vehicle, and take countermeasures.

[0401] "Person detection means" refers to a technical means for identifying the presence of a person inside a vehicle and acquiring their biometric data.

[0402] A "radio wave transceiver" is a device that transmits and receives radio waves inside a vehicle and acquires reflected information.

[0403] "Reflection information" refers to data obtained when radio waves are reflected off objects or people inside a vehicle.

[0404] "Biometric data" refers to information such as a person's heart rate and other health indicators.

[0405] "Communication means" refers to the technical means used to transfer analyzed information.

[0406] An "information processing device" is a device that analyzes the content of received data and comprehensively evaluates the environmental conditions inside a vehicle.

[0407] "Intelligent processing" is a technology that uses artificial intelligence to analyze data and evaluate the conditions inside a vehicle.

[0408] "Risk assessment" is a process of evaluating the safety of a location based on various data from inside the vehicle.

[0409] "Destination" refers to the recipient to whom a notification is sent when a threat is detected.

[0410] This system, designed to ensure the safety of individuals, employs multiple radio transceivers installed inside the vehicle to monitor the interior conditions in real time. Terminals transmit radio waves to acquire reflected information, which is then analyzed to estimate the presence of individuals inside the vehicle and their biometric data. This data is then transmitted to an information processing device (server) via communication means.

[0411] After receiving data, the server analyzes biometric data such as the person's heart rate and emotional state, and uses intelligent processing to evaluate the in-vehicle environment. Specifically, it uses an artificial intelligence framework (e.g., TensorFlow or PyTorch) to analyze the data and assess the risk. If the risk is determined to be high, it quickly sends a notification to the registered connection destination. This notification includes the situation and specific details of the risk that require action.

[0412] For example, if a passenger is feeling uncomfortable or stressed in the vehicle, or if their heart rate suddenly increases, the system will immediately analyze the information and send a notification to the passenger to alert them. It can also take measures to protect the passenger's health, such as recommending that they take an appropriate break.

[0413] Examples of prompts for the generative AI model associated with this system are as follows:

[0414] "I want to design a real-time safety monitoring system for autonomous vehicles. Please propose an approach to build a system that analyzes passenger biometric data and emotional information and sends notifications when an anomaly is detected."

[0415] In this way, the invention effectively monitors safety within the vehicle and enables a rapid response.

[0416] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0417] Step 1:

[0418] The device transmits radio waves using a radio transceiver inside the vehicle and acquires reflected information from people and objects inside the vehicle. The input is a radio signal, and the output is the acquired reflected information. By transmitting and receiving radio waves, data is collected to sense the physical presence inside the vehicle.

[0419] Step 2:

[0420] The system analyzes reflective data acquired by the terminal to estimate the location and biometric data (such as heart rate) of a person inside the vehicle. The input is reflective data, and the output is the estimated person's data. A signal processing algorithm is used to convert the reflective data into person identification and heart rate estimation.

[0421] Step 3:

[0422] The terminal transmits the analysis results to the server via a communication method. The input is data on the estimated person, and the output is the transmission of data to the server. A communication protocol is used to reliably transfer the data.

[0423] Step 4:

[0424] Based on the data received by the server, an artificial intelligence framework is used to analyze the data and evaluate the environmental conditions inside the vehicle. The input consists of biometric and environmental data from inside the vehicle, and the output is the evaluation result of the environmental conditions. An AI model is used to recognize patterns in the data and determine potential hazards.

[0425] Step 5:

[0426] Based on the evaluation results, the server sends a notification to registered connections if the risk level exceeds a certain threshold. The input is the evaluation result, and the output is the notification sent. The warning is sent to the appropriate person using email or a messaging service.

[0427] Step 6:

[0428] The system checks notifications received by the user and takes appropriate action as needed. The input is the notification, and the output is the user's response. Based on the notification, the user takes action to ensure their own safety or the safety of a third party.

[0429] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0430] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0431] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0432] [Third Embodiment]

[0433] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0434] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0435] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0436] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0437] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0438] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0439] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0440] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0441] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0442] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0443] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0444] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0445] This invention is a system aimed at preventing accidents caused by people being left unattended inside vehicles. This system is implemented using a radio wave transceiver and server device installed inside the vehicle, and a user communication terminal equipped with a notification function.

[0446] Person detection using devices

[0447] The terminal uses a radio transceiver installed inside the vehicle to transmit radio waves into the vehicle's interior space and acquires the reflected data. This data forms the basis for determining whether a person is present. By using wireless technology such as Wi-Fi, it is possible to overcome visual limitations and detect people without blind spots.

[0448] Heart rate estimation by the device

[0449] The device further analyzes subtle movements from reflective data to estimate a person's heart rate. This kind of biometric information is particularly important for the early detection of health risks in hot environments.

[0450] Server-based data analysis

[0451] The analyzed data is sent to a server. The server uses artificial intelligence algorithms to analyze the data in real time and assess the risk based on the in-vehicle environment and biometric information. For example, if the in-vehicle temperature exceeds a set safety range or if there is an abnormality in the heart rate, the system immediately recognizes the risk.

[0452] Notification and alert features

[0453] When the server detects a threat, it automatically sends a notification containing specific threat information to registered contacts. The user's communication device receives this notification and issues a visual or audible alert, allowing the user to respond quickly.

[0454] Specific example

[0455] The device automatically begins scanning when the vehicle is parked. For example, if a child is left unattended in the car, the server detects a rise in temperature and a change in heart rate inside the vehicle, and as a result sends an alert to the driver's smartphone. Upon receiving the alert, the user can quickly return to the car and move the child to a safe place.

[0456] In this way, the present invention provides a practical and effective solution for ensuring the safety of people inside a vehicle and preventing unfortunate accidents caused by being left behind.

[0457] The following describes the processing flow.

[0458] Step 1:

[0459] The terminal begins transmitting radio waves when it detects that the vehicle's engine has stopped and the doors have been locked. It receives the reflected radio waves and acquires environmental information data from inside the vehicle.

[0460] Step 2:

[0461] The device analyzes the acquired reflection data to identify the presence of a person. It also estimates the heart rate of a person inside the vehicle by detecting subtle movements. This analyzed data is temporarily stored in internal memory.

[0462] Step 3:

[0463] The device transmits the presence data and heart rate data of the analyzed person to a server via its built-in communication device. Communication takes place periodically, and the data is updated in real time.

[0464] Step 4:

[0465] The server processes the received data in real time and uses artificial intelligence to assess the risks of the in-car environment. Specifically, it determines whether the heart rate is abnormal and whether the in-car temperature has reached a dangerous level.

[0466] Step 5:

[0467] If the server determines that the set danger threshold has been reached, it will generate a notification containing information about the dangers inside the vehicle for the registered contacts.

[0468] Step 6:

[0469] The server sends the generated notification to the user's communication device. This notification contains details of the risk and urges the user to take immediate action.

[0470] Step 7:

[0471] The user receives a notification on their communication device, and an alert is issued. Upon acknowledging the alert, the user immediately returns to their vehicle and begins the safety check procedure.

[0472] (Example 1)

[0473] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0474] Conventional vehicle safety systems have struggled to reliably detect the presence of people and precisely evaluate their biological activity. Furthermore, there is a need to quickly assess risks based on environmental conditions and issue appropriate warnings. Therefore, improvements are needed to ensure the safety of people inside vehicles, particularly to prevent heatstroke and other health risks.

[0475] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0476] In this invention, the server includes means for transmitting radio waves to detect the presence of a person and acquiring reflected data, means for analyzing the acquired reflected data to estimate the presence and biological activity of a person, and means for sending a warning to a registered communication destination when the assessed risk exceeds a certain standard. This enables high-precision safety of a person inside a vehicle and allows for rapid risk assessment and warning issuance.

[0477] "Radio waves that detect the presence of a person" are wireless signals that are transmitted and received to identify a person inside a vehicle.

[0478] "Reflection data" refers to a collection of information acquired after radio waves strike a person or object and are reflected back.

[0479] A "means for estimating biological activity" refers to a system that analyzes reflection data to calculate biological values ​​related to a person, such as heart rate.

[0480] A "central processing unit" is a processing unit that receives, analyzes, and evaluates all data transmitted from a vehicle.

[0481] A "machine learning model" is an algorithm that uses large amounts of data to recognize specific patterns and analyze information.

[0482] "Means of risk assessment" refers to the process of determining whether the conditions inside a vehicle are safe or dangerous by using biometric activity and environmental data.

[0483] "A means of issuing warnings" refers to a function that automatically sends alerts to registered recipients when a risk that threatens a person's safety is detected.

[0484] "Environmental parameters" are data used to monitor physical conditions that may affect safety, such as temperature and humidity inside a vehicle.

[0485] A "communication destination" is a contact that has been pre-configured to receive a warning in the event of a risk.

[0486] This invention is a system for ensuring the safety of people inside vehicles and preventing accidents caused by being left behind. This system is implemented as follows.

[0487] The terminal uses a radio wave transceiver installed inside the vehicle to transmit radio waves throughout the vehicle. These radio waves reflect off people and objects, and the data is acquired by the terminal. This reflected data is important basic information for detecting the presence of people. By utilizing wireless technologies such as Wi-Fi, the terminal can eliminate blind spots inside the vehicle and detect people with high accuracy without relying on vision.

[0488] Furthermore, the device analyzes reflective data from subtle movements to estimate a person's heart rate. Heart rate, a form of biometric information, is particularly important for quickly identifying health risks, especially in high-temperature environments. This estimation utilizes advanced signal processing technology.

[0489] The analyzed data is sent to the server in real time. The server uses machine learning models to analyze the environmental conditions in detail from the received data and assess the risks. For example, the server has the ability to automatically recognize dangers such as when the temperature inside the car exceeds a safe range or when the heart rate shows an abnormality.

[0490] When a danger is detected, the server sends a notification containing the danger information to registered contacts. The user's communication device receives this notification and is presented with an alert visually or audibly. This allows the user to respond quickly and ensure the safety of anyone left inside the vehicle.

[0491] As a concrete example, the device has a function that automatically starts scanning when a vehicle is parked. For instance, if a child is left unattended in the car, the server will detect a rise in the interior temperature or a change in heart rate and send an alert to the driver's mobile phone. After receiving the alert, the user can quickly return to the vehicle and move the child to a safe place.

[0492] An example of an input prompt for a generated AI model is, "Explain how the system works to prevent people from being left behind, based on data acquired from sensors installed inside the vehicle."

[0493] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0494] Step 1:

[0495] The terminal transmits radio waves using a radio transceiver installed inside the vehicle. Input requires setting the radio wave frequency and intensity. These radio waves fill the vehicle and reflect off people and objects. Specifically, the terminal continuously transmits radio waves at regular intervals to constantly obtain the latest reflection data.

[0496] Step 2:

[0497] The device acquires reflected radio wave data. The input is the signal data of the reflected radio waves. This data is analyzed and used as basic information to determine the presence of a person. The device applies a machine learning algorithm to perform the specific operation of detecting the presence or absence of a person based on the reflected data. The output is the result of the determination of whether or not a person is present.

[0498] Step 3:

[0499] The device analyzes minute movements in reflected data to estimate a person's heart rate. The input is reflected data that reflects minute movements. The device performs signal processing and aims to capture periodic changes corresponding to the heart rate. The output is the estimated heart rate value.

[0500] Step 4:

[0501] Biometric information and person presence data analyzed from the terminal are transmitted to the server in real time. The input is the analyzed data from the terminal. The server receives this and immediately begins the next processing.

[0502] Step 5:

[0503] The server uses machine learning models to analyze the received data and assess the risks based on the in-vehicle environmental conditions and biometric information. Inputs include analyzed biometric data and environmental data (e.g., temperature). The server uses multiple algorithms to calculate the risk under specific conditions and determine the level of danger. The output is a result regarding the presence and degree of danger.

[0504] Step 6:

[0505] The server sends an alert to registered contacts if it determines that the risk exceeds a certain threshold. The input is the risk assessment result from step 5. Specifically, the server uses an automated message generation function to generate an appropriate alert message and sends it over the communication network. The output is the alert notification sent to the recipient.

[0506] Step 7:

[0507] The user's communication terminal receives a warning notification from the server and issues an alert. The input is the warning message received from the server. The user can then check the audio or visual alert on their terminal and take the necessary action. The output is the alert received by the user.

[0508] (Application Example 1)

[0509] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0510] Traditional store and facility management methods have made it difficult to detect the risks associated with unattended individuals or sudden illnesses after closing hours or during unattended periods. Leaving this problem unaddressed increases the risk of accidents in unattended spaces, making it difficult to ensure safety. Therefore, there is a need for a system that ensures safety within spaces even during unattended hours and allows for rapid response.

[0511] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0512] In this invention, the server includes means for acquiring radio wave reflection information within a monitored space using a radio wave transceiver as a means for detecting people, means for estimating and analyzing biological signals from the acquired reflection information, and communication means for evaluating potential risks based on the analyzed information and transmitting it to a recipient along with a warning. This enables rapid detection of the presence of people or the risk of sudden illness within a monitored space during unmanned hours, and allows for immediate response.

[0513] A "person detection system" is a mechanism that uses radio wave transceivers to acquire reflected radio wave information and detect the presence of an object within a monitored space.

[0514] A "radio wave transceiver" is a device that transmits radio waves and receives information obtained from their reflection. This technology allows for the identification of objects in space using this information.

[0515] "Reflection information" refers to data obtained when radio waves strike objects within a monitored space and are reflected back. This data forms the basis for understanding the situation and presence of objects within that space.

[0516] "Biosignals" are indicators of the presence of life, such as human heart rate and respiration, and are data from which information can be estimated from subtle movements.

[0517] An "information processing device" is a system that receives acquired data, analyzes its contents using artificial intelligence, evaluates the situation in a monitored space, and determines whether or not there are potential risks.

[0518] "Potential hazards" refer to risks such as accidents or sudden illnesses within the monitored space, and when these exceed a certain threshold, safety measures become necessary.

[0519] "Communication method" refers to a method for sending warnings and details based on analysis results to registered recipients.

[0520] To implement this invention, it is necessary to install a radio wave transceiver that serves as a means for detecting people in the surveillance space. The radio wave transceiver transmits radio waves and monitors the reflected information. This reflected information is important for determining the presence of objects or people in the space, and the obtained information is transmitted to an information processing device.

[0521] The server functions as an information processing unit, analyzing subtle movements from received reflection information to estimate biological signals. It utilizes AI algorithms and analyzes data using software such as TensorFlow and PyTorch. Based on the analysis, it assesses potential risks in the space, and if the level is determined to exceed a certain threshold, it sends detailed information, including a warning, to registered recipients.

[0522] The user's communication device receives warnings sent from the server and provides notifications visually or audibly. This allows the user to immediately recognize the situation and take appropriate action.

[0523] As a concrete example, in an unmanned store, if an employee loses consciousness after closing time, a radio transmitter / receiver will detect their presence, and the server will analyze any abnormalities related to their heart rate. Another example of a prompt message is, "If a person is detected in the store after closing time, how will an alert be quickly sent to the administrator?"

[0524] Through these processes, the invention provides a system that enhances security in surveillance spaces and enables rapid risk response.

[0525] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0526] Step 1:

[0527] The terminal activates a radio transceiver and transmits radio waves within the monitored space. The input is the environment within the space, and the output is the reflected radio wave information. The terminal acquires this reflected information and collects it as data.

[0528] Step 2:

[0529] The server analyzes the reflected information received from the terminal. Here, the input is radio wave reflected information, and the output is the estimated result of biological signals based on subtle movements. The server uses an AI algorithm to estimate the presence of a person and their biological signals (e.g., heart rate) from this data.

[0530] Step 3:

[0531] The server assesses potential risks based on analyzed biosignals and on-site data. Inputs are estimated biosignals and environmental data such as temperature, while output is the risk level. The server performs this assessment against criteria set by an AI model to determine if the potential risk exceeds the threshold.

[0532] Step 4:

[0533] If the server determines, based on its assessment, that the potential risk exceeds the threshold, it will send a warning to the registered recipients. The input is the risk assessment result, and the output is the warning notification. The server generates the message and sends it to the user's communication terminal.

[0534] Step 5:

[0535] The user's communication terminal receives a warning notification from the server and displays an alert to the user visually or audibly. The input is the warning notification data, and the output is the notification to the user. The user reviews this information and takes appropriate action depending on the situation.

[0536] This system enables early detection of risks and the implementation of safety measures even in unmanned monitoring spaces.

[0537] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0538] This invention is a system that comprehensively evaluates the conditions inside a vehicle, including human emotional information, to improve the safety of people inside the vehicle. It incorporates an emotion engine using emotion recognition technology and integrates information from various sensors installed inside the vehicle to perform a comprehensive risk assessment.

[0539] Detection of people and environment using devices

[0540] The terminal uses multiple radio transceivers within the vehicle to acquire reflected radio waves and determine whether a person is present inside. In addition, it estimates heart rate and monitors the person's biological state in real time. This data is collected along with temperature data from temperature sensors installed in the vehicle and updated in real time across the entire system.

[0541] Introducing an emotional engine

[0542] The emotion engine analyzes the voices of people inside the vehicle and estimates their emotions through speech recognition technology. This emotion information is transmitted to a server along with the people's biometric and environmental data. The emotional state is used as an important evaluation metric, especially when negative emotions such as stress or discomfort are detected.

[0543] Server-based data analysis and notification

[0544] After receiving various data, the server uses artificial intelligence to comprehensively analyze the conditions inside the vehicle. This analysis takes into account the temperature inside the vehicle, heart rate, and emotional information, and an overall risk level is assessed. If the risk level is determined to be high, the server generates a message notifying the user that immediate action is required.

[0545] Notification and response process

[0546] The generated notification is sent to the user's communication device to inform them that immediate action is required. The notification includes information about the specific danger detected, such as high temperature, abnormal heart rate, or negative emotional state, allowing the user to take appropriate action promptly by reviewing the notification.

[0547] Specific example

[0548] For example, if a toddler is left inside a car, the emotional engine detects the toddler's stress level from their crying, and the temperature sensor records a rapid rise in the car's interior temperature. When the server recognizes danger based on this information, it sends a notification to the user's terminal, prompting immediate action to ensure the toddler's safety. In this way, it is a system that uses a combination of information to ensure a person's safety.

[0549] The following describes the processing flow.

[0550] Step 1:

[0551] The device detects when the vehicle is parked and the engine is turned off, and activates a radio transmitter. It transmits radio waves into the vehicle and receives the reflected data to confirm the presence of a person inside.

[0552] Step 2:

[0553] The device analyzes subtle movements and biometric information of a person from the acquired radio wave reflection data to estimate their heart rate. Additionally, a built-in temperature sensor measures the temperature inside the vehicle and updates this information in real time.

[0554] Step 3:

[0555] The device uses in-car microphones to collect audio and an emotion engine to determine the person's emotional state from the audio. This emotion data is used to detect the person's stress level and negative states.

[0556] Step 4:

[0557] The device combines data on the person's presence, heart rate, temperature, and emotions, and sends it to the server as a data packet.

[0558] Step 5:

[0559] The server analyzes the received data and uses artificial intelligence algorithms to assess the overall risk of the in-vehicle environment. This assessment is based on the risk associated with variations and combinations of each data point.

[0560] Step 6:

[0561] If the server determines there is a risk, it will immediately generate and send a notification to registered contacts containing specific information about the dangers inside the vehicle. The notification will detail any abnormalities in emotional state, temperature, and heart rate.

[0562] Step 7:

[0563] Upon receiving a notification, users are expected to take immediate action to check the safety of the vehicle's interior and take appropriate measures. For example, they may be required to return to the vehicle and check its safety if necessary.

[0564] (Example 2)

[0565] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0566] In recent years, there has been a growing demand for improved safety within vehicles. However, conventional systems struggle to comprehensively evaluate occupants' biometric information and emotional states, making it difficult to provide timely danger alerts. Therefore, there is a need to develop systems that can instantly detect dangers, particularly those arising from temperature changes and emotional state shifts within vehicles, and ensure safety.

[0567] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0568] In this invention, the server includes means for using multiple radio wave transmitting and receiving devices as a means for detecting people, means for collecting and analyzing voice data to estimate emotional states, and means for comprehensively evaluating the situation inside the vehicle and assessing the degree of danger using a generative AI model. This makes it possible to analyze the biometric information and emotional states of occupants in real time and to promptly notify the user when danger is recognized.

[0569] A "radio wave transmitting and receiving device" is a device that transmits radio waves and detects the surrounding environment through their reflection or reception.

[0570] "Reflection data" is information that captures the characteristics of radio waves when they hit an object and are reflected, and it is data that can be used to estimate position and distance.

[0571] "Biometric information" refers to data that indicates a person's physical condition, such as heart rate and body temperature.

[0572] "Voice analysis means" refers to technologies and devices for processing, identifying, and inferring emotional states from acquired voice data.

[0573] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to analyze data and make judgments about a situation.

[0574] "Means of assessing risk" refer to methods or devices used to evaluate a situation based on collected data and determine the degree of risk.

[0575] A "notification means" is a device or technology used to convey alerts or messages to users or related systems based on the information obtained.

[0576] This invention is a system designed to enhance occupant safety, accurately capturing and comprehensively evaluating human emotional and biological information within a vehicle. The specific technical configuration and functions are described below.

[0577] In this system, the terminal uses multiple radio wave transceivers to acquire reflective data inside the vehicle and estimates the presence of people and heart rate information from it. In addition, the terminal collects audio inside the vehicle via a microphone and uses audio analysis to infer emotional information. This information is transmitted to a server via a communication device.

[0578] Based on the received data, the server runs a generated AI model to comprehensively evaluate the situation inside the vehicle and assess the level of risk. In this process, biometric information such as interior temperature and heart rate, as well as emotional information obtained through voice analysis, are important factors. In particular, when a risk is recognized, the server quickly sends a notification to the user's communication terminal to prompt immediate action.

[0579] As a concrete example, consider a case where a toddler is left inside a car. The device collects the toddler's cries and detects a high level of stress through emotion analysis. In addition, a temperature sensor records the rapid rise in temperature inside the car. The server analyzes this information, and when a high level of danger is recognized, a notification is sent to the user. An example of a prompt used at this time would be an instruction given to the generating AI model such as, "Assess the danger based on the temperature and emotion data inside the car and send a notification."

[0580] This system significantly contributes to crew safety and enables a rapid response, especially in highly urgent situations.

[0581] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0582] Step 1:

[0583] Person detection and biometric information acquisition using a device

[0584] The terminal acquires reflected data from radio wave transceivers installed inside the vehicle. By analyzing this reflected data, the terminal determines whether a person is present inside the vehicle. Furthermore, it uses a biometric monitoring sensor to measure the person's heart rate. The input is reflected radio wave data, and the output is information about the presence of a person and their heart rate.

[0585] Step 2:

[0586] Acquisition of environmental and emotional information

[0587] The device collects audio data using a microphone installed inside the vehicle. This audio data is analyzed by an audio analysis device within the device to estimate emotional information. It also measures the vehicle's interior temperature using a temperature sensor and acquires this information as environmental data. The inputs are audio data and temperature information, and the outputs are emotional information and temperature data.

[0588] Step 3:

[0589] Sending data

[0590] The terminal activates communication means to send acquired biometric, emotional, and environmental information to the server. The data is encrypted using a security protocol and transmitted securely. Various information acquired by the terminal is the input, and data to be transferred to the server is generated as the output.

[0591] Step 4:

[0592] Server-based data analysis and risk assessment.

[0593] The server runs a generative AI model to process the received data. The model integrates biometric, emotional, and environmental information to comprehensively assess the conditions inside the vehicle. The inputs are the received biometric, emotional, and environmental information, and the output is a risk assessment result.

[0594] Step 5:

[0595] Generating a danger notification message

[0596] If the server determines from the analysis results that the risk level is high, it generates a notification message informing the user that action is required. This message includes specific details about the risk. The input is the risk assessment result, and the output is a warning message.

[0597] Step 6:

[0598] Notification and response to users

[0599] The server sends the generated notification to the user's communication terminal. Upon receiving the notification on the terminal, the user promptly checks the situation and takes necessary actions. The input includes the warning message sent to the user, and the output includes the notification displayed on the user's terminal.

[0600] (Application Example 2)

[0601] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0602] To ensure the safety of people inside vehicles, it is necessary to comprehensively evaluate biometric information such as heart rate and emotional state, along with the state of the in-vehicle environment, and to quickly detect and respond to any abnormalities. However, conventional vehicle monitoring systems have focused only on individual elements, resulting in insufficient overall risk assessment. In particular, with the spread of autonomous vehicles, there is a need to automatically monitor passenger safety and take appropriate measures as needed.

[0603] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0604] In this invention, the server includes means for analyzing the biometric data and emotional state of a person inside the vehicle to monitor safety, means for comprehensively analyzing temperature and other in-vehicle environmental information to assess risks, and means for quickly sending notifications to registered destinations when an anomaly is detected. This makes it possible to comprehensively assess the safety of a person, quickly identify hazards inside the vehicle, and take countermeasures.

[0605] "Person detection means" refers to a technical means for identifying the presence of a person inside a vehicle and acquiring their biometric data.

[0606] A "radio wave transceiver" is a device that transmits and receives radio waves inside a vehicle and acquires reflected information.

[0607] "Reflection information" refers to data obtained when radio waves are reflected off objects or people inside a vehicle.

[0608] "Biometric data" refers to information such as a person's heart rate and other health indicators.

[0609] "Communication means" refers to the technical means used to transfer analyzed information.

[0610] An "information processing device" is a device that analyzes the content of received data and comprehensively evaluates the environmental conditions inside a vehicle.

[0611] "Intelligent processing" is a technology that uses artificial intelligence to analyze data and evaluate the conditions inside a vehicle.

[0612] "Risk assessment" is a process of evaluating the safety of a location based on various data from inside the vehicle.

[0613] "Destination" refers to the recipient to whom a notification is sent when a threat is detected.

[0614] This system, designed to ensure the safety of individuals, employs multiple radio transceivers installed inside the vehicle to monitor the interior conditions in real time. Terminals transmit radio waves to acquire reflected information, which is then analyzed to estimate the presence of individuals inside the vehicle and their biometric data. This data is then transmitted to an information processing device (server) via communication means.

[0615] After receiving data, the server analyzes biometric data such as the person's heart rate and emotional state, and uses intelligent processing to evaluate the in-vehicle environment. Specifically, it uses an artificial intelligence framework (e.g., TensorFlow or PyTorch) to analyze the data and assess the risk. If the risk is determined to be high, it quickly sends a notification to the registered connection destination. This notification includes the situation and specific details of the risk that require action.

[0616] For example, if a passenger is feeling uncomfortable or stressed in the vehicle, or if their heart rate suddenly increases, the system will immediately analyze the information and send a notification to the passenger to alert them. It can also take measures to protect the passenger's health, such as recommending that they take an appropriate break.

[0617] Examples of prompts for the generative AI model associated with this system are as follows:

[0618] "I want to design a real-time safety monitoring system for autonomous vehicles. Please propose an approach to build a system that analyzes passenger biometric data and emotional information and sends notifications when an anomaly is detected."

[0619] In this way, the invention effectively monitors safety within the vehicle and enables a rapid response.

[0620] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0621] Step 1:

[0622] The device transmits radio waves using a radio transceiver inside the vehicle and acquires reflected information from people and objects inside the vehicle. The input is a radio signal, and the output is the acquired reflected information. By transmitting and receiving radio waves, data is collected to sense the physical presence inside the vehicle.

[0623] Step 2:

[0624] The system analyzes reflective data acquired by the terminal to estimate the location and biometric data (such as heart rate) of a person inside the vehicle. The input is reflective data, and the output is the estimated person's data. A signal processing algorithm is used to convert the reflective data into person identification and heart rate estimation.

[0625] Step 3:

[0626] The terminal transmits the analysis results to the server via a communication method. The input is data on the estimated person, and the output is the transmission of data to the server. A communication protocol is used to reliably transfer the data.

[0627] Step 4:

[0628] Based on the data received by the server, an artificial intelligence framework is used to analyze the data and evaluate the environmental conditions inside the vehicle. The input consists of biometric and environmental data from inside the vehicle, and the output is the evaluation result of the environmental conditions. An AI model is used to recognize patterns in the data and determine potential hazards.

[0629] Step 5:

[0630] Based on the evaluation results, the server sends a notification to registered connections if the risk level exceeds a certain threshold. The input is the evaluation result, and the output is the notification sent. The warning is sent to the appropriate person using email or a messaging service.

[0631] Step 6:

[0632] The system checks notifications received by the user and takes appropriate action as needed. The input is the notification, and the output is the user's response. Based on the notification, the user takes action to ensure their own safety or the safety of a third party.

[0633] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0634] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0635] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0636] [Fourth Embodiment]

[0637] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0638] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0639] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0640] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0641] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0642] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0643] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0644] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0645] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0646] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0647] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0648] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0649] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0650] This invention is a system aimed at preventing accidents caused by people being left unattended inside vehicles. This system is implemented using a radio wave transceiver and server device installed inside the vehicle, and a user communication terminal equipped with a notification function.

[0651] Person detection using devices

[0652] The terminal uses a radio transceiver installed inside the vehicle to transmit radio waves into the vehicle's interior space and acquires the reflected data. This data forms the basis for determining whether a person is present. By using wireless technology such as Wi-Fi, it is possible to overcome visual limitations and detect people without blind spots.

[0653] Heart rate estimation by the device

[0654] The device further analyzes subtle movements from reflective data to estimate a person's heart rate. This kind of biometric information is particularly important for the early detection of health risks in hot environments.

[0655] Server-based data analysis

[0656] The analyzed data is sent to a server. The server uses artificial intelligence algorithms to analyze the data in real time and assess the risk based on the in-vehicle environment and biometric information. For example, if the in-vehicle temperature exceeds a set safety range or if there is an abnormality in the heart rate, the system immediately recognizes the risk.

[0657] Notification and alert features

[0658] When the server detects a threat, it automatically sends a notification containing specific threat information to registered contacts. The user's communication device receives this notification and issues a visual or audible alert, allowing the user to respond quickly.

[0659] Specific example

[0660] The device automatically begins scanning when the vehicle is parked. For example, if a child is left unattended in the car, the server detects a rise in temperature and a change in heart rate inside the vehicle, and as a result sends an alert to the driver's smartphone. Upon receiving the alert, the user can quickly return to the car and move the child to a safe place.

[0661] In this way, the present invention provides a practical and effective solution for ensuring the safety of people inside a vehicle and preventing unfortunate accidents caused by being left behind.

[0662] The following describes the processing flow.

[0663] Step 1:

[0664] The terminal begins transmitting radio waves when it detects that the vehicle's engine has stopped and the doors have been locked. It receives the reflected radio waves and acquires environmental information data from inside the vehicle.

[0665] Step 2:

[0666] The device analyzes the acquired reflection data to identify the presence of a person. It also estimates the heart rate of a person inside the vehicle by detecting subtle movements. This analyzed data is temporarily stored in internal memory.

[0667] Step 3:

[0668] The device transmits the presence data and heart rate data of the analyzed person to a server via its built-in communication device. Communication takes place periodically, and the data is updated in real time.

[0669] Step 4:

[0670] The server processes the received data in real time and uses artificial intelligence to assess the risks of the in-car environment. Specifically, it determines whether the heart rate is abnormal and whether the in-car temperature has reached a dangerous level.

[0671] Step 5:

[0672] If the server determines that the set danger threshold has been reached, it will generate a notification containing information about the dangers inside the vehicle for the registered contacts.

[0673] Step 6:

[0674] The server sends the generated notification to the user's communication device. This notification contains details of the risk and urges the user to take immediate action.

[0675] Step 7:

[0676] The user receives a notification on their communication device, and an alert is issued. Upon acknowledging the alert, the user immediately returns to their vehicle and begins the safety check procedure.

[0677] (Example 1)

[0678] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0679] Conventional vehicle safety systems have struggled to reliably detect the presence of people and precisely evaluate their biological activity. Furthermore, there is a need to quickly assess risks based on environmental conditions and issue appropriate warnings. Therefore, improvements are needed to ensure the safety of people inside vehicles, particularly to prevent heatstroke and other health risks.

[0680] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0681] In this invention, the server includes means for transmitting radio waves to detect the presence of a person and acquiring reflected data, means for analyzing the acquired reflected data to estimate the presence and biological activity of a person, and means for sending a warning to a registered communication destination when the assessed risk exceeds a certain standard. This enables high-precision safety of a person inside a vehicle and allows for rapid risk assessment and warning issuance.

[0682] "Radio waves that detect the presence of a person" are wireless signals that are transmitted and received to identify a person inside a vehicle.

[0683] "Reflection data" refers to a collection of information acquired after radio waves strike a person or object and are reflected back.

[0684] A "means for estimating biological activity" refers to a system that analyzes reflection data to calculate biological values ​​related to a person, such as heart rate.

[0685] A "central processing unit" is a processing unit that receives, analyzes, and evaluates all data transmitted from a vehicle.

[0686] A "machine learning model" is an algorithm that uses large amounts of data to recognize specific patterns and analyze information.

[0687] "Means of risk assessment" refers to the process of determining whether the conditions inside a vehicle are safe or dangerous by using biometric activity and environmental data.

[0688] "A means of issuing warnings" refers to a function that automatically sends alerts to registered recipients when a risk that threatens a person's safety is detected.

[0689] "Environmental parameters" are data used to monitor physical conditions that may affect safety, such as temperature and humidity inside a vehicle.

[0690] A "communication destination" is a contact that has been pre-configured to receive a warning in the event of a risk.

[0691] This invention is a system for ensuring the safety of people inside vehicles and preventing accidents caused by being left behind. This system is implemented as follows.

[0692] The terminal uses a radio wave transceiver installed inside the vehicle to transmit radio waves throughout the vehicle. These radio waves reflect off people and objects, and the data is acquired by the terminal. This reflected data is important basic information for detecting the presence of people. By utilizing wireless technologies such as Wi-Fi, the terminal can eliminate blind spots inside the vehicle and detect people with high accuracy without relying on vision.

[0693] Furthermore, the device analyzes reflective data from subtle movements to estimate a person's heart rate. Heart rate, a form of biometric information, is particularly important for quickly identifying health risks, especially in high-temperature environments. This estimation utilizes advanced signal processing technology.

[0694] The analyzed data is sent to the server in real time. The server uses machine learning models to analyze the environmental conditions in detail from the received data and assess the risks. For example, the server has the ability to automatically recognize dangers such as when the temperature inside the car exceeds a safe range or when the heart rate shows an abnormality.

[0695] When a danger is detected, the server sends a notification containing the danger information to registered contacts. The user's communication device receives this notification and is presented with an alert visually or audibly. This allows the user to respond quickly and ensure the safety of anyone left inside the vehicle.

[0696] As a concrete example, the device has a function that automatically starts scanning when a vehicle is parked. For instance, if a child is left unattended in the car, the server will detect a rise in the interior temperature or a change in heart rate and send an alert to the driver's mobile phone. After receiving the alert, the user can quickly return to the vehicle and move the child to a safe place.

[0697] An example of an input prompt for a generated AI model is, "Explain how the system works to prevent people from being left behind, based on data acquired from sensors installed inside the vehicle."

[0698] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0699] Step 1:

[0700] The terminal transmits radio waves using a radio transceiver installed inside the vehicle. Input requires setting the radio wave frequency and intensity. These radio waves fill the vehicle and reflect off people and objects. Specifically, the terminal continuously transmits radio waves at regular intervals to constantly obtain the latest reflection data.

[0701] Step 2:

[0702] The device acquires reflected radio wave data. The input is the signal data of the reflected radio waves. This data is analyzed and used as basic information to determine the presence of a person. The device applies a machine learning algorithm to perform the specific operation of detecting the presence or absence of a person based on the reflected data. The output is the result of the determination of whether or not a person is present.

[0703] Step 3:

[0704] The device analyzes minute movements in reflected data to estimate a person's heart rate. The input is reflected data that reflects minute movements. The device performs signal processing and aims to capture periodic changes corresponding to the heart rate. The output is the estimated heart rate value.

[0705] Step 4:

[0706] Biometric information and person presence data analyzed from the terminal are transmitted to the server in real time. The input is the analyzed data from the terminal. The server receives this and immediately begins the next processing.

[0707] Step 5:

[0708] The server uses machine learning models to analyze the received data and assess the risks based on the in-vehicle environmental conditions and biometric information. Inputs include analyzed biometric data and environmental data (e.g., temperature). The server uses multiple algorithms to calculate the risk under specific conditions and determine the level of danger. The output is a result regarding the presence and degree of danger.

[0709] Step 6:

[0710] The server sends an alert to registered contacts if it determines that the risk exceeds a certain threshold. The input is the risk assessment result from step 5. Specifically, the server uses an automated message generation function to generate an appropriate alert message and sends it over the communication network. The output is the alert notification sent to the recipient.

[0711] Step 7:

[0712] The user's communication terminal receives a warning notification from the server and issues an alert. The input is the warning message received from the server. The user can then check the audio or visual alert on their terminal and take the necessary action. The output is the alert received by the user.

[0713] (Application Example 1)

[0714] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0715] Traditional store and facility management methods have made it difficult to detect the risks associated with unattended individuals or sudden illnesses after closing hours or during unattended periods. Leaving this problem unaddressed increases the risk of accidents in unattended spaces, making it difficult to ensure safety. Therefore, there is a need for a system that ensures safety within spaces even during unattended hours and allows for rapid response.

[0716] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0717] In this invention, the server includes means for acquiring radio wave reflection information within a monitored space using a radio wave transceiver as a means for detecting people, means for estimating and analyzing biological signals from the acquired reflection information, and communication means for evaluating potential risks based on the analyzed information and transmitting it to a recipient along with a warning. This enables rapid detection of the presence of people or the risk of sudden illness within a monitored space during unmanned hours, and allows for immediate response.

[0718] A "person detection system" is a mechanism that uses radio wave transceivers to acquire reflected radio wave information and detect the presence of an object within a monitored space.

[0719] A "radio wave transceiver" is a device that transmits radio waves and receives information obtained from their reflection. This technology allows for the identification of objects in space using this information.

[0720] "Reflection information" refers to data obtained when radio waves strike objects within a monitored space and are reflected back. This data forms the basis for understanding the situation and presence of objects within that space.

[0721] "Biosignals" are indicators of the presence of life, such as human heart rate and respiration, and are data from which information can be estimated from subtle movements.

[0722] An "information processing device" is a system that receives acquired data, analyzes its contents using artificial intelligence, evaluates the situation in a monitored space, and determines whether or not there are potential risks.

[0723] "Potential hazards" refer to risks such as accidents or sudden illnesses within the monitored space, and when these exceed a certain threshold, safety measures become necessary.

[0724] "Communication method" refers to a method for sending warnings and details based on analysis results to registered recipients.

[0725] To implement this invention, it is necessary to install a radio wave transceiver that serves as a means for detecting people in the surveillance space. The radio wave transceiver transmits radio waves and monitors the reflected information. This reflected information is important for determining the presence of objects or people in the space, and the obtained information is transmitted to an information processing device.

[0726] The server functions as an information processing unit, analyzing subtle movements from received reflection information to estimate biological signals. It utilizes AI algorithms and analyzes data using software such as TensorFlow and PyTorch. Based on the analysis, it assesses potential risks in the space, and if the level is determined to exceed a certain threshold, it sends detailed information, including a warning, to registered recipients.

[0727] The user's communication device receives warnings sent from the server and provides notifications visually or audibly. This allows the user to immediately recognize the situation and take appropriate action.

[0728] As a concrete example, in an unmanned store, if an employee loses consciousness after closing time, a radio transmitter / receiver will detect their presence, and the server will analyze any abnormalities related to their heart rate. Another example of a prompt message is, "If a person is detected in the store after closing time, how will an alert be quickly sent to the administrator?"

[0729] Through these processes, the invention provides a system that enhances security in surveillance spaces and enables rapid risk response.

[0730] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0731] Step 1:

[0732] The terminal activates a radio transceiver and transmits radio waves within the monitored space. The input is the environment within the space, and the output is the reflected radio wave information. The terminal acquires this reflected information and collects it as data.

[0733] Step 2:

[0734] The server analyzes the reflected information received from the terminal. Here, the input is radio wave reflected information, and the output is the estimated result of biological signals based on subtle movements. The server uses an AI algorithm to estimate the presence of a person and their biological signals (e.g., heart rate) from this data.

[0735] Step 3:

[0736] The server assesses potential risks based on analyzed biosignals and on-site data. Inputs are estimated biosignals and environmental data such as temperature, while output is the risk level. The server performs this assessment against criteria set by an AI model to determine if the potential risk exceeds the threshold.

[0737] Step 4:

[0738] If the server determines, based on its assessment, that the potential risk exceeds the threshold, it will send a warning to the registered recipients. The input is the risk assessment result, and the output is the warning notification. The server generates the message and sends it to the user's communication terminal.

[0739] Step 5:

[0740] The user's communication terminal receives a warning notification from the server and displays an alert to the user visually or audibly. The input is the warning notification data, and the output is the notification to the user. The user reviews this information and takes appropriate action depending on the situation.

[0741] This system enables early detection of risks and the implementation of safety measures even in unmanned monitoring spaces.

[0742] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0743] This invention is a system that comprehensively evaluates the conditions inside a vehicle, including human emotional information, to improve the safety of people inside the vehicle. It incorporates an emotion engine using emotion recognition technology and integrates information from various sensors installed inside the vehicle to perform a comprehensive risk assessment.

[0744] Detection of people and environment using devices

[0745] The terminal uses multiple radio transceivers within the vehicle to acquire reflected radio waves and determine whether a person is present inside. In addition, it estimates heart rate and monitors the person's biological state in real time. This data is collected along with temperature data from temperature sensors installed in the vehicle and updated in real time across the entire system.

[0746] Introducing an emotional engine

[0747] The emotion engine analyzes the voices of people inside the vehicle and estimates their emotions through speech recognition technology. This emotion information is transmitted to a server along with the people's biometric and environmental data. The emotional state is used as an important evaluation metric, especially when negative emotions such as stress or discomfort are detected.

[0748] Server-based data analysis and notification

[0749] After receiving various data, the server uses artificial intelligence to comprehensively analyze the conditions inside the vehicle. This analysis takes into account the temperature inside the vehicle, heart rate, and emotional information, and an overall risk level is assessed. If the risk level is determined to be high, the server generates a message notifying the user that immediate action is required.

[0750] Notification and response process

[0751] The generated notification is sent to the user's communication device to inform them that immediate action is required. The notification includes information about the specific danger detected, such as high temperature, abnormal heart rate, or negative emotional state, allowing the user to take appropriate action promptly by reviewing the notification.

[0752] Specific example

[0753] For example, if a toddler is left inside a car, the emotional engine detects the toddler's stress level from their crying, and the temperature sensor records a rapid rise in the car's interior temperature. When the server recognizes danger based on this information, it sends a notification to the user's terminal, prompting immediate action to ensure the toddler's safety. In this way, it is a system that uses a combination of information to ensure a person's safety.

[0754] The following describes the processing flow.

[0755] Step 1:

[0756] The device detects when the vehicle is parked and the engine is turned off, and activates a radio transmitter. It transmits radio waves into the vehicle and receives the reflected data to confirm the presence of a person inside.

[0757] Step 2:

[0758] The device analyzes subtle movements and biometric information of a person from the acquired radio wave reflection data to estimate their heart rate. Additionally, a built-in temperature sensor measures the temperature inside the vehicle and updates this information in real time.

[0759] Step 3:

[0760] The device uses in-car microphones to collect audio and an emotion engine to determine the person's emotional state from the audio. This emotion data is used to detect the person's stress level and negative states.

[0761] Step 4:

[0762] The device combines data on the person's presence, heart rate, temperature, and emotions, and sends it to the server as a data packet.

[0763] Step 5:

[0764] The server analyzes the received data and uses artificial intelligence algorithms to assess the overall risk of the in-vehicle environment. This assessment is based on the risk associated with variations and combinations of each data point.

[0765] Step 6:

[0766] If the server determines there is a risk, it will immediately generate and send a notification to registered contacts containing specific information about the dangers inside the vehicle. The notification will detail any abnormalities in emotional state, temperature, and heart rate.

[0767] Step 7:

[0768] Upon receiving a notification, users are expected to take immediate action to check the safety of the vehicle's interior and take appropriate measures. For example, they may be required to return to the vehicle and check its safety if necessary.

[0769] (Example 2)

[0770] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0771] In recent years, there has been a growing demand for improved safety within vehicles. However, conventional systems struggle to comprehensively evaluate occupants' biometric information and emotional states, making it difficult to provide timely danger alerts. Therefore, there is a need to develop systems that can instantly detect dangers, particularly those arising from temperature changes and emotional state shifts within vehicles, and ensure safety.

[0772] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0773] In this invention, the server includes means for using multiple radio wave transmitting and receiving devices as a means for detecting people, means for collecting and analyzing voice data to estimate emotional states, and means for comprehensively evaluating the situation inside the vehicle and assessing the degree of danger using a generative AI model. This makes it possible to analyze the biometric information and emotional states of occupants in real time and to promptly notify the user when danger is recognized.

[0774] A "radio wave transmitting and receiving device" is a device that transmits radio waves and detects the surrounding environment through their reflection or reception.

[0775] "Reflection data" is information that captures the characteristics of radio waves when they hit an object and are reflected, and it is data that can be used to estimate position and distance.

[0776] "Biometric information" refers to data that indicates a person's physical condition, such as heart rate and body temperature.

[0777] "Voice analysis means" refers to technologies and devices for processing, identifying, and inferring emotional states from acquired voice data.

[0778] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to analyze data and make judgments about a situation.

[0779] "Means of assessing risk" refer to methods or devices used to evaluate a situation based on collected data and determine the degree of risk.

[0780] A "notification means" is a device or technology used to convey alerts or messages to users or related systems based on the information obtained.

[0781] This invention is a system designed to enhance occupant safety, accurately capturing and comprehensively evaluating human emotional and biological information within a vehicle. The specific technical configuration and functions are described below.

[0782] In this system, the terminal uses multiple radio wave transceivers to acquire reflective data inside the vehicle and estimates the presence of people and heart rate information from it. In addition, the terminal collects audio inside the vehicle via a microphone and uses audio analysis to infer emotional information. This information is transmitted to a server via a communication device.

[0783] Based on the received data, the server runs a generated AI model to comprehensively evaluate the situation inside the vehicle and assess the level of risk. In this process, biometric information such as interior temperature and heart rate, as well as emotional information obtained through voice analysis, are important factors. In particular, when a risk is recognized, the server quickly sends a notification to the user's communication terminal to prompt immediate action.

[0784] As a concrete example, consider a case where a toddler is left inside a car. The device collects the toddler's cries and detects a high level of stress through emotion analysis. In addition, a temperature sensor records the rapid rise in temperature inside the car. The server analyzes this information, and when a high level of danger is recognized, a notification is sent to the user. An example of a prompt used at this time would be an instruction given to the generating AI model such as, "Assess the danger based on the temperature and emotion data inside the car and send a notification."

[0785] This system significantly contributes to crew safety and enables a rapid response, especially in highly urgent situations.

[0786] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0787] Step 1:

[0788] Person detection and biometric information acquisition using a device

[0789] The terminal acquires reflected data from radio wave transceivers installed inside the vehicle. By analyzing this reflected data, the terminal determines whether a person is present inside the vehicle. Furthermore, it uses a biometric monitoring sensor to measure the person's heart rate. The input is reflected radio wave data, and the output is information about the presence of a person and their heart rate.

[0790] Step 2:

[0791] Acquisition of environmental and emotional information

[0792] The device collects audio data using a microphone installed inside the vehicle. This audio data is analyzed by an audio analysis device within the device to estimate emotional information. It also measures the vehicle's interior temperature using a temperature sensor and acquires this information as environmental data. The inputs are audio data and temperature information, and the outputs are emotional information and temperature data.

[0793] Step 3:

[0794] Sending data

[0795] The terminal activates communication means to send acquired biometric, emotional, and environmental information to the server. The data is encrypted using a security protocol and transmitted securely. Various information acquired by the terminal is the input, and data to be transferred to the server is generated as the output.

[0796] Step 4:

[0797] Server-based data analysis and risk assessment.

[0798] The server runs a generative AI model to process the received data. The model integrates biometric, emotional, and environmental information to comprehensively assess the conditions inside the vehicle. The inputs are the received biometric, emotional, and environmental information, and the output is a risk assessment result.

[0799] Step 5:

[0800] Generating a danger notification message

[0801] If the server determines from the analysis results that the risk level is high, it generates a notification message informing the user that action is required. This message includes specific details about the risk. The input is the risk assessment result, and the output is a warning message.

[0802] Step 6:

[0803] Notification and response to users

[0804] The server sends the generated notification to the user's communication terminal. Upon receiving the notification on the terminal, the user promptly checks the situation and takes necessary actions. The input includes the warning message sent to the user, and the output includes the notification displayed on the user's terminal.

[0805] (Application Example 2)

[0806] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0807] To ensure the safety of people inside vehicles, it is necessary to comprehensively evaluate biometric information such as heart rate and emotional state, along with the state of the in-vehicle environment, and to quickly detect and respond to any abnormalities. However, conventional vehicle monitoring systems have focused only on individual elements, resulting in insufficient overall risk assessment. In particular, with the spread of autonomous vehicles, there is a need to automatically monitor passenger safety and take appropriate measures as needed.

[0808] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0809] In this invention, the server includes means for analyzing the biometric data and emotional state of a person inside the vehicle to monitor safety, means for comprehensively analyzing temperature and other in-vehicle environmental information to assess risks, and means for quickly sending notifications to registered destinations when an anomaly is detected. This makes it possible to comprehensively assess the safety of a person, quickly identify hazards inside the vehicle, and take countermeasures.

[0810] "Person detection means" refers to a technical means for identifying the presence of a person inside a vehicle and acquiring their biometric data.

[0811] A "radio wave transceiver" is a device that transmits and receives radio waves inside a vehicle and acquires reflected information.

[0812] "Reflection information" refers to data obtained when radio waves are reflected off objects or people inside a vehicle.

[0813] "Biometric data" refers to information such as a person's heart rate and other health indicators.

[0814] "Communication means" refers to the technical means used to transfer analyzed information.

[0815] An "information processing device" is a device that analyzes the content of received data and comprehensively evaluates the environmental conditions inside a vehicle.

[0816] "Intelligent processing" is a technology that uses artificial intelligence to analyze data and evaluate the conditions inside a vehicle.

[0817] "Risk assessment" is a process of evaluating the safety of a location based on various data from inside the vehicle.

[0818] "Destination" refers to the recipient to whom a notification is sent when a threat is detected.

[0819] This system, designed to ensure the safety of individuals, employs multiple radio transceivers installed inside the vehicle to monitor the interior conditions in real time. Terminals transmit radio waves to acquire reflected information, which is then analyzed to estimate the presence of individuals inside the vehicle and their biometric data. This data is then transmitted to an information processing device (server) via communication means.

[0820] After receiving data, the server analyzes biometric data such as the person's heart rate and emotional state, and uses intelligent processing to evaluate the in-vehicle environment. Specifically, it uses an artificial intelligence framework (e.g., TensorFlow or PyTorch) to analyze the data and assess the risk. If the risk is determined to be high, it quickly sends a notification to the registered connection destination. This notification includes the situation and specific details of the risk that require action.

[0821] For example, if a passenger is feeling uncomfortable or stressed in the vehicle, or if their heart rate suddenly increases, the system will immediately analyze the information and send a notification to the passenger to alert them. It can also take measures to protect the passenger's health, such as recommending that they take an appropriate break.

[0822] Examples of prompts for the generative AI model associated with this system are as follows:

[0823] "I want to design a real-time safety monitoring system for autonomous vehicles. Please propose an approach to build a system that analyzes passenger biometric data and emotional information and sends notifications when an anomaly is detected."

[0824] In this way, the invention effectively monitors safety within the vehicle and enables a rapid response.

[0825] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0826] Step 1:

[0827] The device transmits radio waves using a radio transceiver inside the vehicle and acquires reflected information from people and objects inside the vehicle. The input is a radio signal, and the output is the acquired reflected information. By transmitting and receiving radio waves, data is collected to sense the physical presence inside the vehicle.

[0828] Step 2:

[0829] The system analyzes reflective data acquired by the terminal to estimate the location and biometric data (such as heart rate) of a person inside the vehicle. The input is reflective data, and the output is the estimated person's data. A signal processing algorithm is used to convert the reflective data into person identification and heart rate estimation.

[0830] Step 3:

[0831] The terminal transmits the analysis results to the server via a communication method. The input is data on the estimated person, and the output is the transmission of data to the server. A communication protocol is used to reliably transfer the data.

[0832] Step 4:

[0833] Based on the data received by the server, an artificial intelligence framework is used to analyze the data and evaluate the environmental conditions inside the vehicle. The input consists of biometric and environmental data from inside the vehicle, and the output is the evaluation result of the environmental conditions. An AI model is used to recognize patterns in the data and determine potential hazards.

[0834] Step 5:

[0835] Based on the evaluation results, the server sends a notification to registered connections if the risk level exceeds a certain threshold. The input is the evaluation result, and the output is the notification sent. The warning is sent to the appropriate person using email or a messaging service.

[0836] Step 6:

[0837] The system checks notifications received by the user and takes appropriate action as needed. The input is the notification, and the output is the user's response. Based on the notification, the user takes action to ensure their own safety or the safety of a third party.

[0838] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0839] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0840] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0841] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0842] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0843] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0844] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0845] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0846] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0847] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0848] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0849] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0850] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0851] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0852] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0853] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0854] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0855] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0856] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0857] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0858] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0859] The following is further disclosed regarding the embodiments described above.

[0860] (Claim 1)

[0861] The vehicle is equipped with multiple radio wave transceivers as a means of detecting people, and a means of acquiring radio wave reflection data,

[0862] A means for analyzing acquired reflection data to estimate the presence of a person inside the vehicle and their heart rate,

[0863] A communication means for transmitting the analyzed data to a server device,

[0864] Based on the received data, a means of analyzing the environmental conditions inside the vehicle using artificial intelligence and assessing the risks,

[0865] A means of sending a notification to registered contacts when the level of risk is determined to be above a certain level,

[0866] A system that includes this.

[0867] (Claim 2)

[0868] The system according to claim 1, further comprising means for measuring the temperature inside a vehicle and transmitting the temperature data to a server device.

[0869] (Claim 3)

[0870] The system according to claim 1, further comprising means for sending a notification to a contact person, including the details of the detected risk.

[0871] "Example 1"

[0872] (Claim 1)

[0873] A means of transmitting radio waves to detect the presence of a person and acquiring reflected data,

[0874] A means for analyzing acquired reflection data to estimate the presence and biological activity of a person,

[0875] A communication means for transmitting the analyzed biological activity data to a central processing unit,

[0876] A means of analyzing environmental conditions and assessing risks using machine learning models based on received bioactivity data,

[0877] A means of issuing a warning to registered communication recipients when the assessed risk exceeds a certain threshold,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] The system according to claim 1, further comprising means for monitoring environmental parameters and transmitting monitoring data to a central processing unit.

[0881] (Claim 3)

[0882] The system according to claim 1, further comprising means for transmitting to the recipient details of the assessed risk in the warning content to be transmitted.

[0883] "Application Example 1"

[0884] (Claim 1)

[0885] The system includes multiple radio wave transceivers as a means of detecting people, and a means of acquiring radio wave reflection information.

[0886] A means for analyzing acquired reflection information to estimate the presence of an object within the surveillance space and its biological signals,

[0887] A communication means for transmitting the analyzed data to an information processing device that receives it,

[0888] Based on the received data, a means of analyzing the situation in the monitored space using artificial intelligence and evaluating potential risks,

[0889] A means of sending a warning to a registered recipient when the potential risk exceeds a set threshold,

[0890] A system that includes this.

[0891] (Claim 2)

[0892] The system according to claim 1, further comprising means for measuring the physical conditions of a monitoring space and transmitting the data to an information processing device.

[0893] (Claim 3)

[0894] The system according to claim 1, further comprising means for sending a notification to a recipient, including details of the detected potential hazard in the warning content.

[0895] "Example 2 of combining an emotion engine"

[0896] (Claim 1)

[0897] The vehicle is equipped with multiple radio wave transmitting and receiving devices as a means of detecting people, and means of acquiring reflected data obtained from them,

[0898] A means for analyzing acquired reflection data to estimate whether a person is present inside the vehicle and to determine their heart rate information,

[0899] A voice analysis means that collects voice data, analyzes it, and estimates emotional state,

[0900] A communication means for sending the analyzed data to a server that receives it,

[0901] Based on received biometric data, emotional information, and environmental data, a generative AI model is used to comprehensively evaluate the situation inside the vehicle and assess the level of risk.

[0902] A means of sending a warning to registered communication devices when the level of danger exceeds a certain level,

[0903] A system that includes this.

[0904] (Claim 2)

[0905] The system according to claim 1, further comprising means for sensing the temperature inside a vehicle and transmitting the data to a server.

[0906] (Claim 3)

[0907] The system according to claim 1, further comprising means for transmitting to a communication device the notification content, including details of the detected danger.

[0908] "Application example 2 when combining with an emotional engine"

[0909] (Claim 1)

[0910] The vehicle is equipped with multiple radio wave transceivers as a means of detecting people, and a means of acquiring radio wave reflection information,

[0911] A means for analyzing acquired reflective information to identify a person inside the vehicle and estimate their biometric data,

[0912] A communication means for transferring the analyzed information to an information processing device that receives it,

[0913] A means of analyzing the environmental conditions inside the vehicle using intelligent processing based on the received information and evaluating the risks,

[0914] A means of forwarding a notification to the registered connection destination when the risk level is determined to be above a certain level,

[0915] A means of monitoring the safety of passengers in an automated vehicle by analyzing biometric data and emotional states,

[0916] A system that includes this.

[0917] (Claim 2)

[0918] The system according to claim 1, further comprising means for measuring the temperature inside a vehicle and transferring the temperature information to an information processing device.

[0919] (Claim 3)

[0920] The system according to claim 1, further comprising means for forwarding the notification content, including the details of the detected danger, to the destination. [Explanation of Symbols]

[0921] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. The system includes multiple radio wave transceivers as a means of detecting people, and a means of acquiring radio wave reflection information. A means for analyzing acquired reflection information to estimate the presence of an object within the surveillance space and its biological signals, A communication means for transmitting the analyzed data to an information processing device that receives it, Based on the received data, a means of analyzing the situation in the monitored space using artificial intelligence and evaluating potential risks, A means of sending a warning to a registered recipient when the potential risk exceeds a set threshold, A system that includes this.

2. The system according to claim 1, further comprising means for measuring the physical conditions of a monitoring space and transmitting the data to an information processing device.

3. The system according to claim 1, further comprising means for transmitting to a recipient a warning that includes details of the detected potential hazard.

Images