system

Abstract

We provide the system. [Solution] A means for acquiring the user's biometric information using a biometric information detection device, A means for analyzing the aforementioned biological information and learning normal behavioral patterns, Means for detecting abnormalities by comparing them with the aforementioned normal behavioral patterns, A means for generating a notification and sending it to an external device when an anomaly is detected, A system that includes this.

Description

Technical Field

[0001] The technology of this disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, effectively ensuring the safety of children and the elderly is a major issue. In conventional technologies, monitoring only location information has been the main focus, which is insufficient for immediately detecting dangerous situations. In particular, in urgent situations such as bullying, abduction, and accidents, a quick response is required, but there is a lack of systems that can handle this. Against this background, there is a need for a technology that can quickly detect abnormal or critical situations and notify relevant parties.

Means for Solving the Problems

[0005] This invention provides a system that acquires a user's biometric information in real time using a wearable biometric information detection device. By analyzing the acquired biometric information and learning normal behavioral patterns, the system can immediately detect abnormalities when they occur. When an abnormality is detected, it generates a notification and sends it to an external device, enabling a rapid response. Furthermore, this system improves the accuracy of abnormality detection by performing analysis based on multifaceted information such as heart rate, blood oxygen saturation, and acceleration data. This allows for a higher level of user safety.

[0006] A "biometric information detection device" is a device used to acquire biometric information such as the user's heart rate, blood oxygen saturation, and body movements.

[0007] "Biometric information" refers to data obtained directly from the human body, such as heart rate, blood oxygen saturation, and impact data based on acceleration.

[0008] "Normal behavioral patterns" refer to the user's usual physiological and behavioral states based on biometric information, and are a set of baseline values ​​that change over time.

[0009] "Detecting anomalies" means determining whether the acquired biometric information deviates from pre-learned normal behavioral patterns.

[0010] "Generating a notification and sending it to an external device" means creating information indicating the detected abnormal situation and transmitting it to a designated receiving device via the network. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3]This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention provides a wearable device for acquiring biometric information and a system for managing and analyzing that data. The system aims to ensure the safety of children and the elderly by immediately detecting abnormal behavior and notifying relevant parties. To implement this invention, the following configuration and processing are performed.

[0033] First, the device uses a wearable device to continuously collect the user's biometric information, such as heart rate, blood oxygen saturation, and impact data from an accelerometer. This data is transmitted to a server in real time via a secure communication protocol. The server receives the transmitted biometric information and stores it in a database.

[0034] Next, the server uses machine learning based on the collected biometric information to learn normal behavioral patterns. During this learning process, a model is created that reflects general physiological data and the characteristics of individual users, enabling real-time anomaly detection.

[0035] When an anomaly is detected, the server automatically generates a notification. This notification includes the type of anomaly and specific numerical changes, and is sent to a designated external device, such as a parent's device. The parent, as the user, can immediately check the situation and take appropriate action upon receiving this notification. The notification includes the time and location of the anomaly, as well as detailed biometric data, which can be used to assess the situation.

[0036] For example, if a child's heart rate suddenly increases during school recess, the device detects this change and sends it to the server. The server then uses this information to determine that the child's behavior deviates from normal patterns and immediately notifies the parents. This allows parents to quickly contact the school to check on their child's safety or request assistance.

[0037] The above is one embodiment of the present invention, and this system can effectively improve the safety of children and the elderly.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The device acquires the user's heart rate, blood oxygen saturation, and impact data from an accelerometer via a wearable device at regular sampling intervals. The acquired data is stored in a temporary buffer.

[0041] Step 2:

[0042] The device encrypts the data and sends it to the server using a secure communication protocol. This transmission occurs in real time for each data collection cycle.

[0043] Step 3:

[0044] The server receives biometric information transmitted from the terminal and stores it in a database. The stored data is used for subsequent analysis and model updates.

[0045] Step 4:

[0046] The server analyzes the received biometric data and uses machine learning algorithms to learn normal behavioral patterns. This model is updated in real time and serves as a criterion for monitoring anomalies.

[0047] Step 5:

[0048] The server compares newly received biometric data in real time with a trained model and detects anomalies that exceed a threshold. If an anomaly is detected, a process is performed to identify the type of anomaly.

[0049] Step 6:

[0050] The server generates a notification based on the identified anomaly. The notification includes information such as the nature of the anomaly, the time it occurred, and the location. This information is essential for prompting appropriate action.

[0051] Step 7:

[0052] The server sends the generated notification to a designated external device (e.g., a parent's device). The communication is designed to occur instantly.

[0053] Step 8:

[0054] Parents, as users, can review received notifications and understand their child's current situation. Depending on the circumstances, they can contact the school or other relevant organizations to take prompt action.

[0055] This series of processes allows the system to detect anomalies in real time and enable a rapid response.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] In modern society, the safety of children and the elderly is a critical issue. In particular, there is a growing need for systems that can quickly detect abnormalities in their health status and behavior and respond appropriately. However, existing technologies struggle to efficiently detect abnormalities from biometric data and promptly notify relevant parties. Solving this problem is essential.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for receiving data using a secure communication protocol, means for learning normal behavioral patterns from the received biometric information using a machine learning algorithm, and means for generating and transmitting notifications to external devices when an anomaly is detected. This makes it possible to monitor the biometric information of children and the elderly in real time and to quickly notify relevant parties in the event of an anomaly.

[0061] A "biometric information detection device" is a device that acquires information about the user's body in real time, specifically by collecting data such as heart rate, blood oxygen saturation, and acceleration using sensors.

[0062] A "communication protocol" is a set of rules and procedures for securely sending and receiving data between a terminal and a server, guaranteeing information encryption and reliable transmission.

[0063] The "central device" is a computer system that collects and analyzes received biological information, and has the function of storing data and detecting anomalies.

[0064] A "machine learning algorithm" is a mathematical method used to learn from past data, recognize patterns, and analyze and predict future data. In this invention, it is used for the analysis of biological information.

[0065] An "external device" is a device used to receive notifications when an abnormality is detected, and typically refers to a mobile device or computer owned by the user's parent or administrator.

[0066] A "notification" is a warning message generated when an anomaly is detected, and it includes the type of anomaly, detailed numerical information, and location information.

[0067] This invention is a system that collects users' biometric information in real time and provides rapid notification in the event of an abnormality. Its aim is to improve the safety of children and the elderly.

[0068] The device continuously acquires the user's heart rate, blood oxygen saturation, and impact data based on acceleration using a biometric detection device. This device includes a heart rate sensor, oxygen saturation monitor, and accelerometer, enabling accurate data collection. This data is transmitted from the device to the server using Bluetooth or Wi-Fi via a secure communication protocol.

[0069] After receiving the transmitted biometric information, the server stores it in a database and analyzes the data using machine learning frameworks such as TENSORFLOW® and PyTorch. This allows the server to learn each user's normal behavior patterns and detect anomalies in real time. When an anomaly is detected, the server immediately generates a notification about the anomaly and sends it to a designated external device, such as a parent's mobile device.

[0070] Parents, as users, can receive notifications from their devices, allowing them to immediately check the situation and take necessary actions if any abnormalities occur with their children or elderly relatives. These notifications include the type of abnormality, the time of occurrence, and specific numerical changes and location information, supporting quick and effective decision-making.

[0071] For example, if a child's heart rate suddenly increases during lunchtime, the device detects this change and sends it to a server. The server identifies the anomaly based on this data and sends a notification to the parent's external device. Upon receiving this notification, the parent can immediately contact the school to check on their child's situation. In this way, the system enables rapid safety checks.

[0072] An example of a prompt might be, "Please provide specific notification content to be sent to parents if their child's heart rate suddenly increased at school yesterday." This would allow the AI ​​model to generate an appropriate and specific notification message.

[0073] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0074] Step 1:

[0075] The device acquires input from a biometric information detection device, including the user's heart rate, blood oxygen saturation, and impact data. This data is measured in real time by various built-in sensors. After preprocessing, such as noise reduction and data standardization, the acquired data is temporarily stored in the device's memory.

[0076] Step 2:

[0077] The device transmits pre-processed biometric information to the server using a secure communication protocol. Specifically, the data is encrypted and sent to the server via Bluetooth or Wi-Fi. This ensures data security and privacy.

[0078] Step 3:

[0079] The server receives biometric data transmitted from the terminal and stores it in a database. The database also stores each user's past biometric information, which can be used to track user behavior patterns. Each data item is stored with a timestamp, making it accessible for later processing.

[0080] Step 4:

[0081] The server learns normal behavioral patterns using machine learning algorithms while referencing biometric information stored in a database. At this stage, the model is trained using historical data, and analysis is performed to detect anomalies based on new and historical data. Specifically, when an abnormal fluctuation occurs, a threshold is set to detect that change as a deviation from the pattern.

[0082] Step 5:

[0083] The server compares newly received biometric information with a learned model and detects anomalies in real time. This detection occurs when a set threshold is exceeded, and if an anomaly is confirmed, an alert flag is set.

[0084] Step 6:

[0085] If an anomaly is detected, the server generates a detailed notification. This notification includes the type of anomaly, the time of occurrence, and the identification of the affected user. Because this notification is generated in real time, a rapid response is possible.

[0086] Step 7:

[0087] The server sends the generated notification to a designated external device, such as a parent's device. The notification is sent via SMS, email, or a dedicated app.

[0088] Step 8:

[0089] Parents, as users, receive notifications on their devices and review their contents. Based on the notifications, parents check their child's safety and take appropriate action, such as contacting schools or medical institutions, as needed. At this stage, the notification content needs to be clear in order to easily understand its impact.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] There is a need for technology that can quickly and accurately monitor the health status of users in various environments, precisely detect abnormalities, appropriately notify relevant parties, and visually understand the situation. Such technology is especially necessary to ensure the safety of children and the elderly in situations where parents cannot directly supervise them.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes means for acquiring user health data using a biometric information detection device, means for analyzing the health data and learning normal behavioral patterns, and means for detecting abnormalities by comparing them with the normal behavioral patterns. This makes it possible to monitor the user's health status in real time in various situations, reliably detect abnormalities, and provide visual notification.

[0095] A "biometric information detection device" is a device used to acquire a user's health data and primarily functions as a wearable device.

[0096] "Health data" refers to biological data, including the number of periodic physical activity levels, blood oxygen levels, and movement-based stimulus data.

[0097] "Normal behavioral patterns" refer to the standard or average patterns of biometric information that a user exhibits when healthy.

[0098] "External devices" refer to devices used to receive notifications based on health data, such as communication terminals.

[0099] A "visual display device" is a device that visually notifies the user of information when an abnormality is detected, and includes eyewear-type displays, among others.

[0100] An "abnormality" refers to a condition that deviates from the normal pattern of health data and is an event that requires immediate attention.

[0101] A "smart device" is an electronic device with advanced processing capabilities and communication functions, and includes personal digital assistants (PADs).

[0102] To realize this invention, it is necessary to acquire and analyze user health data using a series of devices and systems.

[0103] The server uses a wearable device as a biometric information detection device. This device has the function of continuously acquiring the user's cyclical activity rate (heart rate), blood oxygen concentration, and movement-based stimulation data. This health data is transmitted to the server in real time and stored in a database for analysis.

[0104] The server uses a generative AI model based on collected health data to learn normal behavioral patterns. This model reflects both general health data and the individual user's characteristics, giving it the ability to instantly detect deviations from normal behavioral patterns. If an anomaly is detected, the server generates a notification for the smart device and also provides a visual warning to the user via a visual display device.

[0105] For example, if a user's heart rate suddenly increases while exercising outdoors, the wearable device will detect this anomaly. The server will analyze this and determine it to be an anomaly, and display a message on the smart device saying, "A sudden increase in heart rate has been detected. Please rest." A visual display will show this message in the user's field of vision, prompting immediate action.

[0106] An example of a prompt message might be: "Please provide specific interface design proposals for how this health data analysis system should quickly and effectively notify the user when it detects an anomaly."

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] The device uses a wearable biometric device to acquire the user's health data. This data includes heart rate, blood oxygen saturation, and acceleration. The collected health data is transmitted to the server in real time.

[0110] Step 2:

[0111] The server receives health data acquired from the terminal and stores it in a database. This stored data serves as foundational information for later analysis. Health data is the input, and the output remains in the form of data stored in the database.

[0112] Step 3:

[0113] The server analyzes stored health data using an AI model to learn normal behavioral patterns. During this process, it builds a model that reflects the characteristics of each user, preparing it to detect anomalies. Past health data is used as input to the model, and a model for determining normal ranges is generated as output.

[0114] Step 4:

[0115] The server uses a model built through analysis to compare real-time transmitted health data with normal patterns. This comparison detects anomalies. Current health data is used as input, and the presence or absence of an anomaly is determined as output.

[0116] Step 5:

[0117] When an anomaly is detected, the server initiates a process to generate a notification for the smart device. Specifically, it prepares information such as "A sudden increase in heart rate has been detected" as the notification content. The input is the anomaly information, and the output is the notification message.

[0118] Step 6:

[0119] The generated notification is visually displayed on the user's visual display device. This visualization allows the user to immediately recognize anomalies and take corrective action. The input is the content of the notification, and the output is the information displayed in the user's field of vision.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] This invention is a system for ensuring user safety using biometric information and emotional data derived therefrom. This system incorporates an emotion engine that monitors the user's biometric information in real time, detects abnormalities, and recognizes the user's emotional state. This emotion engine uses the obtained biometric data to evaluate the user's emotional state and provides information to address stress levels and abnormal situations.

[0122] First, the device acquires heart rate, blood oxygen saturation, and impact data from an accelerometer via a wearable device. Next, when the device transmits this data to the server, it encrypts it and protects the data using a secure communication protocol.

[0123] The server stores and analyzes the received data. Here, the emotion engine estimates the user's emotional state based on biometric data. For example, it calculates the stress level by considering heart rate and the rate of change in blood oxygen saturation. The emotion engine also works in conjunction with an anomaly detection model, and is designed to detect abnormalities with higher accuracy when the emotional state deviates from the normal state.

[0124] If an anomaly is detected, the server automatically generates a notification, including information about the user's emotional state, and sends it to the parent's device. Through this notification, the user can immediately verify the user's safety and take necessary actions quickly.

[0125] As a concrete example, suppose a child experiences a sudden increase in heart rate under some kind of pressure, and their stress level is also assessed as high. In this case, the server detects the stress-related anomaly through its emotion engine and notifies the parent with a detailed report, including the emotional state. This notification allows the parent to infer the nature and cause of the stress and obtain information to provide appropriate care.

[0126] Thus, the system of the present invention can more effectively detect abnormalities by combining biometric information and emotion evaluation data, and can provide a more comprehensive means of ensuring user safety.

[0127] The following describes the processing flow.

[0128] Step 1:

[0129] The device acquires the user's biometric information, such as heart rate, blood oxygen saturation, and impact data from an accelerometer, from the wearable device in real time at regular sampling intervals. This data is temporarily stored within the device.

[0130] Step 2:

[0131] The device transmits the collected biometric information to the server via a secure communication protocol. The transmitted data is encrypted, and measures are taken to ensure data confidentiality.

[0132] Step 3:

[0133] The server stores the received biometric data in a database and prepares it for data analysis. This stored data is used for both anomaly detection and sentiment analysis.

[0134] Step 4:

[0135] The server uses machine learning algorithms to learn normal behavioral patterns from incoming data. Simultaneously, the emotion engine estimates the user's emotional state based on changes in heart rate and blood oxygen saturation. Stress levels and emotional changes are calculated and recorded.

[0136] Step 5:

[0137] The server applies anomaly detection algorithms and analyzes newly received biometric information in real time. When determining anomalies, emotion evaluation data from the emotion engine is used to improve the accuracy of anomaly detection.

[0138] Step 6:

[0139] The server automatically generates a notification when it detects an anomaly or a significant change in emotion. This notification includes information about the type of anomaly, the time it occurred, and the emotion state.

[0140] Step 7:

[0141] The server quickly sends generated notifications to the parent's device. The notifications are timely and tailored to allow for immediate action.

[0142] Step 8:

[0143] Parents, as users, receive notifications on their devices and can check the detailed situation through the application. This allows them to understand whether the user is at risk and take appropriate action immediately.

[0144] This series of processes enables the system to effectively manage user safety by utilizing emotional information.

[0145] (Example 2)

[0146] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0147] In modern society, there is a growing need to effectively and in real time monitor users' health and emotional states to ensure their safety. However, conventional methods are limited to simple monitoring of biometric information, making comprehensive safety management that takes emotional states into account difficult. Data security and the accuracy of anomaly detection are also challenges.

[0148] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0149] In this invention, the server includes means for acquiring the user's biometric information using a biometric information acquisition device, means for encrypting the biometric information and transmitting it to the server via a secure communication protocol, and means for the server to analyze the biometric information and estimate the user's emotional state using a generative AI model. This enables highly accurate anomaly detection by combining the user's emotional state and biometric information, thereby improving security.

[0150] A "biometric information acquisition device" is a device for acquiring a user's physical data in real time, and it has sensors that detect information including heart rate, blood oxygen saturation, and acceleration-based data.

[0151] Encryption is a technology used to protect data, a process that transforms information using specific algorithms to make it difficult for anyone other than authorized individuals to decipher.

[0152] A "secure communication protocol" is a set of standards for securely transmitting data, including technologies to prevent eavesdropping and tampering between the sender and receiver.

[0153] A "server" is a computer system that provides services to other computers on a network, and its role is to store, process, and analyze data.

[0154] A "generative AI model" is a computational model that uses machine learning algorithms to learn patterns from data and perform inferences and predictions based on new data.

[0155] "Emotional state estimation" is the process of using an algorithm to determine the user's psychological state and emotions based on acquired data.

[0156] "Anomaly detection" is the process of identifying unusual changes or patterns based on acquired biometric and emotional data, and reporting them as risks.

[0157] An "external information processing device" refers to a terminal that receives notifications from a server and presents information to the user, such as a smartphone or computer.

[0158] This invention is a system for ensuring safety by monitoring the user's biometric information and emotional state in real time.

[0159] First, the device uses a wearable device to acquire information from the user's heart rate, blood oxygen saturation, and accelerometer. This device is worn on the user's body and has the capability to sense biometric information with high precision.

[0160] The terminal encrypts the acquired data using AES encryption technology and sends it to the server via a secure communication protocol such as SSL / TLS. This protects the data from unauthorized access and eavesdropping from external sources.

[0161] The server decrypts the received data, stores it securely, and analyzes it using a generative AI model. The AI ​​model analyzes fluctuations in heart rate and blood oxygen saturation to estimate the user's emotional state. Specifically, it evaluates the user's stress level using factors such as the rate of change in heart rate.

[0162] If an anomaly is detected, the server automatically generates a notification and sends a report containing details of the emotional state and biometric information to the parent's device. This notification allows parents to immediately check on the user's safety. For example, if a child experiences a sudden increase in heart rate and stress levels due to feeling pressure, the server will immediately send a notification.

[0163] As a concrete example of use, a prompt such as, "Please provide an example of a system in operation that detects emotional stress associated with a child's sudden increase in heart rate and notifies of the abnormality," can be used. In this way, the present invention makes it possible to protect the user's safety in real time.

[0164] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0165] Step 1:

[0166] The device acquires data from the wearable device, including the user's heart rate, blood oxygen saturation, and accelerometer. The input is analog data sent from the sensors, which is then converted to digital data for recording. Specifically, the heart rate sensor captures the heart rate cycle, and the blood oxygen sensor measures oxygen saturation and converts it into a digital signal.

[0167] Step 2:

[0168] The device encrypts the acquired biometric data using the AES encryption algorithm. In this process, digitized biometric information is used as input, and encrypted data is generated as output. Specifically, encryption is performed using an encryption key for each block of data.

[0169] Step 3:

[0170] The terminal sends encrypted data to the server over the network using the secure SSL / TLS protocol. Here, encrypted data is received as input, and the data reaches the server securely as output. Specifically, data packet generation and network handshake are performed.

[0171] Step 4:

[0172] The server decrypts the received encrypted data and converts it into an analyzable format. The input is encrypted data sent from the terminal, and the output is readable biometric data. Specifically, a decryption algorithm is applied using a predetermined key.

[0173] Step 5:

[0174] The server analyzes decoded biometric data using a generative AI model to estimate the user's emotional state. The input is biometric information, and the output generates data determining stress levels and emotional states. Specifically, the AI ​​model performs data analysis and pattern recognition.

[0175] Step 6:

[0176] The server detects anomalies based on emotional state and biometric information. Stress levels and emotional state assessments are used as input, and the output is a determination of whether or not an anomaly is present. Specifically, the anomaly detection algorithm is executed through machine learning.

[0177] Step 7:

[0178] If an anomaly is detected, the server generates a notification and sends it to the parent's information terminal. Here, information about the anomaly is taken as input, and a detailed notification report is generated as output. Specifically, the notification generation tool formats and sends the data.

[0179] (Application Example 2)

[0180] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0181] As autonomous driving technology evolves, a challenge remains: the lack of systems that can grasp the operator's biometric information and emotional state in real time and immediately implement safety measures in the event of an anomaly. In particular, there is a need for the introduction of rapid anomaly detection and notification systems based on changes in the operator's health and emotional state.

[0182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0183] In this invention, the server includes means for acquiring the operator's biometric information using a biometric information detection mechanism, means for analyzing the biometric information and learning a standard movement pattern, and means for detecting anomalies by comparing them with the standard movement pattern. This enables rapid anomaly detection in response to the operator's health status and emotional changes, and the implementation of safety measures by an automatic control device when an anomaly occurs.

[0184] A "biometric information detection mechanism" is a combination of hardware and software for acquiring impact data based on the operator's heart rate, blood oxygen saturation, and acceleration.

[0185] An "operator" is a person who uses this system, and whose biometric information is monitored in real time.

[0186] A "standard dynamic pattern" is a pattern of data learned by recording the operator's normal health status and behavior.

[0187] "Means for detecting abnormalities" refer to hardware and software mechanisms for identifying abnormalities in the operator's health condition by comparing acquired biological information with standard dynamic patterns.

[0188] A "communication device" is a device that transmits notifications generated when an anomaly is detected to the operator, and also provides real-time information about the operator's environment.

[0189] An "automatic control system" is a device that safely controls machinery such as vehicles when it detects an abnormality in the operator.

[0190] The system for implementing this invention provides a function to ensure safety by monitoring the biometric information of operators in real time when they use autonomous vehicles. Through the cooperation of a server and communication devices, the system can instantly detect changes in the operator's health status and emotions, and take safety measures as necessary.

[0191] The server acquires impact data based on the operator's heart rate, blood oxygen saturation, and acceleration via a biometric detection mechanism. This data is encrypted and transmitted to the server via a secure communication protocol. The server analyzes the received biometric information and detects anomalies by comparing it to standard dynamic patterns. If an anomaly is detected, the server automatically generates a notification and displays a warning to the operator via communication equipment.

[0192] As a concrete example, if the operator's heart rate suddenly increases while driving on a highway, the server detects this anomaly. The communication equipment immediately displays a warning on the operator's head-mounted display, and the automatic control system selects a safe stopping point and brings the vehicle to a halt. This ensures the operator's safety.

[0193] An example of a prompt to input into the generating AI model is, "While driving on a highway, the passenger's heart rate suddenly increased. Please suggest safe measures appropriate to this situation." Using this prompt allows for more effective safety measures to be suggested.

[0194] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0195] Step 1:

[0196] The terminal acquires the operator's heart rate, blood oxygen saturation, and impact data based on acceleration via a biometric information detection mechanism. This biometric information is temporarily stored in local memory. The input is the operator's biometric data, and the output is organized biometric information data. Specifically, it aggregates data from each sensor and performs digital signal processing.

[0197] Step 2:

[0198] The device encrypts the acquired biometric data using an encryption algorithm and sends it to the server via a secure communication protocol. The input is organized biometric data, and the output is encrypted data. Specifically, it encrypts the data using AES encryption and sends it to the server via HTTPS.

[0199] Step 3:

[0200] The server decrypts the received encrypted data and performs analysis to compare it with standard dynamic patterns. The input is encrypted data, and the output is analyzed biometric information. Specifically, it compares the data with a database and executes an anomaly detection algorithm.

[0201] Step 4:

[0202] When the server detects an anomaly, it uses a generative AI model to generate appropriate countermeasures based on the operator's state and creates prompt messages. The input is analyzed biometric information, and the output is the prompt message for the countermeasure. Specifically, it runs an emotion engine and provides prompts to the generative AI.

[0203] Step 5:

[0204] The server sends a notification generated based on the prompt message to the communication device, displaying a warning to the operator. The input is the prompt message, and the output is the display on the operator's device. Specifically, it formats a text message and displays it on the screen in real time.

[0205] Step 6:

[0206] The user reviews the displayed notification, confirms the safety measures, and takes appropriate action. Input is the notification message, and output is instructions for action or safety measures. Specific actions include reviewing the displayed information and requesting manual operations or additional information.

[0207] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0208] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0209] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0210] [Second Embodiment]

[0211] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0212] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0213] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0214] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0215] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0216] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0217] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0218] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0219] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0220] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0221] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0222] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0223] This invention provides a wearable device for acquiring biometric information and a system for managing and analyzing that data. The system aims to ensure the safety of children and the elderly by immediately detecting abnormal behavior and notifying relevant parties. To implement this invention, the following configuration and processing are performed.

[0224] First, the device uses a wearable device to continuously collect the user's biometric information, such as heart rate, blood oxygen saturation, and impact data from an accelerometer. This data is transmitted to a server in real time via a secure communication protocol. The server receives the transmitted biometric information and stores it in a database.

[0225] Next, the server uses machine learning based on the collected biometric information to learn normal behavioral patterns. During this learning process, a model is created that reflects general physiological data and the characteristics of individual users, enabling real-time anomaly detection.

[0226] When an anomaly is detected, the server automatically generates a notification. This notification includes the type of anomaly and specific numerical changes, and is sent to a designated external device, such as a parent's device. The parent, as the user, can immediately check the situation and take appropriate action upon receiving this notification. The notification includes the time and location of the anomaly, as well as detailed biometric data, which can be used to assess the situation.

[0227] For example, if a child's heart rate suddenly increases during school recess, the device detects this change and sends it to the server. The server then uses this information to determine that the child's behavior deviates from normal patterns and immediately notifies the parents. This allows parents to quickly contact the school to check on their child's safety or request assistance.

[0228] The above is one embodiment of the present invention, and this system can effectively improve the safety of children and the elderly.

[0229] The following describes the processing flow.

[0230] Step 1:

[0231] The device acquires the user's heart rate, blood oxygen saturation, and impact data from an accelerometer via a wearable device at regular sampling intervals. The acquired data is stored in a temporary buffer.

[0232] Step 2:

[0233] The device encrypts the data and sends it to the server using a secure communication protocol. This transmission occurs in real time for each data collection cycle.

[0234] Step 3:

[0235] The server receives biometric information transmitted from the terminal and stores it in a database. The stored data is used for subsequent analysis and model updates.

[0236] Step 4:

[0237] The server analyzes the received biometric data and uses machine learning algorithms to learn normal behavioral patterns. This model is updated in real time and serves as a criterion for monitoring anomalies.

[0238] Step 5:

[0239] The server compares newly received biometric data in real time with a trained model and detects anomalies that exceed a threshold. If an anomaly is detected, a process is performed to identify the type of anomaly.

[0240] Step 6:

[0241] The server generates a notification based on the identified anomaly. The notification includes information such as the nature of the anomaly, the time it occurred, and the location. This information is essential for prompting appropriate action.

[0242] Step 7:

[0243] The server sends the generated notification to a designated external device (e.g., a parent's device). The communication is designed to occur instantly.

[0244] Step 8:

[0245] Parents, as users, can review received notifications and understand their child's current situation. Depending on the circumstances, they can contact the school or other relevant organizations to take prompt action.

[0246] This series of processes allows the system to detect anomalies in real time and enable a rapid response.

[0247] (Example 1)

[0248] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0249] In modern society, the safety of children and the elderly is a critical issue. In particular, there is a growing need for systems that can quickly detect abnormalities in their health status and behavior and respond appropriately. However, existing technologies struggle to efficiently detect abnormalities from biometric data and promptly notify relevant parties. Solving this problem is essential.

[0250] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0251] In this invention, the server includes means for receiving data using a secure communication protocol, means for learning normal behavioral patterns from the received biometric information using a machine learning algorithm, and means for generating and transmitting notifications to external devices when an anomaly is detected. This makes it possible to monitor the biometric information of children and the elderly in real time and to quickly notify relevant parties in the event of an anomaly.

[0252] A "biometric information detection device" is a device that acquires information about the user's body in real time, specifically by collecting data such as heart rate, blood oxygen saturation, and acceleration using sensors.

[0253] A "communication protocol" is a set of rules and procedures for securely sending and receiving data between a terminal and a server, guaranteeing information encryption and reliable transmission.

[0254] The "central device" is a computer system that collects and analyzes received biological information, and has the function of storing data and detecting anomalies.

[0255] A "machine learning algorithm" is a mathematical method used to learn from past data, recognize patterns, and analyze and predict future data. In this invention, it is used for the analysis of biological information.

[0256] An "external device" is a device used to receive notifications when an abnormality is detected, and typically refers to a mobile device or computer owned by the user's parent or administrator.

[0257] A "notification" is a warning message generated when an anomaly is detected, and it includes the type of anomaly, detailed numerical information, and location information.

[0258] This invention is a system that collects users' biometric information in real time and provides rapid notification in the event of an abnormality. Its aim is to improve the safety of children and the elderly.

[0259] The device continuously acquires the user's heart rate, blood oxygen saturation, and impact data based on acceleration using a biometric detection device. This device includes a heart rate sensor, oxygen saturation monitor, and accelerometer, enabling accurate data collection. This data is transmitted from the device to the server using Bluetooth or Wi-Fi via a secure communication protocol.

[0260] After receiving the transmitted biometric information, the server stores it in a database and analyzes the data using machine learning frameworks such as TensorFlow and PyTorch. This allows the server to learn each user's normal behavior patterns and detect anomalies in real time. When an anomaly is detected, the server immediately generates a notification about the anomaly and sends it to a designated external device, such as a parent's mobile device.

[0261] Parents, as users, can receive notifications from their devices, allowing them to immediately check the situation and take necessary actions if any abnormalities occur with their children or elderly relatives. These notifications include the type of abnormality, the time of occurrence, and specific numerical changes and location information, supporting quick and effective decision-making.

[0262] For example, if a child's heart rate suddenly increases during lunchtime, the device detects this change and sends it to a server. The server identifies the anomaly based on this data and sends a notification to the parent's external device. Upon receiving this notification, the parent can immediately contact the school to check on their child's situation. In this way, the system enables rapid safety checks.

[0263] An example of a prompt might be, "Please provide specific notification content to be sent to parents if their child's heart rate suddenly increased at school yesterday." This would allow the AI ​​model to generate an appropriate and specific notification message.

[0264] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0265] Step 1:

[0266] The device acquires input from a biometric information detection device, including the user's heart rate, blood oxygen saturation, and impact data. This data is measured in real time by various built-in sensors. After preprocessing, such as noise reduction and data standardization, the acquired data is temporarily stored in the device's memory.

[0267] Step 2:

[0268] The device transmits pre-processed biometric information to the server using a secure communication protocol. Specifically, the data is encrypted and sent to the server via Bluetooth or Wi-Fi. This ensures data security and privacy.

[0269] Step 3:

[0270] The server receives biometric data transmitted from the terminal and stores it in a database. The database also stores each user's past biometric information, which can be used to track user behavior patterns. Each data item is stored with a timestamp, making it accessible for later processing.

[0271] Step 4:

[0272] The server learns normal behavioral patterns using machine learning algorithms while referencing biometric information stored in a database. At this stage, the model is trained using historical data, and analysis is performed to detect anomalies based on new and historical data. Specifically, when an abnormal fluctuation occurs, a threshold is set to detect that change as a deviation from the pattern.

[0273] Step 5:

[0274] The server compares newly received biometric information with a learned model and detects anomalies in real time. This detection occurs when a set threshold is exceeded, and if an anomaly is confirmed, an alert flag is set.

[0275] Step 6:

[0276] If an anomaly is detected, the server generates a detailed notification. This notification includes the type of anomaly, the time of occurrence, and the identification of the affected user. Because this notification is generated in real time, a rapid response is possible.

[0277] Step 7:

[0278] The server transmits the generated notification to a specified external device, such as the guardian's terminal. The notification is transmitted via SMS, email, or a dedicated app.

[0279] Step 8:

[0280] The user, who is the guardian, receives the notification on the terminal and checks its content. Based on the notification, the guardian checks the safety status of the child and takes actions such as contacting the school or medical institution if necessary. At this stage, the notification content needs to be clear to facilitate understanding of the impact.

[0281] (Application Example 1)

[0282] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0283] There is a need for a technology that can quickly and accurately monitor the health status of users in various environments, accurately detect abnormalities, appropriately notify relevant parties, and enable them to visually grasp the situation. Such a technology is particularly necessary to ensure the safety of children and the elderly in situations where guardians cannot directly monitor.

[0284] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0285] In this invention, the server includes means for acquiring the health data of the user using a biological information detection device, means for analyzing the health data and learning the normal behavior pattern, and means for detecting an abnormality by comparing with the normal behavior pattern. Thereby, it becomes possible to monitor the health status of the user in real time in various situations, reliably detect abnormalities, and visually notify them.

[0286] A "biological information detection device" is a device for acquiring the health data of a user, and mainly functions as a wearable device.

[0287] "Health data" refers to data related to a living body, including the periodic activity count of the body, the oxygen concentration in the blood, and stimulation data based on movement.

[0288] "Normal behavior pattern" refers to the standard or average pattern of biological information shown by a user when they are healthy.

[0289] An "external device" is a device for receiving notifications based on health data, and a communication terminal, etc. falls under this category.

[0290] A "visual display device" is a device for visually notifying a user of information when an abnormality is detected, and includes an eyewear-type display, etc.

[0291] "Abnormality" refers to a state deviating from the pattern of normal health data, and refers to an event that requires immediate response.

[0292] A "smart device" is an electronic device with high processing power and a communication function, and includes a mobile information terminal, etc.

[0293] To implement the present invention, it is necessary to acquire and analyze the health data of a user using a series of devices and systems.

[0294] The server uses a wearable device as a biological information detection device. This device has the function of continuously acquiring the periodic activity count (heart rate) of the user's body, the oxygen concentration in the blood, and stimulation data based on movement. This health data is transmitted to the server in real time and stored in a database for analysis.

[0295] The server uses a generative AI model based on collected health data to learn normal behavioral patterns. This model reflects both general health data and the individual user's characteristics, giving it the ability to instantly detect deviations from normal behavioral patterns. If an anomaly is detected, the server generates a notification for the smart device and also provides a visual warning to the user via a visual display device.

[0296] For example, if a user's heart rate suddenly increases while exercising outdoors, the wearable device will detect this anomaly. The server will analyze this and determine it to be an anomaly, and display a message on the smart device saying, "A sudden increase in heart rate has been detected. Please rest." A visual display will show this message in the user's field of vision, prompting immediate action.

[0297] An example of a prompt message might be: "Please provide specific interface design proposals for how this health data analysis system should quickly and effectively notify the user when it detects an anomaly."

[0298] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0299] Step 1:

[0300] The device uses a wearable biometric device to acquire the user's health data. This data includes heart rate, blood oxygen saturation, and acceleration. The collected health data is transmitted to the server in real time.

[0301] Step 2:

[0302] The server receives health data acquired from the terminal and stores it in a database. This stored data serves as foundational information for later analysis. Health data is the input, and the output remains in the form of data stored in the database.

[0303] Step 3:

[0304] The server analyzes the stored health data using the generative AI model and learns normal behavior patterns. In this process, a model reflecting the characteristics of each user is constructed, preparing to detect outliers. Past health data is used as input to the model, and a discrimination model within the normal range is generated as output.

[0305] Step 4:

[0306] The server uses the model constructed through analysis to compare the real-time transmitted health data with the normal pattern. Abnormalities are detected through this comparison. Current health data is used as data input, and the presence or absence of abnormalities is determined as output.

[0307] Step 5:

[0308] When an abnormality is detected by the server, a process of generating a notification to the smart device is performed. Specifically, information such as "A sudden increase in heart rate has been detected" is prepared as the notification content. Abnormal information is used as input, and a notification message is used as output.

[0309] Step 6:

[0310] The generated notification is visually displayed on the user's visual display device. Through this visualization, the user can immediately recognize the abnormality and take countermeasures. The content of the notification is used as input, and the information displayed in the user's field of vision is used as output.

[0311] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion recognition model 59 and perform specific processing using the user's emotion.

[0312] This invention is a system for ensuring user safety using biometric information and emotional data derived therefrom. This system incorporates an emotion engine that monitors the user's biometric information in real time, detects abnormalities, and recognizes the user's emotional state. This emotion engine uses the obtained biometric data to evaluate the user's emotional state and provides information for dealing with stressful situations and abnormal circumstances.

[0313] First, the device acquires heart rate, blood oxygen saturation, and impact data from an accelerometer via a wearable device. Next, when the device transmits this data to the server, it encrypts it and protects the data using a secure communication protocol.

[0314] The server stores and analyzes the received data. Here, the emotion engine estimates the user's emotional state based on biometric data. For example, it calculates the stress level by considering heart rate and the rate of change in blood oxygen saturation. The emotion engine also works in conjunction with an anomaly detection model, and is designed to detect abnormalities with higher accuracy when the emotional state deviates from the normal state.

[0315] If an anomaly is detected, the server automatically generates a notification, including information about the user's emotional state, and sends it to the parent's device. Through this notification, the user can immediately verify the user's safety and take necessary actions quickly.

[0316] As a concrete example, suppose a child experiences a sudden increase in heart rate under some kind of pressure, and their stress level is also assessed as high. In this case, the server detects the stress-related anomaly through its emotion engine and notifies the parent with a detailed report, including the emotional state. This notification allows the parent to infer the nature and cause of the stress and obtain information to provide appropriate care.

[0317] Thus, the system of the present invention can more effectively detect abnormalities by combining biometric information and emotion evaluation data, and can provide a more comprehensive means of ensuring user safety.

[0318] The following describes the processing flow.

[0319] Step 1:

[0320] The device acquires the user's biometric information, such as heart rate, blood oxygen saturation, and impact data from an accelerometer, from the wearable device in real time at regular sampling intervals. This data is temporarily stored within the device.

[0321] Step 2:

[0322] The device transmits the collected biometric information to the server via a secure communication protocol. The transmitted data is encrypted, and measures are taken to ensure data confidentiality.

[0323] Step 3:

[0324] The server stores the received biometric data in a database and prepares it for data analysis. This stored data is used for both anomaly detection and sentiment analysis.

[0325] Step 4:

[0326] The server uses machine learning algorithms to learn normal behavioral patterns from incoming data. Simultaneously, the emotion engine estimates the user's emotional state based on changes in heart rate and blood oxygen saturation. Stress levels and emotional changes are calculated and recorded.

[0327] Step 5:

[0328] The server applies anomaly detection algorithms and analyzes newly received biometric information in real time. When determining anomalies, emotion evaluation data from the emotion engine is used to improve the accuracy of anomaly detection.

[0329] Step 6:

[0330] The server automatically generates a notification when it detects an anomaly or a significant change in emotion. This notification includes information about the type of anomaly, the time it occurred, and the emotion state.

[0331] Step 7:

[0332] The server quickly sends generated notifications to the parent's device. The notifications are timely and tailored to allow for immediate action.

[0333] Step 8:

[0334] Parents, as users, receive notifications on their devices and can check the detailed situation through the application. This allows them to understand whether the user is at risk and take appropriate action immediately.

[0335] This series of processes enables the system to effectively manage user safety by utilizing emotional information.

[0336] (Example 2)

[0337] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0338] In modern society, there is a growing need to effectively and in real time monitor users' health and emotional states to ensure their safety. However, conventional methods are limited to simple monitoring of biometric information, making comprehensive safety management that takes emotional states into account difficult. Data security and the accuracy of anomaly detection are also challenges.

[0339] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0340] In this invention, the server includes means for acquiring the user's biometric information using a biometric information acquisition device, means for encrypting the biometric information and transmitting it to the server via a secure communication protocol, and means for the server to analyze the biometric information and estimate the user's emotional state using a generative AI model. This enables highly accurate anomaly detection by combining the user's emotional state and biometric information, thereby improving security.

[0341] A "biometric information acquisition device" is a device for acquiring a user's physical data in real time, and it has sensors that detect information including heart rate, blood oxygen saturation, and acceleration-based data.

[0342] Encryption is a technology used to protect data, a process that transforms information using specific algorithms to make it difficult for anyone other than authorized individuals to decipher.

[0343] A "secure communication protocol" is a set of standards for securely transmitting data, including technologies to prevent eavesdropping and tampering between the sender and receiver.

[0344] A "server" is a computer system that provides services to other computers on a network, and its role is to store, process, and analyze data.

[0345] A "generative AI model" is a computational model that uses machine learning algorithms to learn patterns from data and perform inferences and predictions based on new data.

[0346] "Emotional state estimation" is the process of using an algorithm to determine the user's psychological state and emotions based on acquired data.

[0347] "Anomaly detection" is the process of identifying unusual changes or patterns based on acquired biometric and emotional data, and reporting them as risks.

[0348] An "external information processing device" refers to a terminal that receives notifications from a server and presents information to the user, such as a smartphone or computer.

[0349] This invention is a system for ensuring safety by monitoring the user's biometric information and emotional state in real time.

[0350] First, the device uses a wearable device to acquire information from the user's heart rate, blood oxygen saturation, and accelerometer. This device is worn on the user's body and has the capability to sense biometric information with high precision.

[0351] The terminal encrypts the acquired data using AES encryption technology and sends it to the server via a secure communication protocol such as SSL / TLS. This protects the data from unauthorized access and eavesdropping from external sources.

[0352] The server decrypts the received data, stores it securely, and analyzes it using a generative AI model. The AI ​​model analyzes fluctuations in heart rate and blood oxygen saturation to estimate the user's emotional state. Specifically, it evaluates the user's stress level using factors such as the rate of change in heart rate.

[0353] If an anomaly is detected, the server automatically generates a notification and sends a report containing details of the emotional state and biometric information to the parent's device. This notification allows parents to immediately check on the user's safety. For example, if a child experiences a sudden increase in heart rate and stress levels due to feeling pressure, the server will immediately send a notification.

[0354] As a concrete example of use, a prompt such as, "Please provide an example of a system in operation that detects emotional stress associated with a child's sudden increase in heart rate and notifies of the abnormality," can be used. In this way, the present invention makes it possible to protect the user's safety in real time.

[0355] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0356] Step 1:

[0357] The device acquires data from the wearable device, including the user's heart rate, blood oxygen saturation, and accelerometer. The input is analog data sent from the sensors, which is then converted to digital data for recording. Specifically, the heart rate sensor captures the heart rate cycle, and the blood oxygen sensor measures oxygen saturation and converts it into a digital signal.

[0358] Step 2:

[0359] The device encrypts the acquired biometric data using the AES encryption algorithm. In this process, digitized biometric information is used as input, and encrypted data is generated as output. Specifically, encryption is performed using an encryption key for each block of data.

[0360] Step 3:

[0361] The terminal sends encrypted data to the server over the network using the secure SSL / TLS protocol. Here, encrypted data is received as input, and the data reaches the server securely as output. Specifically, data packet generation and network handshake are performed.

[0362] Step 4:

[0363] The server decrypts the received encrypted data and converts it into an analyzable format. The input is encrypted data sent from the terminal, and the output is readable biometric data. Specifically, a decryption algorithm is applied using a predetermined key.

[0364] Step 5:

[0365] The server analyzes decoded biometric data using a generative AI model to estimate the user's emotional state. The input is biometric information, and the output generates data determining stress levels and emotional states. Specifically, the AI ​​model performs data analysis and pattern recognition.

[0366] Step 6:

[0367] The server detects anomalies based on emotional state and biometric information. Stress levels and emotional state assessments are used as input, and the output is a determination of whether or not an anomaly is present. Specifically, the anomaly detection algorithm is executed through machine learning.

[0368] Step 7:

[0369] If an anomaly is detected, the server generates a notification and sends it to the parent's information terminal. Here, information about the anomaly is taken as input, and a detailed notification report is generated as output. Specifically, the notification generation tool formats and sends the data.

[0370] (Application Example 2)

[0371] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0372] As autonomous driving technology evolves, a challenge remains: the lack of systems that can grasp the operator's biometric information and emotional state in real time and immediately implement safety measures in the event of an anomaly. In particular, there is a need for the introduction of rapid anomaly detection and notification systems based on changes in the operator's health and emotional state.

[0373] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0374] In this invention, the server includes means for acquiring the operator's biometric information using a biometric information detection mechanism, means for analyzing the biometric information and learning a standard movement pattern, and means for detecting anomalies by comparing them with the standard movement pattern. This enables rapid anomaly detection in response to the operator's health status and emotional changes, and the implementation of safety measures by an automatic control device when an anomaly occurs.

[0375] A "biometric information detection mechanism" is a combination of hardware and software for acquiring impact data based on the operator's heart rate, blood oxygen saturation, and acceleration.

[0376] An "operator" is a person who uses this system, and whose biometric information is monitored in real time.

[0377] A "standard dynamic pattern" is a pattern of data learned by recording the operator's normal health status and behavior.

[0378] "Means for detecting abnormalities" refer to hardware and software mechanisms for identifying abnormalities in the operator's health condition by comparing acquired biological information with standard dynamic patterns.

[0379] A "communication device" is a device that transmits notifications generated when an anomaly is detected to the operator, and also provides real-time information about the operator's environment.

[0380] An "automatic control system" is a device that safely controls machinery such as vehicles when it detects an abnormality in the operator.

[0381] The system for implementing this invention provides a function to ensure safety by monitoring the biometric information of operators in real time when they use autonomous vehicles. Through the cooperation of a server and communication devices, the system can instantly detect changes in the operator's health status and emotions, and take safety measures as necessary.

[0382] The server acquires impact data based on the operator's heart rate, blood oxygen saturation, and acceleration via a biometric detection mechanism. This data is encrypted and transmitted to the server via a secure communication protocol. The server analyzes the received biometric information and detects anomalies by comparing it to standard dynamic patterns. If an anomaly is detected, the server automatically generates a notification and displays a warning to the operator via communication equipment.

[0383] As a concrete example, if the operator's heart rate suddenly increases while driving on a highway, the server detects this anomaly. The communication equipment immediately displays a warning on the operator's head-mounted display, and the automatic control system selects a safe stopping point and brings the vehicle to a halt. This ensures the operator's safety.

[0384] An example of a prompt to input into the generating AI model is, "While driving on a highway, the passenger's heart rate suddenly increased. Please suggest safe measures appropriate to this situation." Using this prompt allows for more effective safety measures to be suggested.

[0385] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0386] Step 1:

[0387] The terminal acquires the operator's heart rate, blood oxygen saturation, and impact data based on acceleration via a biometric information detection mechanism. This biometric information is temporarily stored in local memory. The input is the operator's biometric data, and the output is organized biometric information data. Specifically, it aggregates data from each sensor and performs digital signal processing.

[0388] Step 2:

[0389] The device encrypts the acquired biometric data using an encryption algorithm and sends it to the server via a secure communication protocol. The input is organized biometric data, and the output is encrypted data. Specifically, it encrypts the data using AES encryption and sends it to the server via HTTPS.

[0390] Step 3:

[0391] The server decrypts the received encrypted data and performs analysis to compare it with standard dynamic patterns. The input is encrypted data, and the output is analyzed biometric information. Specifically, it compares the data with a database and executes an anomaly detection algorithm.

[0392] Step 4:

[0393] When the server detects an anomaly, it uses a generative AI model to generate appropriate countermeasures based on the operator's state and creates prompt messages. The input is analyzed biometric information, and the output is the prompt message for the countermeasure. Specifically, it runs an emotion engine and provides prompts to the generative AI.

[0394] Step 5:

[0395] The server sends a notification generated based on the prompt message to the communication device, displaying a warning to the operator. The input is the prompt message, and the output is the display on the operator's device. Specifically, it formats a text message and displays it on the screen in real time.

[0396] Step 6:

[0397] The user reviews the displayed notification, confirms the safety measures, and takes appropriate action. Input is the notification message, and output is instructions for action or safety measures. Specific actions include reviewing the displayed information and requesting manual operations or additional information.

[0398] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0399] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0400] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0401] [Third Embodiment]

[0402] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0403] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0404] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0405] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0406] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0407] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0408] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0409] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0410] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0411] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0412] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0413] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0414] This invention provides a wearable device for acquiring biometric information and a system for managing and analyzing that data. The system aims to ensure the safety of children and the elderly by immediately detecting abnormal behavior and notifying relevant parties. To implement this invention, the following configuration and processing are performed.

[0415] First, the device uses a wearable device to continuously collect the user's biometric information, such as heart rate, blood oxygen saturation, and impact data from an accelerometer. This data is transmitted to a server in real time via a secure communication protocol. The server receives the transmitted biometric information and stores it in a database.

[0416] Next, the server uses machine learning based on the collected biometric information to learn normal behavioral patterns. During this learning process, a model is created that reflects general physiological data and the characteristics of individual users, enabling real-time anomaly detection.

[0417] When an anomaly is detected, the server automatically generates a notification. This notification includes the type of anomaly and specific numerical changes, and is sent to a designated external device, such as a parent's device. The parent, as the user, can immediately check the situation and take appropriate action upon receiving this notification. The notification includes the time and location of the anomaly, as well as detailed biometric data, which can be used to assess the situation.

[0418] For example, if a child's heart rate suddenly increases during school recess, the device detects this change and sends it to the server. The server then uses this information to determine that the child's behavior deviates from normal patterns and immediately notifies the parents. This allows parents to quickly contact the school to check on their child's safety or request assistance.

[0419] The above is one embodiment of the present invention, and this system can effectively improve the safety of children and the elderly.

[0420] The following describes the processing flow.

[0421] Step 1:

[0422] The device acquires the user's heart rate, blood oxygen saturation, and impact data from an accelerometer via a wearable device at regular sampling intervals. The acquired data is stored in a temporary buffer.

[0423] Step 2:

[0424] The device encrypts the data and sends it to the server using a secure communication protocol. This transmission occurs in real time for each data collection cycle.

[0425] Step 3:

[0426] The server receives biometric information transmitted from the terminal and stores it in a database. The stored data is used for subsequent analysis and model updates.

[0427] Step 4:

[0428] The server analyzes the received biometric data and uses machine learning algorithms to learn normal behavioral patterns. This model is updated in real time and serves as a criterion for monitoring anomalies.

[0429] Step 5:

[0430] The server compares newly received biometric data in real time with a trained model and detects anomalies that exceed a threshold. If an anomaly is detected, a process is performed to identify the type of anomaly.

[0431] Step 6:

[0432] The server generates a notification based on the identified anomaly. The notification includes information such as the nature of the anomaly, the time it occurred, and the location. This information is essential for prompting appropriate action.

[0433] Step 7:

[0434] The server sends the generated notification to a designated external device (e.g., a parent's device). The communication is designed to occur instantly.

[0435] Step 8:

[0436] Parents, as users, can review received notifications and understand their child's current situation. Depending on the circumstances, they can contact the school or other relevant organizations to take prompt action.

[0437] This series of processes allows the system to detect anomalies in real time and enable a rapid response.

[0438] (Example 1)

[0439] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0440] In modern society, the safety of children and the elderly is a critical issue. In particular, there is a growing need for systems that can quickly detect abnormalities in their health status and behavior and respond appropriately. However, existing technologies struggle to efficiently detect abnormalities from biometric data and promptly notify relevant parties. Solving this problem is essential.

[0441] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0442] In this invention, the server includes means for receiving data using a secure communication protocol, means for learning normal behavioral patterns from the received biometric information using a machine learning algorithm, and means for generating and transmitting notifications to external devices when an anomaly is detected. This makes it possible to monitor the biometric information of children and the elderly in real time and to quickly notify relevant parties in the event of an anomaly.

[0443] A "biometric information detection device" is a device that acquires information about the user's body in real time, specifically by collecting data such as heart rate, blood oxygen saturation, and acceleration using sensors.

[0444] A "communication protocol" is a set of rules and procedures for securely sending and receiving data between a terminal and a server, guaranteeing information encryption and reliable transmission.

[0445] The "central device" is a computer system that collects and analyzes received biological information, and has the function of storing data and detecting anomalies.

[0446] A "machine learning algorithm" is a mathematical method used to learn from past data, recognize patterns, and analyze and predict future data. In this invention, it is used for the analysis of biological information.

[0447] An "external device" is a device used to receive notifications when an abnormality is detected, and typically refers to a mobile device or computer owned by the user's parent or administrator.

[0448] A "notification" is a warning message generated when an anomaly is detected, and it includes the type of anomaly, detailed numerical information, and location information.

[0449] This invention is a system that collects users' biometric information in real time and provides rapid notification in the event of an abnormality. Its aim is to improve the safety of children and the elderly.

[0450] The device continuously acquires the user's heart rate, blood oxygen saturation, and impact data based on acceleration using a biometric detection device. This device includes a heart rate sensor, oxygen saturation monitor, and accelerometer, enabling accurate data collection. This data is transmitted from the device to the server using Bluetooth or Wi-Fi via a secure communication protocol.

[0451] After receiving the transmitted biometric information, the server stores it in a database and analyzes the data using machine learning frameworks such as TensorFlow and PyTorch. This allows the server to learn each user's normal behavior patterns and detect anomalies in real time. When an anomaly is detected, the server immediately generates a notification about the anomaly and sends it to a designated external device, such as a parent's mobile device.

[0452] Parents, as users, can receive notifications from their devices, allowing them to immediately check the situation and take necessary actions if any abnormalities occur with their children or elderly relatives. These notifications include the type of abnormality, the time of occurrence, and specific numerical changes and location information, supporting quick and effective decision-making.

[0453] For example, if a child's heart rate suddenly increases during lunchtime, the device detects this change and sends it to a server. The server identifies the anomaly based on this data and sends a notification to the parent's external device. Upon receiving this notification, the parent can immediately contact the school to check on their child's situation. In this way, the system enables rapid safety checks.

[0454] An example of a prompt might be, "Please provide specific notification content to be sent to parents if their child's heart rate suddenly increased at school yesterday." This would allow the AI ​​model to generate an appropriate and specific notification message.

[0455] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0456] Step 1:

[0457] The device acquires input from a biometric information detection device, including the user's heart rate, blood oxygen saturation, and impact data. This data is measured in real time by various built-in sensors. After preprocessing, such as noise reduction and data standardization, the acquired data is temporarily stored in the device's memory.

[0458] Step 2:

[0459] The device transmits pre-processed biometric information to the server using a secure communication protocol. Specifically, the data is encrypted and sent to the server via Bluetooth or Wi-Fi. This ensures data security and privacy.

[0460] Step 3:

[0461] The server receives biometric data transmitted from the terminal and stores it in a database. The database also stores each user's past biometric information, which can be used to track user behavior patterns. Each data item is stored with a timestamp, making it accessible for later processing.

[0462] Step 4:

[0463] The server learns normal behavioral patterns using machine learning algorithms while referencing biometric information stored in a database. At this stage, the model is trained using historical data, and analysis is performed to detect anomalies based on new and historical data. Specifically, when an abnormal fluctuation occurs, a threshold is set to detect that change as a deviation from the pattern.

[0464] Step 5:

[0465] The server compares newly received biometric information with a learned model and detects anomalies in real time. This detection occurs when a set threshold is exceeded, and if an anomaly is confirmed, an alert flag is set.

[0466] Step 6:

[0467] If an anomaly is detected, the server generates a detailed notification. This notification includes the type of anomaly, the time of occurrence, and the identification of the affected user. Because this notification is generated in real time, a rapid response is possible.

[0468] Step 7:

[0469] The server sends the generated notification to a designated external device, such as a parent's device. The notification is sent via SMS, email, or a dedicated app.

[0470] Step 8:

[0471] Parents, as users, receive notifications on their devices and review their contents. Based on the notifications, parents check their child's safety and take appropriate action, such as contacting schools or medical institutions, as needed. At this stage, the notification content needs to be clear in order to easily understand its impact.

[0472] (Application Example 1)

[0473] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0474] There is a need for technology that can quickly and accurately monitor the health status of users in various environments, precisely detect abnormalities, appropriately notify relevant parties, and visually understand the situation. Such technology is especially necessary to ensure the safety of children and the elderly in situations where parents cannot directly supervise them.

[0475] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0476] In this invention, the server includes means for acquiring user health data using a biometric information detection device, means for analyzing the health data and learning normal behavioral patterns, and means for detecting abnormalities by comparing them with the normal behavioral patterns. This makes it possible to monitor the user's health status in real time in various situations, reliably detect abnormalities, and provide visual notification.

[0477] A "biometric information detection device" is a device used to acquire a user's health data and primarily functions as a wearable device.

[0478] "Health data" refers to biological data, including the number of periodic physical activity levels, blood oxygen levels, and movement-based stimulus data.

[0479] "Normal behavioral patterns" refer to the standard or average patterns of biometric information that a user exhibits when healthy.

[0480] "External devices" refer to devices used to receive notifications based on health data, such as communication terminals.

[0481] A "visual display device" is a device that visually notifies the user of information when an abnormality is detected, and includes eyewear-type displays, among others.

[0482] An "abnormality" refers to a condition that deviates from the normal pattern of health data and is an event that requires immediate attention.

[0483] A "smart device" is an electronic device with advanced processing capabilities and communication functions, and includes personal digital assistants (PADs).

[0484] To realize this invention, it is necessary to acquire and analyze user health data using a series of devices and systems.

[0485] The server uses a wearable device as a biometric information detection device. This device has the function of continuously acquiring the user's cyclical activity rate (heart rate), blood oxygen concentration, and movement-based stimulation data. This health data is transmitted to the server in real time and stored in a database for analysis.

[0486] The server uses a generative AI model based on collected health data to learn normal behavioral patterns. This model reflects both general health data and the individual user's characteristics, giving it the ability to instantly detect deviations from normal behavioral patterns. If an anomaly is detected, the server generates a notification for the smart device and also provides a visual warning to the user via a visual display device.

[0487] For example, if a user's heart rate suddenly increases while exercising outdoors, the wearable device will detect this anomaly. The server will analyze this and determine it to be an anomaly, and display a message on the smart device saying, "A sudden increase in heart rate has been detected. Please rest." A visual display will show this message in the user's field of vision, prompting immediate action.

[0488] An example of a prompt message might be: "Please provide specific interface design proposals for how this health data analysis system should quickly and effectively notify the user when it detects an anomaly."

[0489] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0490] Step 1:

[0491] The device uses a wearable biometric device to acquire the user's health data. This data includes heart rate, blood oxygen saturation, and acceleration. The collected health data is transmitted to the server in real time.

[0492] Step 2:

[0493] The server receives health data acquired from the terminal and stores it in a database. This stored data serves as foundational information for later analysis. Health data is the input, and the output remains in the form of data stored in the database.

[0494] Step 3:

[0495] The server analyzes stored health data using an AI model to learn normal behavioral patterns. During this process, it builds a model that reflects the characteristics of each user, preparing it to detect anomalies. Past health data is used as input to the model, and a model for determining normal ranges is generated as output.

[0496] Step 4:

[0497] The server uses a model built through analysis to compare real-time transmitted health data with normal patterns. This comparison detects anomalies. Current health data is used as input, and the presence or absence of an anomaly is determined as output.

[0498] Step 5:

[0499] When an anomaly is detected, the server initiates a process to generate a notification for the smart device. Specifically, it prepares information such as "A sudden increase in heart rate has been detected" as the notification content. The input is the anomaly information, and the output is the notification message.

[0500] Step 6:

[0501] The generated notification is visually displayed on the user's visual display device. This visualization allows the user to immediately recognize anomalies and take corrective action. The input is the content of the notification, and the output is the information displayed in the user's field of vision.

[0502] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0503] This invention is a system for ensuring user safety using biometric information and emotional data derived therefrom. This system incorporates an emotion engine that monitors the user's biometric information in real time, detects abnormalities, and recognizes the user's emotional state. This emotion engine uses the obtained biometric data to evaluate the user's emotional state and provides information for dealing with stressful situations and abnormal circumstances.

[0504] First, the device acquires heart rate, blood oxygen saturation, and impact data from an accelerometer via a wearable device. Next, when the device transmits this data to the server, it encrypts it and protects the data using a secure communication protocol.

[0505] The server stores and analyzes the received data. Here, the emotion engine estimates the user's emotional state based on biometric data. For example, it calculates the stress level by considering heart rate and the rate of change in blood oxygen saturation. The emotion engine also works in conjunction with an anomaly detection model, and is designed to detect abnormalities with higher accuracy when the emotional state deviates from the normal state.

[0506] If an anomaly is detected, the server automatically generates a notification, including information about the user's emotional state, and sends it to the parent's device. Through this notification, the user can immediately verify the user's safety and take necessary actions quickly.

[0507] As a concrete example, suppose a child experiences a sudden increase in heart rate under some kind of pressure, and their stress level is also assessed as high. In this case, the server detects the stress-related anomaly through its emotion engine and notifies the parent with a detailed report, including the emotional state. This notification allows the parent to infer the nature and cause of the stress and obtain information to provide appropriate care.

[0508] Thus, the system of the present invention can more effectively detect abnormalities by combining biometric information and emotion evaluation data, and can provide a more comprehensive means of ensuring user safety.

[0509] The following describes the processing flow.

[0510] Step 1:

[0511] The device acquires the user's biometric information, such as heart rate, blood oxygen saturation, and impact data from an accelerometer, from the wearable device in real time at regular sampling intervals. This data is temporarily stored within the device.

[0512] Step 2:

[0513] The device transmits the collected biometric information to the server via a secure communication protocol. The transmitted data is encrypted, and measures are taken to ensure data confidentiality.

[0514] Step 3:

[0515] The server stores the received biometric data in a database and prepares it for data analysis. This stored data is used for both anomaly detection and sentiment analysis.

[0516] Step 4:

[0517] The server uses machine learning algorithms to learn normal behavioral patterns from incoming data. Simultaneously, the emotion engine estimates the user's emotional state based on changes in heart rate and blood oxygen saturation. Stress levels and emotional changes are calculated and recorded.

[0518] Step 5:

[0519] The server applies anomaly detection algorithms and analyzes newly received biometric information in real time. When determining anomalies, emotion evaluation data from the emotion engine is used to improve the accuracy of anomaly detection.

[0520] Step 6:

[0521] The server automatically generates a notification when it detects an anomaly or a significant change in emotional state. This notification includes information about the type of anomaly, the time it occurred, and the emotional state.

[0522] Step 7:

[0523] The server quickly sends generated notifications to the parent's device. The notifications are timely and tailored to allow for immediate action.

[0524] Step 8:

[0525] Parents, as users, receive notifications on their devices and can check the detailed situation through the application. This allows them to understand whether the user is at risk and take appropriate action immediately.

[0526] This series of processes enables the system to effectively manage user safety by utilizing emotional information.

[0527] (Example 2)

[0528] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0529] In modern society, there is a growing need to effectively and in real time monitor users' health and emotional states to ensure their safety. However, conventional methods are limited to simple monitoring of biometric information, making comprehensive safety management that takes emotional states into account difficult. Data security and the accuracy of anomaly detection are also challenges.

[0530] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0531] In this invention, the server includes means for acquiring the user's biometric information using a biometric information acquisition device, means for encrypting the biometric information and transmitting it to the server via a secure communication protocol, and means for the server to analyze the biometric information and estimate the user's emotional state using a generative AI model. This enables highly accurate anomaly detection by combining the user's emotional state and biometric information, thereby improving security.

[0532] A "biometric information acquisition device" is a device for acquiring a user's physical data in real time, and it has sensors that detect information including heart rate, blood oxygen saturation, and acceleration-based data.

[0533] Encryption is a technology used to protect data, a process that transforms information using specific algorithms to make it difficult for anyone other than authorized individuals to decipher.

[0534] A "secure communication protocol" is a set of standards for securely transmitting data, including technologies to prevent eavesdropping and tampering between the sender and receiver.

[0535] A "server" is a computer system that provides services to other computers on a network, and its role is to store, process, and analyze data.

[0536] A "generative AI model" is a computational model that uses machine learning algorithms to learn patterns from data and perform inferences and predictions based on new data.

[0537] "Emotional state estimation" is the process of using an algorithm to determine the user's psychological state and emotions based on acquired data.

[0538] "Anomaly detection" is the process of identifying unusual changes or patterns based on acquired biometric and emotional data, and reporting them as risks.

[0539] An "external information processing device" refers to a terminal that receives notifications from a server and presents information to the user, such as a smartphone or computer.

[0540] This invention is a system for ensuring safety by monitoring the user's biometric information and emotional state in real time.

[0541] First, the device uses a wearable device to acquire information from the user's heart rate, blood oxygen saturation, and accelerometer. This device is attached to the user's body and has the capability to sense biometric information with high precision.

[0542] The terminal encrypts the acquired data using AES encryption technology and sends it to the server via a secure communication protocol such as SSL / TLS. This protects the data from unauthorized access and eavesdropping from external sources.

[0543] The server decrypts the received data, stores it securely, and analyzes it using a generative AI model. The AI ​​model analyzes fluctuations in heart rate and blood oxygen saturation to estimate the user's emotional state. Specifically, it evaluates the user's stress level using factors such as the rate of change in heart rate.

[0544] If an anomaly is detected, the server automatically generates a notification and sends a report containing details of the emotional state and biometric information to the parent's device. This notification allows parents to immediately check on the user's safety. For example, if a child experiences a sudden increase in heart rate and stress levels due to feeling pressure, the server will immediately send a notification.

[0545] As a concrete example of use, a prompt such as, "Please provide an example of a system in operation that detects emotional stress associated with a child's sudden increase in heart rate and notifies of the abnormality," can be used. In this way, the present invention makes it possible to protect the user's safety in real time.

[0546] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0547] Step 1:

[0548] The device acquires data from the wearable device, including the user's heart rate, blood oxygen saturation, and accelerometer. The input is analog data sent from the sensors, which is then converted to digital data for recording. Specifically, the heart rate sensor captures the heart rate cycle, and the blood oxygen sensor measures oxygen saturation and converts it into a digital signal.

[0549] Step 2:

[0550] The device encrypts the acquired biometric data using the AES encryption algorithm. In this process, digitized biometric information is used as input, and encrypted data is generated as output. Specifically, encryption is performed using an encryption key for each block of data.

[0551] Step 3:

[0552] The terminal sends encrypted data to the server over the network using the secure SSL / TLS protocol. Here, encrypted data is received as input, and the data reaches the server securely as output. Specifically, data packet generation and network handshake are performed.

[0553] Step 4:

[0554] The server decrypts the received encrypted data and converts it into an analyzable format. The input is encrypted data sent from the terminal, and the output is readable biometric data. Specifically, a decryption algorithm is applied using a predetermined key.

[0555] Step 5:

[0556] The server analyzes decoded biometric data using a generative AI model to estimate the user's emotional state. The input is biometric information, and the output generates data determining stress levels and emotional states. Specifically, the AI ​​model performs data analysis and pattern recognition.

[0557] Step 6:

[0558] The server detects anomalies based on emotional state and biometric information. Stress levels and emotional state assessments are used as input, and the output is a determination of whether or not an anomaly is present. Specifically, the anomaly detection algorithm is executed through machine learning.

[0559] Step 7:

[0560] If an anomaly is detected, the server generates a notification and sends it to the parent's information terminal. Here, information about the anomaly is taken as input, and a detailed notification report is generated as output. Specifically, the notification generation tool formats and sends the data.

[0561] (Application Example 2)

[0562] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0563] As autonomous driving technology evolves, a challenge remains: the lack of systems that can grasp the operator's biometric information and emotional state in real time and immediately implement safety measures in the event of an anomaly. In particular, there is a need for the introduction of rapid anomaly detection and notification systems based on changes in the operator's health and emotional state.

[0564] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0565] In this invention, the server includes means for acquiring the operator's biometric information using a biometric information detection mechanism, means for analyzing the biometric information and learning a standard movement pattern, and means for detecting anomalies by comparing them with the standard movement pattern. This enables rapid anomaly detection in response to the operator's health status and emotional changes, and the implementation of safety measures by an automatic control device when an anomaly occurs.

[0566] A "biometric information detection mechanism" is a combination of hardware and software for acquiring impact data based on the operator's heart rate, blood oxygen saturation, and acceleration.

[0567] An "operator" is a person who uses this system, and whose biometric information is monitored in real time.

[0568] A "standard dynamic pattern" is a pattern of data learned by recording the operator's normal health status and behavior.

[0569] "Means for detecting abnormalities" refer to hardware and software mechanisms for identifying abnormalities in the operator's health condition by comparing acquired biological information with standard dynamic patterns.

[0570] A "communication device" is a device that transmits notifications generated when an anomaly is detected to the operator, and also provides real-time information about the operator's environment.

[0571] An "automatic control system" is a device that safely controls machinery such as vehicles when it detects an abnormality in the operator.

[0572] The system for implementing this invention provides a function to ensure safety by monitoring the biometric information of operators in real time when they use autonomous vehicles. Through the cooperation of a server and communication devices, the system can instantly detect changes in the operator's health status and emotions, and take safety measures as necessary.

[0573] The server acquires impact data based on the operator's heart rate, blood oxygen saturation, and acceleration via a biometric detection mechanism. This data is encrypted and transmitted to the server via a secure communication protocol. The server analyzes the received biometric information and detects anomalies by comparing it to standard dynamic patterns. If an anomaly is detected, the server automatically generates a notification and displays a warning to the operator via communication equipment.

[0574] As a concrete example, if the operator's heart rate suddenly increases while driving on a highway, the server detects this anomaly. The communication equipment immediately displays a warning on the operator's head-mounted display, and the automatic control system selects a safe stopping point and brings the vehicle to a halt. This ensures the operator's safety.

[0575] An example of a prompt to input into the generating AI model is, "While driving on a highway, the passenger's heart rate suddenly increased. Please suggest safe measures appropriate to this situation." Using this prompt allows for more effective safety measures to be suggested.

[0576] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0577] Step 1:

[0578] The terminal acquires the operator's heart rate, blood oxygen saturation, and impact data based on acceleration via a biometric information detection mechanism. This biometric information is temporarily stored in local memory. The input is the operator's biometric data, and the output is organized biometric information data. Specifically, it aggregates data from each sensor and performs digital signal processing.

[0579] Step 2:

[0580] The device encrypts the acquired biometric data using an encryption algorithm and sends it to the server via a secure communication protocol. The input is organized biometric data, and the output is encrypted data. Specifically, it encrypts the data using AES encryption and sends it to the server via HTTPS.

[0581] Step 3:

[0582] The server decrypts the received encrypted data and performs analysis to compare it with standard dynamic patterns. The input is encrypted data, and the output is analyzed biometric information. Specifically, it compares the data with a database and executes an anomaly detection algorithm.

[0583] Step 4:

[0584] When the server detects an anomaly, it uses a generative AI model to generate appropriate countermeasures based on the operator's state and creates prompt messages. The input is analyzed biometric information, and the output is the prompt message for the countermeasure. Specifically, it runs an emotion engine and provides prompts to the generative AI.

[0585] Step 5:

[0586] The server sends a notification generated based on the prompt message to the communication device, displaying a warning to the operator. The input is the prompt message, and the output is the display on the operator's device. Specifically, it formats a text message and displays it on the screen in real time.

[0587] Step 6:

[0588] The user reviews the displayed notification, confirms the safety measures, and takes appropriate action. Input is the notification message, and output is instructions for action or safety measures. Specific actions include reviewing the displayed information and requesting manual operations or additional information.

[0589] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0590] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0591] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0592] [Fourth Embodiment]

[0593] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0594] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0595] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0596] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0597] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0598] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0599] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0600] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0601] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0602] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0603] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0604] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0605] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0606] This invention provides a wearable device for acquiring biometric information and a system for managing and analyzing that data. The system aims to ensure the safety of children and the elderly by immediately detecting abnormal behavior and notifying relevant parties. To implement this invention, the following configuration and processing are performed.

[0607] First, the device uses a wearable device to continuously collect the user's biometric information, such as heart rate, blood oxygen saturation, and impact data from an accelerometer. This data is transmitted to a server in real time via a secure communication protocol. The server receives the transmitted biometric information and stores it in a database.

[0608] Next, the server uses machine learning based on the collected biometric information to learn normal behavioral patterns. During this learning process, a model is created that reflects general physiological data and the characteristics of individual users, enabling real-time anomaly detection.

[0609] When an anomaly is detected, the server automatically generates a notification. This notification includes the type of anomaly and specific numerical changes, and is sent to a designated external device, such as a parent's device. The parent, as the user, can immediately check the situation and take appropriate action upon receiving this notification. The notification includes the time and location of the anomaly, as well as detailed biometric data, which can be used to assess the situation.

[0610] For example, if a child's heart rate suddenly increases during school recess, the device detects this change and sends it to the server. The server then uses this information to determine that the child's behavior deviates from normal patterns and immediately notifies the parents. This allows parents to quickly contact the school to check on their child's safety or request assistance.

[0611] The above is one embodiment of the present invention, and this system can effectively improve the safety of children and the elderly.

[0612] The following describes the processing flow.

[0613] Step 1:

[0614] The device acquires the user's heart rate, blood oxygen saturation, and impact data from an accelerometer via a wearable device at regular sampling intervals. The acquired data is stored in a temporary buffer.

[0615] Step 2:

[0616] The device encrypts the data and sends it to the server using a secure communication protocol. This transmission occurs in real time for each data collection cycle.

[0617] Step 3:

[0618] The server receives biometric information transmitted from the terminal and stores it in a database. The stored data is used for subsequent analysis and model updates.

[0619] Step 4:

[0620] The server analyzes the received biometric data and uses machine learning algorithms to learn normal behavioral patterns. This model is updated in real time and serves as a criterion for monitoring anomalies.

[0621] Step 5:

[0622] The server compares newly received biometric data in real time with a trained model and detects anomalies that exceed a threshold. If an anomaly is detected, a process is performed to identify the type of anomaly.

[0623] Step 6:

[0624] The server generates a notification based on the identified anomaly. The notification includes information such as the nature of the anomaly, the time it occurred, and the location. This information is essential for prompting appropriate action.

[0625] Step 7:

[0626] The server sends the generated notification to a designated external device (e.g., a parent's device). The communication is designed to occur instantly.

[0627] Step 8:

[0628] Parents, as users, can review received notifications and understand their child's current situation. Depending on the circumstances, they can contact the school or other relevant organizations to take prompt action.

[0629] This series of processes allows the system to detect anomalies in real time and enable a rapid response.

[0630] (Example 1)

[0631] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0632] In modern society, the safety of children and the elderly is a critical issue. In particular, there is a growing need for systems that can quickly detect abnormalities in their health status and behavior and respond appropriately. However, existing technologies struggle to efficiently detect abnormalities from biometric data and promptly notify relevant parties. Solving this problem is essential.

[0633] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0634] In this invention, the server includes means for receiving data using a secure communication protocol, means for learning normal behavioral patterns from the received biometric information using a machine learning algorithm, and means for generating and transmitting notifications to external devices when an anomaly is detected. This makes it possible to monitor the biometric information of children and the elderly in real time and to quickly notify relevant parties in the event of an anomaly.

[0635] A "biometric information detection device" is a device that acquires information about the user's body in real time, specifically by collecting data such as heart rate, blood oxygen saturation, and acceleration using sensors.

[0636] A "communication protocol" is a set of rules and procedures for securely sending and receiving data between a terminal and a server, guaranteeing information encryption and reliable transmission.

[0637] The "central device" is a computer system that collects and analyzes received biological information, and has the function of storing data and detecting anomalies.

[0638] A "machine learning algorithm" is a mathematical method used to learn from past data, recognize patterns, and analyze and predict future data. In this invention, it is used for the analysis of biological information.

[0639] An "external device" is a device used to receive notifications when an abnormality is detected, and typically refers to a mobile device or computer owned by the user's parent or administrator.

[0640] A "notification" is a warning message generated when an anomaly is detected, and it includes the type of anomaly, detailed numerical information, and location information.

[0641] This invention is a system that collects users' biometric information in real time and provides rapid notification in the event of an abnormality. Its aim is to improve the safety of children and the elderly.

[0642] The device continuously acquires the user's heart rate, blood oxygen saturation, and impact data based on acceleration using a biometric detection device. This device includes a heart rate sensor, oxygen saturation monitor, and accelerometer, enabling accurate data collection. This data is transmitted from the device to the server using Bluetooth or Wi-Fi via a secure communication protocol.

[0643] After receiving the transmitted biometric information, the server stores it in a database and analyzes the data using machine learning frameworks such as TensorFlow and PyTorch. This allows the server to learn each user's normal behavior patterns and detect anomalies in real time. When an anomaly is detected, the server immediately generates a notification about the anomaly and sends it to a designated external device, such as a parent's mobile device.

[0644] Parents, as users, can receive notifications from their devices, allowing them to immediately check the situation and take necessary actions if any abnormalities occur with their children or elderly relatives. These notifications include the type of abnormality, the time of occurrence, and specific numerical changes and location information, supporting quick and effective decision-making.

[0645] For example, if a child's heart rate suddenly increases during lunchtime, the device detects this change and sends it to a server. The server identifies the anomaly based on this data and sends a notification to the parent's external device. Upon receiving this notification, the parent can immediately contact the school to check on their child's situation. In this way, the system enables rapid safety checks.

[0646] An example of a prompt might be, "Please provide specific notification content to be sent to parents if their child's heart rate suddenly increased at school yesterday." This would allow the AI ​​model to generate an appropriate and specific notification message.

[0647] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0648] Step 1:

[0649] The device acquires input from a biometric information detection device, including the user's heart rate, blood oxygen saturation, and impact data. This data is measured in real time by various built-in sensors. After preprocessing, such as noise reduction and data standardization, the acquired data is temporarily stored in the device's memory.

[0650] Step 2:

[0651] The device transmits pre-processed biometric information to the server using a secure communication protocol. Specifically, the data is encrypted and sent to the server via Bluetooth or Wi-Fi. This ensures data security and privacy.

[0652] Step 3:

[0653] The server receives biometric data transmitted from the terminal and stores it in a database. The database also stores each user's past biometric information, which can be used to track user behavior patterns. Each data item is stored with a timestamp, making it accessible for later processing.

[0654] Step 4:

[0655] The server learns normal behavioral patterns using machine learning algorithms while referencing biometric information stored in a database. At this stage, the model is trained using historical data, and analysis is performed to detect anomalies based on new and historical data. Specifically, when an abnormal fluctuation occurs, a threshold is set to detect that change as a deviation from the pattern.

[0656] Step 5:

[0657] The server compares newly received biometric information with a learned model and detects anomalies in real time. This detection occurs when a set threshold is exceeded, and if an anomaly is confirmed, an alert flag is set.

[0658] Step 6:

[0659] If an anomaly is detected, the server generates a detailed notification. This notification includes the type of anomaly, the time of occurrence, and the identification of the affected user. Because this notification is generated in real time, a rapid response is possible.

[0660] Step 7:

[0661] The server sends the generated notification to a designated external device, such as a parent's device. The notification is sent via SMS, email, or a dedicated app.

[0662] Step 8:

[0663] Parents, as users, receive notifications on their devices and review their contents. Based on the notifications, parents check their child's safety and take appropriate action, such as contacting schools or medical institutions, as needed. At this stage, the notification content needs to be clear in order to easily understand its impact.

[0664] (Application Example 1)

[0665] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0666] There is a need for technology that can quickly and accurately monitor the health status of users in various environments, precisely detect abnormalities, appropriately notify relevant parties, and visually understand the situation. Such technology is especially necessary to ensure the safety of children and the elderly in situations where parents cannot directly supervise them.

[0667] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0668] In this invention, the server includes means for acquiring user health data using a biometric information detection device, means for analyzing the health data and learning normal behavioral patterns, and means for detecting abnormalities by comparing them with the normal behavioral patterns. This makes it possible to monitor the user's health status in real time in various situations, reliably detect abnormalities, and provide visual notification.

[0669] A "biometric information detection device" is a device used to acquire a user's health data and primarily functions as a wearable device.

[0670] "Health data" refers to biological data, including the number of periodic physical activity levels, blood oxygen levels, and movement-based stimulus data.

[0671] "Normal behavioral patterns" refer to the standard or average patterns of biometric information that a user exhibits when healthy.

[0672] "External devices" refer to devices used to receive notifications based on health data, such as communication terminals.

[0673] A "visual display device" is a device that visually notifies the user of information when an abnormality is detected, and includes eyewear-type displays, among others.

[0674] An "abnormality" refers to a condition that deviates from the normal pattern of health data and is an event that requires immediate attention.

[0675] A "smart device" is an electronic device with advanced processing capabilities and communication functions, and includes personal digital assistants (PADs).

[0676] To realize this invention, it is necessary to acquire and analyze user health data using a series of devices and systems.

[0677] The server uses a wearable device as a biometric information detection device. This device has the function of continuously acquiring the user's cyclical activity rate (heart rate), blood oxygen concentration, and movement-based stimulation data. This health data is transmitted to the server in real time and stored in a database for analysis.

[0678] The server uses a generative AI model based on collected health data to learn normal behavioral patterns. This model reflects both general health data and the individual user's characteristics, giving it the ability to instantly detect deviations from normal behavioral patterns. If an anomaly is detected, the server generates a notification for the smart device and also provides a visual warning to the user via a visual display device.

[0679] For example, if a user's heart rate suddenly increases while exercising outdoors, the wearable device will detect this anomaly. The server will analyze this and determine it to be an anomaly, and display a message on the smart device saying, "A sudden increase in heart rate has been detected. Please rest." A visual display will show this message in the user's field of vision, prompting immediate action.

[0680] An example of a prompt message might be: "Please provide specific interface design proposals for how this health data analysis system should quickly and effectively notify the user when it detects an anomaly."

[0681] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0682] Step 1:

[0683] The device uses a wearable biometric device to acquire the user's health data. This data includes heart rate, blood oxygen saturation, and acceleration. The collected health data is transmitted to the server in real time.

[0684] Step 2:

[0685] The server receives health data acquired from the terminal and stores it in a database. This stored data serves as foundational information for later analysis. Health data is the input, and the output remains in the form of data stored in the database.

[0686] Step 3:

[0687] The server analyzes stored health data using an AI model to learn normal behavioral patterns. During this process, it builds a model that reflects the characteristics of each user, preparing it to detect anomalies. Past health data is used as input to the model, and a model for determining normal ranges is generated as output.

[0688] Step 4:

[0689] The server uses a model built through analysis to compare real-time transmitted health data with normal patterns. This comparison detects anomalies. Current health data is used as input, and the presence or absence of an anomaly is determined as output.

[0690] Step 5:

[0691] When an anomaly is detected, the server initiates a process to generate a notification for the smart device. Specifically, it prepares information such as "A sudden increase in heart rate has been detected" as the notification content. The input is the anomaly information, and the output is the notification message.

[0692] Step 6:

[0693] The generated notification is visually displayed on the user's visual display device. This visualization allows the user to immediately recognize anomalies and take corrective action. The input is the content of the notification, and the output is the information displayed in the user's field of vision.

[0694] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0695] This invention is a system for ensuring user safety using biometric information and emotional data derived therefrom. This system incorporates an emotion engine that monitors the user's biometric information in real time, detects abnormalities, and recognizes the user's emotional state. This emotion engine uses the obtained biometric data to evaluate the user's emotional state and provides information for dealing with stressful situations and abnormal circumstances.

[0696] First, the device acquires heart rate, blood oxygen saturation, and impact data from an accelerometer via a wearable device. Next, when the device transmits this data to the server, it encrypts it and protects the data using a secure communication protocol.

[0697] The server stores and analyzes the received data. Here, the emotion engine estimates the user's emotional state based on biometric data. For example, it calculates the stress level by considering heart rate and the rate of change in blood oxygen saturation. The emotion engine also works in conjunction with an anomaly detection model, and is designed to detect abnormalities with higher accuracy when the emotional state deviates from the normal state.

[0698] If an anomaly is detected, the server automatically generates a notification, including information about the user's emotional state, and sends it to the parent's device. Through this notification, the user can immediately verify the user's safety and take necessary actions quickly.

[0699] As a concrete example, suppose a child experiences a sudden increase in heart rate under some kind of pressure, and their stress level is also assessed as high. In this case, the server detects the stress-related anomaly through its emotion engine and notifies the parent with a detailed report, including the emotional state. This notification allows the parent to infer the nature and cause of the stress and obtain information to provide appropriate care.

[0700] Thus, the system of the present invention can more effectively detect abnormalities by combining biometric information and emotion evaluation data, and can provide a more comprehensive means of ensuring user safety.

[0701] The following describes the processing flow.

[0702] Step 1:

[0703] The device acquires the user's biometric information, such as heart rate, blood oxygen saturation, and impact data from an accelerometer, from the wearable device in real time at regular sampling intervals. This data is temporarily stored within the device.

[0704] Step 2:

[0705] The device transmits the collected biometric information to the server via a secure communication protocol. The transmitted data is encrypted, and measures are taken to ensure data confidentiality.

[0706] Step 3:

[0707] The server stores the received biometric data in a database and prepares it for data analysis. This stored data is used for both anomaly detection and sentiment analysis.

[0708] Step 4:

[0709] The server uses machine learning algorithms to learn normal behavioral patterns from incoming data. Simultaneously, the emotion engine estimates the user's emotional state based on changes in heart rate and blood oxygen saturation. Stress levels and emotional changes are calculated and recorded.

[0710] Step 5:

[0711] The server applies anomaly detection algorithms and analyzes newly received biometric information in real time. When determining anomalies, emotion evaluation data from the emotion engine is used to improve the accuracy of anomaly detection.

[0712] Step 6:

[0713] The server automatically generates a notification when it detects an anomaly or a significant change in emotion. This notification includes information about the type of anomaly, the time it occurred, and the emotion state.

[0714] Step 7:

[0715] The server quickly sends generated notifications to the parent's device. The notifications are timely and tailored to allow for immediate action.

[0716] Step 8:

[0717] Parents, as users, receive notifications on their devices and can check the detailed situation through the application. This allows them to understand whether the user is at risk and take appropriate action immediately.

[0718] This series of processes enables the system to effectively manage user safety by utilizing emotional information.

[0719] (Example 2)

[0720] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0721] In modern society, there is a growing need to effectively and in real time monitor users' health and emotional states to ensure their safety. However, conventional methods are limited to simple monitoring of biometric information, making comprehensive safety management that takes emotional states into account difficult. Data security and the accuracy of anomaly detection are also challenges.

[0722] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0723] In this invention, the server includes means for acquiring the user's biometric information using a biometric information acquisition device, means for encrypting the biometric information and transmitting it to the server via a secure communication protocol, and means for the server to analyze the biometric information and estimate the user's emotional state using a generative AI model. This enables highly accurate anomaly detection by combining the user's emotional state and biometric information, thereby improving security.

[0724] A "biometric information acquisition device" is a device for acquiring a user's physical data in real time, and it has sensors that detect information including heart rate, blood oxygen saturation, and acceleration-based data.

[0725] Encryption is a technology used to protect data, a process that transforms information using specific algorithms to make it difficult for anyone other than authorized individuals to decipher.

[0726] A "secure communication protocol" is a set of standards for securely transmitting data, including technologies to prevent eavesdropping and tampering between the sender and receiver.

[0727] A "server" is a computer system that provides services to other computers on a network, and its role is to store, process, and analyze data.

[0728] A "generative AI model" is a computational model that uses machine learning algorithms to learn patterns from data and perform inferences and predictions based on new data.

[0729] "Emotional state estimation" is the process of using an algorithm to determine the user's psychological state and emotions based on acquired data.

[0730] "Anomaly detection" is the process of identifying unusual changes or patterns based on acquired biometric and emotional data, and reporting them as risks.

[0731] An "external information processing device" refers to a terminal that receives notifications from a server and presents information to the user, such as a smartphone or computer.

[0732] This invention is a system for ensuring safety by monitoring the user's biometric information and emotional state in real time.

[0733] First, the device uses a wearable device to acquire information from the user's heart rate, blood oxygen saturation, and accelerometer. This device is worn on the user's body and has the capability to sense biometric information with high precision.

[0734] The terminal encrypts the acquired data using AES encryption technology and sends it to the server via a secure communication protocol such as SSL / TLS. This protects the data from unauthorized access and eavesdropping from external sources.

[0735] The server decrypts the received data, stores it securely, and analyzes it using a generative AI model. The AI ​​model analyzes fluctuations in heart rate and blood oxygen saturation to estimate the user's emotional state. Specifically, it evaluates the user's stress level using factors such as the rate of change in heart rate.

[0736] If an anomaly is detected, the server automatically generates a notification and sends a report containing details of the emotional state and biometric information to the parent's device. This notification allows parents to immediately check on the user's safety. For example, if a child experiences a sudden increase in heart rate and stress levels due to feeling pressure, the server will immediately send a notification.

[0737] As a concrete example of use, a prompt such as, "Please provide an example of a system in operation that detects emotional stress associated with a child's sudden increase in heart rate and notifies of the abnormality," can be used. In this way, the present invention makes it possible to protect the user's safety in real time.

[0738] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0739] Step 1:

[0740] The device acquires data from the wearable device, including the user's heart rate, blood oxygen saturation, and accelerometer. The input is analog data sent from the sensors, which is then converted to digital data for recording. Specifically, the heart rate sensor captures the heart rate cycle, and the blood oxygen sensor measures oxygen saturation and converts it into a digital signal.

[0741] Step 2:

[0742] The device encrypts the acquired biometric data using the AES encryption algorithm. In this process, digitized biometric information is used as input, and encrypted data is generated as output. Specifically, encryption is performed using an encryption key for each block of data.

[0743] Step 3:

[0744] The terminal sends encrypted data to the server over the network using the secure SSL / TLS protocol. Here, encrypted data is received as input, and the data reaches the server securely as output. Specifically, data packet generation and network handshake are performed.

[0745] Step 4:

[0746] The server decrypts the received encrypted data and converts it into an analyzable format. The input is encrypted data sent from the terminal, and the output is readable biometric data. Specifically, a decryption algorithm is applied using a predetermined key.

[0747] Step 5:

[0748] The server analyzes decoded biometric data using a generative AI model to estimate the user's emotional state. The input is biometric information, and the output generates data determining stress levels and emotional states. Specifically, the AI ​​model performs data analysis and pattern recognition.

[0749] Step 6:

[0750] The server detects anomalies based on emotional state and biometric information. Stress levels and emotional state assessments are used as input, and the output is a determination of whether or not an anomaly is present. Specifically, the anomaly detection algorithm is executed through machine learning.

[0751] Step 7:

[0752] If an anomaly is detected, the server generates a notification and sends it to the parent's information terminal. Here, information about the anomaly is taken as input, and a detailed notification report is generated as output. Specifically, the notification generation tool formats and sends the data.

[0753] (Application Example 2)

[0754] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0755] As autonomous driving technology evolves, a challenge remains: the lack of systems that can grasp the operator's biometric information and emotional state in real time and immediately implement safety measures in the event of an anomaly. In particular, there is a need for the introduction of rapid anomaly detection and notification systems based on changes in the operator's health and emotional state.

[0756] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0757] In this invention, the server includes means for acquiring the operator's biometric information using a biometric information detection mechanism, means for analyzing the biometric information and learning a standard movement pattern, and means for detecting anomalies by comparing them with the standard movement pattern. This enables rapid anomaly detection in response to the operator's health status and emotional changes, and the implementation of safety measures by an automatic control device when an anomaly occurs.

[0758] A "biometric information detection mechanism" is a combination of hardware and software for acquiring impact data based on the operator's heart rate, blood oxygen saturation, and acceleration.

[0759] An "operator" is a person who uses this system, and whose biometric information is monitored in real time.

[0760] A "standard dynamic pattern" is a pattern of data learned by recording the operator's normal health status and behavior.

[0761] "Means for detecting abnormalities" refer to hardware and software mechanisms for identifying abnormalities in the operator's health condition by comparing acquired biological information with standard dynamic patterns.

[0762] A "communication device" is a device that transmits notifications generated when an anomaly is detected to the operator, and also provides real-time information about the operator's environment.

[0763] An "automatic control system" is a device that safely controls machinery such as vehicles when it detects an abnormality in the operator.

[0764] The system for implementing this invention provides a function to ensure safety by monitoring the biometric information of operators in real time when they use autonomous vehicles. Through the cooperation of a server and communication devices, the system can instantly detect changes in the operator's health status and emotions, and take safety measures as necessary.

[0765] The server acquires impact data based on the operator's heart rate, blood oxygen saturation, and acceleration via a biometric detection mechanism. This data is encrypted and transmitted to the server via a secure communication protocol. The server analyzes the received biometric information and detects anomalies by comparing it to standard dynamic patterns. If an anomaly is detected, the server automatically generates a notification and displays a warning to the operator via communication equipment.

[0766] As a concrete example, if the operator's heart rate suddenly increases while driving on a highway, the server detects this anomaly. The communication equipment immediately displays a warning on the operator's head-mounted display, and the automatic control system selects a safe stopping point and brings the vehicle to a halt. This ensures the operator's safety.

[0767] An example of a prompt to input into the generating AI model is, "While driving on a highway, the passenger's heart rate suddenly increased. Please suggest safe measures appropriate to this situation." Using this prompt allows for more effective safety measures to be suggested.

[0768] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0769] Step 1:

[0770] The terminal acquires the operator's heart rate, blood oxygen saturation, and impact data based on acceleration via a biometric information detection mechanism. This biometric information is temporarily stored in local memory. The input is the operator's biometric data, and the output is organized biometric information data. Specifically, it aggregates data from each sensor and performs digital signal processing.

[0771] Step 2:

[0772] The device encrypts the acquired biometric data using an encryption algorithm and sends it to the server via a secure communication protocol. The input is organized biometric data, and the output is encrypted data. Specifically, it encrypts the data using AES encryption and sends it to the server via HTTPS.

[0773] Step 3:

[0774] The server decrypts the received encrypted data and performs analysis to compare it with standard dynamic patterns. The input is encrypted data, and the output is analyzed biometric information. Specifically, it compares the data with a database and executes an anomaly detection algorithm.

[0775] Step 4:

[0776] When the server detects an anomaly, it uses a generative AI model to generate appropriate countermeasures based on the operator's state and creates prompt messages. The input is analyzed biometric information, and the output is the prompt message for the countermeasure. Specifically, it runs an emotion engine and provides prompts to the generative AI.

[0777] Step 5:

[0778] The server sends a notification generated based on the prompt message to the communication device, displaying a warning to the operator. The input is the prompt message, and the output is the display on the operator's device. Specifically, it formats a text message and displays it on the screen in real time.

[0779] Step 6:

[0780] The user reviews the displayed notification, confirms the safety measures, and takes appropriate action. Input is the notification message, and output is instructions for action or safety measures. Specific actions include reviewing the displayed information and requesting manual operations or additional information.

[0781] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0782] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0783] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0784] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0785] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0786] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0787] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0788] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0789] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0790] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0791] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0792] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0793] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0794] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0795] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0796] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using this memory.

[0797] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0798] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0799] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0800] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0801] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0802] The following is further disclosed regarding the embodiments described above.

[0803] (Claim 1)

[0804] A means for acquiring the user's biometric information using a biometric information detection device,

[0805] A means for analyzing the aforementioned biological information and learning normal behavioral patterns,

[0806] Means for detecting abnormalities by comparing them with the aforementioned normal behavioral patterns,

[0807] A means for generating a notification and sending it to an external device when an anomaly is detected,

[0808] A system that includes this.

[0809] (Claim 2)

[0810] The system according to claim 1, wherein the biological information includes heart rate, blood oxygen concentration, and impact data based on acceleration.

[0811] (Claim 3)

[0812] The system according to claim 1, wherein the external device is related to the parent's terminal.

[0813] "Example 1"

[0814] (Claim 1)

[0815] A means for acquiring the user's biometric information using a biometric information detection device,

[0816] Means for transmitting the aforementioned biometric information to a central device using a secure communication protocol,

[0817] The central device includes means for storing biological information received,

[0818] A means of learning normal behavioral patterns from stored biometric information using machine learning algorithms,

[0819] Means for detecting abnormalities by comparing them with the aforementioned normal behavioral patterns,

[0820] A means for generating a notification including the type of abnormality and specific numerical changes when an abnormality is detected, and transmitting it to an external device,

[0821] A means by which an external device receives an abnormality reported by the user and takes appropriate action according to the situation,

[0822] A system that includes this.

[0823] (Claim 2)

[0824] The system according to claim 1, wherein the biometric information includes heart rate, blood oxygen concentration, and activity-based impact data.

[0825] (Claim 3)

[0826] The system according to claim 1, wherein the external device is related to a terminal for verifying the safety of the user.

[0827] "Application Example 1"

[0828] (Claim 1)

[0829] A means of acquiring a user's health data using a biometric information detection device,

[0830] A means for analyzing the aforementioned health data and learning normal behavioral patterns,

[0831] Means for detecting abnormalities compared to the normal behavioral pattern,

[0832] A means for generating a notification on a smart device and transmitting information to an external device when an anomaly is detected,

[0833] A means of visually notifying abnormalities using a visual display device,

[0834] A system that includes this.

[0835] (Claim 2)

[0836] The system according to claim 1, wherein the health data includes the number of periodic activity levels of the organism, the oxygen concentration in the blood, and motion-based stimulation data.

[0837] (Claim 3)

[0838] The system according to claim 1, wherein the external device is related to a parent's communication terminal.

[0839] "Example 2 of combining an emotion engine"

[0840] (Claim 1)

[0841] A means of acquiring a user's biometric information using a biometric information acquisition device,

[0842] A means for encrypting the aforementioned biometric information and transmitting it to a server via a secure communication protocol,

[0843] A means for analyzing the aforementioned biometric information on a server and estimating the user's emotional state using a generated AI model,

[0844] A means of detecting anomalies by combining the user's emotional state and biometric information,

[0845] A means for generating a notification when an anomaly is detected and transmitting it to an external information processing device,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, wherein the biological information includes heart rate, blood oxygen concentration, and impact data based on acceleration.

[0849] (Claim 3)

[0850] The system according to claim 1, wherein the external information processing device is related to a parent's information terminal.

[0851] "Application example 2 when combining with an emotional engine"

[0852] (Claim 1)

[0853] A means for acquiring the operator's biological information using a biological information detection mechanism,

[0854] A means for analyzing the aforementioned biological information and learning standard dynamic patterns,

[0855] Means for detecting an anomaly compared with the standard dynamic pattern,

[0856] A means for generating a notification and sending it to a communication device when an anomaly is detected,

[0857] The aforementioned communication device provides means for presenting environmental information to the operator in real time,

[0858] A means of implementing safety measures in the event of an abnormality in conjunction with an automatic control system,

[0859] A system that includes this.

[0860] (Claim 2)

[0861] The system according to claim 1, wherein the biological information includes cardiac rate, blood oxygen concentration, and impact data based on acceleration.

[0862] (Claim 3)

[0863] The system according to claim 1, wherein the communication device is related to an operator management device. [Explanation of Symbols]

[0864] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring the user's biometric information using a biometric information detection device, A means for analyzing the aforementioned biological information and learning normal behavioral patterns, Means for detecting abnormalities by comparing them with the aforementioned normal behavioral patterns, A means for generating a notification and sending it to an external device when an anomaly is detected, A system that includes this.

2. The system according to claim 1, wherein the biological information includes heart rate, blood oxygen concentration, and impact data based on acceleration.

3. The system according to claim 1, wherein the external device is related to the parent's terminal.

Images