System

Abstract

A system is provided.SOLUTION: A system including terminal means equipped with a camera and a sensor for capturing health data in real time, means for encrypting and transmitting the captured image data and audio data to a server, server means for analyzing the received data using a generative artificial intelligence model and determining a medical condition, means for generating appropriate advice and primary treatment instructions for a user based on the analysis result, means for transmitting the generated advice and instructions to the terminal to notify the user, and means for using the recorded data to cooperate with a medical institution.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern society is facing problems such as rising medical costs, increasing strain on medical facilities, and a sharp increase in medical demand due to an aging society. These problems make it difficult to provide prompt and appropriate medical care when medical facilities are closed or for people living in remote areas. Furthermore, many patients have poor self-management skills, leading to unnecessary medical expenses. There is a need for a system that can solve these problems and provide efficient and prompt health management and medical support. [Means for solving the problem]

[0005] This invention relates to a system including a terminal equipped with a camera and sensors for capturing health data in real time, a server for encrypting the captured image and audio data and transmitting them to a server, a server for analyzing the received data using a generative artificial intelligence model to determine the condition of the patient, a server for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, a server for transmitting the generated advice and instructions to the terminal and notifying the user, and a system for using the recorded data to connect with a medical institution. The system further includes a database for comparing the analysis results and current health data with past health data and medical history, a database for recording the analysis results and advice content, and a database for continuously training the generative artificial intelligence model to improve its accuracy based on user feedback and diagnostic results from medical institutions.

[0006] "Cameras and sensors" are devices that capture images of the user's face and body, as well as various vital signs (body temperature, heart rate, etc.) in real time.

[0007] A "terminal" is a wearable device worn by a user that has the function of collecting health data in real time, encrypting it, and transmitting it to a server.

[0008] "Encryption" is the process of ensuring the privacy of captured data by making it readable only to authorized recipients.

[0009] A "server" is a computer system that receives encrypted data sent from a terminal and performs analysis and data management.

[0010] A "generative artificial intelligence model" is an advanced AI algorithm used to analyze health data and provide medical diagnosis and advice.

[0011] "Analysis" is the process of evaluating collected health data and determining symptoms and health conditions.

[0012] "Advice" and "primary treatment instructions" are specific instructions for action or recommendations regarding health management provided to the user based on the analysis results.

[0013] "Notification" refers to the act of visually or audibly communicating generated advice or instructions for primary action to the user.

[0014] A "database" is a recording system that stores analysis results and advice over the long term for future reference and analysis.

[0015] "Feedback" refers to information collected to improve system performance, such as users' reactions after using the system and medical institutions' diagnosis results.

[0016] "Continuous training" is the process of regularly improving the algorithm based on feedback so that the generative artificial intelligence model can provide up-to-date and accurate medical advice. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that captures and analyzes a user's health data in real time and provides appropriate advice and primary treatment instructions. To implement this system, the following major components are required:

[0039] 1. Device and System Configuration

[0040] Device: AI glasses are a wearable device worn by the user. Equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.), they capture health data in real time. They also have the ability to collect image and audio data, encrypt it, and send it to a server.

[0041] Server: Receives the captured data and analyzes the medical condition using a generative artificial intelligence model. Based on the analysis results, it generates appropriate advice and instructions for primary treatment and sends them to the device.

[0042] Generative artificial intelligence model: An advanced AI algorithm that analyzes health data, determines medical conditions, and generates advice.

[0043] 2. System operation explanation

[0044] The system operates as follows.

[0045] Data collection: The user puts on the AI ​​glasses and activates them. The device's camera and sensors capture real-time health data, such as the user's complexion, facial expressions, body temperature, and heart rate. Using the voice input function, the user records their current symptoms and feelings.

[0046] Data encryption and transmission: The device organizes the captured image and audio data in real time, encrypts it for privacy purposes, and then transmits the data to the server.

[0047] Data analysis: The server decrypts the received encrypted data and inputs it into a generative AI model. The model analyzes facial color, facial expressions, and physical characteristics from the image data, and uses natural language processing to extract the user's complaints and subjective symptoms from the voice data. Past health data and medical history are also referenced, and the data is compared with current data.

[0048] Advice generation: Based on the analysis results, the generative AI model generates advice and instructions for first-line treatment for the user. For example, if cold symptoms are observed, basic advice such as "drink plenty of fluids and rest" is provided. If more urgent symptoms are observed, emergency response instructions such as "call an ambulance immediately" are issued.

[0049] Notification and implementation: The server sends the generated advice and instructions to the terminal, which notifies the user visually or audibly. The user follows the displayed advice and instructions and takes the necessary steps or actions.

[0050] Specific examples

[0051] Example 1: Early symptoms of a cold

[0052] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[0053] 2. Device: Captures the user's complexion and body temperature, and records symptoms such as "sore throat" or "headache" via voice input.

[0054] 3. Server: Analyzes the captured data and determines whether early symptoms of a cold are present.

[0055] 4. Server: Generate the advice "You have a sore throat and headache, so drink plenty of fluids and get plenty of rest."

[0056] 5. Device: Advice is displayed visually on the screen and also provided as an audio notification.

[0057] Example 2: Acute abdominal pain

[0058] 1. User: A user who suddenly experiences severe abdominal pain in the middle of the night puts on the AI ​​glasses.

[0059] 2. Device: Capture your facial color with the camera and record "I suddenly have a stomachache" through voice input.

[0060] 3. Server: Analyzes the data and determines that the patient has acute, severe abdominal pain and is in a high state of urgency.

[0061] 4. Server: Generate the advice "The patient is experiencing severe abdominal pain, please call an ambulance immediately."

[0062] 5. Device: Advice is displayed on the screen and also notified by voice.

[0063] In this way, the present invention is a system that provides prompt and appropriate medical care even in remote locations or at night by linking terminals and servers, which is expected to improve users' self-management abilities, reduce unnecessary medical expenses, and ease the burden on medical facilities.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] The user puts on the AI ​​glasses and turns them on. The AI ​​glasses are equipped with a camera and multiple sensors, which then become ready.

[0067] Step 2:

[0068] The device uses cameras and sensors to capture real-time health data such as the user's complexion, facial expressions, body temperature, and heart rate, while also recording current symptoms and sensations via voice input.

[0069] Step 3:

[0070] The device encrypts the image and audio data it captures. Encryption is for privacy reasons and to prevent unauthorized access to the data.

[0071] Step 4:

[0072] The device sends the encrypted data to the server, which transmits it using a secure communication protocol.

[0073] Step 5:

[0074] The server decrypts the received encrypted data and converts it into a data format for analysis.

[0075] Step 6:

[0076] The server analyzes the image data using computer vision technology, which evaluates the user's complexion, facial expressions, and physical features.

[0077] Step 7:

[0078] The server analyzes the voice data using natural language processing (NLP) algorithms, which extracts the user's complaints and subjective symptoms as text data.

[0079] Step 8:

[0080] The server references past health data and medical history and compares it with current data to more accurately assess the user's current health status.

[0081] Step 9:

[0082] The server uses a generative artificial intelligence model to analyze the data and determine the condition.

[0083] Step 10:

[0084] Based on the analysis results, the server generates appropriate advice and first-line treatment instructions to provide to the user, such as basic home care advice and instructions for more urgent situations.

[0085] Step 11:

[0086] The server sends generated advice and instructions to the terminal.

[0087] Step 12:

[0088] The advice and instructions received by the terminal are visually displayed on the display and also notified to the user audibly.

[0089] Step 13:

[0090] The user follows the displayed advice and instructions and takes the necessary steps or actions, such as drinking more fluids or taking emergency care.

[0091] Step 14:

[0092] The server records the analysis results, generated advice, and the user's reactions in a database, which will be used for future analysis and health management.

[0093] Step 15:

[0094] If the server obtains the user's consent, it will send the analysis results and important health information to the affiliated medical institution.

[0095] Step 16:

[0096] When users provide feedback after using the system, the server receives that information and uses it to improve the accuracy of the generative artificial intelligence model.

[0097] This series of steps will create a system that, through the collaboration between the AI ​​glasses and a server, will analyze the user's health data in real time and immediately provide appropriate advice and instructions for primary treatment.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] Health management has become increasingly important in recent years, and there is a particular need for systems that can monitor individual health conditions in real time. However, current health management systems have difficulty collecting and analyzing data in real time, and providing appropriate advice and treatment. Furthermore, they lack a means of linking with medical institutions and other information processing systems while adequately protecting the privacy of collected data. This poses a challenge, preventing users from receiving prompt and appropriate medical care.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes terminal means equipped with a camera and multiple sensors and capturing the user's health data in real time, means for encrypting the captured image and audio data and transmitting them to the server, server means for decrypting the received encrypted data, analyzing the data using a generative artificial intelligence model, and determining the medical condition, means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, means for transmitting the generated advice and instructions to the terminal and notifying the user visually and audibly, means for using the recorded data to link with other information processing systems, means for referencing past health data and medical history and comparing them with current health data, means for recording the analysis results and advice content in a recording device, and means for continuously training the generative artificial intelligence model to improve accuracy based on user feedback and diagnostic results from other information processing systems. This allows the user's health condition to be understood in real time, enabling prompt and appropriate medical treatment.

[0103] The "terminal means" is a device worn by a user and equipped with a camera and multiple sensors to capture health data in real time.

[0104] A "data encryption means" is a device that has the function of encrypting captured image and audio data to protect the user's privacy.

[0105] The "server means" is a system that has the function of receiving and decrypting encrypted data, and analyzes the data using a generative artificial intelligence model to determine the condition of the disease.

[0106] A "generative artificial intelligence model" is an artificial intelligence algorithm used to analyze captured health and audio data and generate appropriate advice and primary treatment instructions for the user.

[0107] The "advice generation means" is a function that automatically generates appropriate advice and instructions for primary treatment for the user based on the analysis results.

[0108] The "notification means" is a device that transmits the generated advice and instructions for primary action to the terminal and notifies the user visually and audibly.

[0109] The "linking means" is a function for linking the recorded health data of the user with other information processing systems.

[0110] "Comparison means" refers to the function of referencing past health data and medical history and comparing it with current health data.

[0111] The "recording device" is a memory device or database for storing and managing the analysis results and the generated advice content.

[0112] The "training means" is a function that continuously trains the generative artificial intelligence model based on feedback from users and diagnostic results from other information processing systems, thereby improving its accuracy.

[0113] This invention is a system that captures a user's health data in real time, analyzes it using a generative artificial intelligence model, and provides appropriate advice and primary treatment instructions based on the results. To implement this system, the following major components are required:

[0114] 1. Device and System Configuration

[0115] The wearable device (AI glasses) worn by the user is equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.). This device captures the user's health data such as complexion, facial expression, body temperature, and heart rate in real time, and records the user's subjective symptoms using a voice input function.

[0116] The device organizes the captured image and audio data, encrypts the data using an advanced encryption algorithm (e.g., AES 256), and transmits it over the Internet to a server.

[0117] The server decrypts the received encrypted data and inputs it into a generative artificial intelligence model. Analysis is performed using tools such as OpenCV (image processing) and Google® Cloud Speech-to-Text (voice analysis). The generative artificial intelligence model analyzes the captured data and determines the patient's condition based on factors such as facial color, facial expression, body temperature, and heart rate.

[0118] Based on the analysis results, the generative AI model generates appropriate advice and instructions for first-line treatment for the user. For example, if cold symptoms are observed, the model generates advice such as "Drink plenty of fluids and rest," and if symptoms require urgent treatment, the model generates instructions such as "Call an ambulance immediately."

[0119] The server sends the generated advice and instructions to the terminal, which then notifies the user visually or audibly, allowing the user to take the necessary steps or actions based on the information.

[0120] Specific examples

[0121] Example 1: Early symptoms of a cold

[0122] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[0123] 2. Device: Captures the user's complexion and body temperature, and records voice input such as "I have a sore throat" or "I have a headache."

[0124] 3. Terminal: Encrypts the captured data and sends it to the server.

[0125] 4. Server: Decrypts the received data and inputs it into the generative artificial intelligence model.

[0126] 5. Server: The AI ​​model identifies the early symptoms of a cold and generates advice such as "drink plenty of fluids and get plenty of rest."

[0127] 6. Device: Advice is displayed visually and also given as an audio notification.

[0128] Example prompt sentence:

[0129] "The user's complexion is pale and their body temperature is 38.0 degrees. Their voice input has been recorded as 'sore throat' and 'headache'. Based on this information, please analyze their current health condition and generate appropriate advice."

[0130] Example 2: Acute abdominal pain

[0131] 1. User: A user who suddenly experiences severe abdominal pain in the middle of the night puts on the AI ​​glasses.

[0132] 2. Device: Capture facial color with the camera and record voice input such as "I suddenly have a stomachache."

[0133] 3. Terminal: Encrypts the captured data and sends it to the server.

[0134] 4. Server: Decrypts the data and feeds it into the generative artificial intelligence model.

[0135] 5. Server: The AI ​​model determines that the patient has acute, severe abdominal pain and is highly urgent, and generates instructions such as "Call an ambulance immediately."

[0136] 6. Terminal: Provides visual and audio instructions.

[0137] Example prompt sentence:

[0138] "We have audio data that indicates the user looks pale and has sudden, severe stomach pain. Based on this information, please analyze their health condition and generate urgent instructions."

[0139] This system is expected to improve users' self-management capabilities and reduce the burden on medical facilities by monitoring users' health conditions in real time and quickly providing appropriate advice and instructions for primary treatment.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1:

[0142] The user puts on the AI ​​glasses and turns them on. The device's camera and multiple sensors capture the user's health data in real time. Input data includes the user's complexion, facial expression, body temperature, heart rate, etc. Specifically, the camera captures the user's complexion and facial expression, the temperature sensor measures body temperature, and the heart rate sensor measures heart rate. The output data is the captured image data and sensor data.

[0143] Step 2:

[0144] The terminal encrypts the captured image data and audio data. The input data is the image data and audio data captured in step 1. Specifically, the terminal encrypts the data using an encryption algorithm such as AES 256. The encrypted data becomes the output data.

[0145] Step 3:

[0146] The terminal sends the encrypted data to the server. The input data is the data encrypted in step 2. In concrete terms, the terminal sends data to the server via the Internet. The encrypted data received by the server becomes the output data.

[0147] Step 4:

[0148] The server receives and decrypts encrypted data. The input data is the encrypted data sent from the terminal. Specifically, the server decrypts the encrypted data and obtains the original image data and audio data. The decrypted data becomes the output data.

[0149] Step 5:

[0150] The server inputs the decoded data into a generative artificial intelligence model for analysis. The input data consists of decoded image data and audio data. Specifically, the server uses an image processing tool (e.g., OpenCV) to analyze facial color and facial expressions, and uses an audio analysis tool (e.g., Google Cloud Speech-to-Text) to convert the audio data into natural language. The generative artificial intelligence model uses this data to determine the patient's condition and outputs the analysis results for generating advice.

[0151] Step 6:

[0152] Based on the analysis results, the generative AI model generates appropriate advice and instructions for primary treatment. The input data is the analysis results obtained in step 5. Specifically, the model refers to past health data and medical history and compares it with current data. For example, if cold symptoms are observed, the model generates advice such as "drink plenty of fluids and take plenty of rest." The generated advice becomes the output data.

[0153] Step 7:

[0154] The server sends the generated advice and instructions to the terminal. The input data is the advice and instructions generated in step 6. In concrete terms, the server sends the generated advice and instructions to the terminal, and the terminal receives them. The data sent to the terminal becomes the output data.

[0155] Step 8:

[0156] The device notifies the user of the received advice and instructions visually or audibly. The input data is the advice and instructions received from the server. Specifically, the device's transparent display or bone conduction speaker is used to notify the user of the advice and instructions for primary treatment visually and audibly. The user can receive the notification and take the necessary action.

[0157] (Application example 1)

[0158] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] There is a need for a system that can monitor users' health status in real time and provide not only appropriate advice and primary treatment instructions, but also optimal diet and nutritional support. However, conventional systems have difficulty meeting these diverse needs, and one issue is the lack of specific advice on diet and nutrition.

[0160] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0161] In this invention, the server includes a terminal means equipped with a camera and a sensor for capturing health data in real time, a means for encrypting the captured image data and audio data and transmitting it to the server, a server means for analyzing the received data using a generative artificial intelligence model to determine the condition of the patient, a means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, a means for generating optimal diet and nutritional support for the user based on the analysis results, a means for transmitting the generated advice and instructions to the terminal and notifying the user, and a means for using the recorded data to connect with medical institutions. This makes it possible to comprehensively manage the user's health condition and quickly provide appropriate advice and nutritional support.

[0162] The "terminal means" is a wearable device worn by a user, and is a device equipped with a camera and sensors for capturing health data in real time.

[0163] The "means for encrypting captured image data and audio data" is a technology for encrypting image data and audio data acquired from a terminal to protect privacy and transmitting the data safely to a server.

[0164] The "server means" is a device that analyzes the received encrypted data, determines the condition using a generative artificial intelligence model, and generates advice and instructions for primary treatment for the user based on the analysis results.

[0165] A "generative artificial intelligence model" is an advanced AI algorithm that analyzes captured health data, determines medical conditions, and generates advice.

[0166] The "means for generating advice and instructions for primary treatment" is a technology that generates appropriate advice and instructions for primary treatment for the user based on the results of analysis using a generative artificial intelligence model.

[0167] The "means for generating meals and nutritional support" is a technology that generates meals and nutritional support that are optimal for the user based on the analysis results.

[0168] The "means for notifying advice and instructions" is a technique for transmitting the generated advice and instructions to the terminal and notifying the user visually or audibly.

[0169] "Means for collaboration with medical institutions" refers to technology for sharing recorded data with medical institutions and collaborating with them.

[0170] A "database" is a data storage device for recording and storing analysis results and advice content.

[0171] "Feedback from users and diagnostic results from medical institutions" refers to information provided by users and medical institutions that contributes to the continuous training and improvement of generative artificial intelligence models.

[0172] The present invention is a system that monitors the user's health condition in real time and provides optimal advice and instructions for primary treatment, as well as appropriate diet and nutritional support. Hereinafter, specific embodiments of the present invention will be described.

[0173] System Configuration

[0174] 1. Terminal means

[0175] The wearable device is equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.). The device captures real-time health data and collects image and audio data. Using the voice input function, users can record their current symptoms and feelings by voice.

[0176] 2. Data Encryption and Transmission

[0177] The device organizes the captured image and audio data in real time and encrypts them for privacy reasons (using a data encryption library such as OpenSSL), then sends the encrypted data to a cloud server.

[0178] 3. Server Means

[0179] The cloud server decrypts the received encrypted data and inputs it into a generative artificial intelligence model (e.g., GPT-4 (registered trademark)). The model analyzes facial color, facial expressions, and physical characteristics from the image data, and extracts the user's complaints and subjective symptoms from the voice data using natural language processing. It compares this with current data while also referencing past health data and medical history to determine the condition of the patient.

[0180] 4. Advice Generation

[0181] Based on the analysis results, the generative AI model generates advice and instructions for first-line treatment for the user. Furthermore, based on the analysis results, it also generates optimal dietary and nutritional support for the user. For example, if the user shows early symptoms of a cold, it generates advice such as "It would be good to eat a diet rich in vitamin C."

[0182] 5. Notice and Enforcement

[0183] The generated advice and instructions are sent from the server to the device. The device displays them visually and also notifies the user by voice. The user can follow the displayed advice and instructions and take the necessary measures or actions. The corresponding ingredients are also displayed preferentially on the food delivery selection screen.

[0184] 6. Collaboration with medical institutions

[0185] The recorded data will be used to connect with medical institutions as needed, allowing medical institutions to refer to the user's health data and provide appropriate diagnosis and treatment.

[0186] Specific examples

[0187] Example 1: Suggested diet for preventing colds

[0188] User: A user who feels a slight cold puts on the smart glasses.

[0189] Device: Captures heart rate and facial color, and records voice messages such as "My throat has been a bit sore lately."

[0190] Server: Analyzes the captured data and determines whether it is an early symptom of a cold.

[0191] Server: The advice is to "consider a diet that includes citrus fruits and vegetables that are rich in vitamin C."

[0192] Device: Advice is displayed and voice notification is given. The relevant ingredients are also displayed preferentially on the food delivery selection screen.

[0193] Prompt Sentence Examples

[0194] "My throat has been a bit sore lately. What foods would be good to prevent a cold?"

[0195] In this way, the system of the present invention comprehensively manages the user's health condition and quickly provides appropriate advice and nutritional support.

[0196] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0197] Step 1:

[0198] The user puts on the smart glasses (terminal means) and activates the device. Data is collected as input from cameras, temperature sensors, heart rate sensors, etc. to capture the user's health status in real time. The terminal acquires biometric information (complexion, heart rate, body temperature, etc.) from these sensors and voice input, and organizes it as primary data.

[0199] Step 2:

[0200] Encrypt the organized primary data. The device uses a data encryption library (e.g., OpenSSL) to encrypt the collected health data for privacy purposes. It receives the primary data as input and generates encrypted data as output.

[0201] Step 3:

[0202] The encrypted data is sent to the cloud server. The device sends the encrypted data to the server via the Internet. The device receives the encrypted data as input and sends the data to the cloud server as output.

[0203] Step 4:

[0204] The cloud server decrypts the received encrypted data. When the server receives the data, it uses a decryption library to decrypt it. It takes the encrypted data as input and gets the decrypted data as output.

[0205] Step 5:

[0206] The decoded data is input into a generative AI model for analysis. The server uses a generative AI model (e.g., GPT-4) to receive the decoded data as input, analyze facial color, facial expressions, and physical features from the image data, and extract the user's complaints and subjective symptoms from the voice data using natural language processing. The output is a diagnosis of the disease condition and the user's current health condition.

[0207] Step 6:

[0208] Based on the analysis results, the server generates advice, initial treatment instructions, and optimal diet and nutritional support recommendations. Utilizing a generative AI model, the server receives the condition assessment results as input and generates advice, diet, and nutritional support as output. For example, if foods rich in vitamin C are needed, the server creates a specific recommendation such as, "A diet containing foods rich in vitamin C would be good."

[0209] Step 7:

[0210] The generated advice and instructions are sent to the terminal. The server sends the generated advice and instructions to the terminal via the Internet. It receives advice and instructions as input and sends them to the terminal as output.

[0211] Step 8:

[0212] The device notifies the user of the received advice or instructions. The device displays the advice or instructions on the screen or notifies the user by voice. The device receives advice or instructions from the server as input and notifies the user visually or audibly as output. For example, the device may display advice such as "It would be good to eat meals that include ingredients rich in vitamin C," and prioritize the corresponding ingredients on the food delivery selection screen.

[0213] Step 9:

[0214] The user acts according to the advice and instructions received. The user selects the necessary diet and nutritional support based on the displayed advice examples and takes action accordingly. Specifically, the user performs an action such as ordering a meal using the recommended ingredients.

[0215] Through this series of steps, the user is provided with real-time advice and dietary suggestions based on their health status.

[0216] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0217] This invention combines a system that captures a user's health data in real time and provides advice and primary treatment instructions based on the analysis results with an emotion engine that recognizes the user's emotions. To implement this system, the following main components are required:

[0218] 1. Device and System Configuration

[0219] Device: AI glasses are a wearable device worn by the user. Equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.), they capture emotions along with health data. They also have the ability to collect image and audio data, encrypt it, and send it to a server.

[0220] Server: Receives the captured data and analyzes the medical condition using a generative artificial intelligence model and emotion engine. Based on the analysis results, it generates advice and primary treatment instructions and sends them to the device.

[0221] Generative artificial intelligence model: An advanced AI algorithm that analyzes health data, determines medical conditions, and generates advice.

[0222] Emotion engine: An AI algorithm that recognizes emotions from the user's facial expressions and voice data and integrates that data into the analysis results.

[0223] 2. System operation explanation

[0224] The system operates as follows.

[0225] Data collection: The user puts on the AI ​​glasses and turns them on. The device's camera and sensors capture real-time health data, such as the user's complexion, facial expressions, body temperature, and heart rate. At the same time, the voice input function records the user's current symptoms and sensations, and the camera records the user's facial expressions.

[0226] Data encryption and transmission: The device organizes the captured image data, voice data, and facial expression data in real time, encrypts them for privacy purposes, and then transmits the data to the server.

[0227] Data analysis: The server decrypts the received encrypted data and converts it into a data format for analysis. Image data is analyzed using computer vision technology to evaluate facial color, facial expressions, and physical characteristics. Voice data is analyzed using natural language processing (NLP) algorithms to extract the user's complaints and subjective symptoms as text data.

[0228] Emotion Recognition: The server uses an emotion engine to analyze the user's emotions from facial expressions and voice, for example, to determine whether the user is stressed or relaxed.

[0229] Data integration and analysis result generation: The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the condition.

[0230] Advice generation: Based on the analysis results, advice and instructions for first-line treatment are generated for the user. Emotional data is also taken into consideration at this time. For example, if the user is feeling stressed, advice on how to relax is added.

[0231] Notification and implementation: The server sends the generated advice and instructions to the terminal, which notifies the user visually or audibly. The user follows the displayed advice and instructions and takes the necessary steps or actions.

[0232] Specific examples

[0233] Example 1: Early symptoms of a cold

[0234] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[0235] 2. Device: Captures the user's complexion, body temperature, and facial expressions, and records symptoms such as "sore throat" or "headache" via voice input.

[0236] 3. Server: Analyzes the captured data and determines whether the user is showing early symptoms of a cold. At the same time, it recognizes that the user's facial expression indicates fatigue.

[0237] 4. Server: Generate the following advice: "You have a sore throat and headache, so drink plenty of fluids and get plenty of rest. You also feel fatigued, so it's recommended that you get plenty of rest."

[0238] 5. Device: Advice is displayed visually on the screen and also provided as an audio notification.

[0239] Example 2: Acute abdominal pain and stress

[0240] 1. User: Feels severe abdominal pain in the middle of the night and puts on the AI ​​glasses.

[0241] 2. Device: The camera captures the user's facial expression and facial color, and the user is given a voice command such as "I suddenly have a stomachache." The device also determines whether the user is feeling stressed based on the user's facial expression.

[0242] 3. Server: Analyzes the data, detects acute severe abdominal pain and high stress levels, and determines that the situation is urgent.

[0243] 4. Server: Generate the following advice: "Severe abdominal pain has been detected, so please call an ambulance immediately. High stress levels have also been detected, so please adopt relaxation techniques once safety is assured."

[0244] 5. Device: Advice is displayed on the screen and also notified by voice.

[0245] In this way, the present invention, which combines an emotion engine, is a system that provides more accurate and personalized medical support while taking into account the user's emotional state. This is expected to improve users' self-management abilities, reduce unnecessary medical expenses, and ease the burden on medical facilities.

[0246] The processing flow will be explained below.

[0247] Step 1:

[0248] The user puts on the AI ​​glasses and turns them on. The AI ​​glasses are equipped with a camera and multiple sensors, which then become ready.

[0249] Step 2:

[0250] The device uses cameras and sensors to capture real-time health data such as the user's complexion, facial expressions, body temperature, and heart rate, while also recording current symptoms and sensations via voice input.

[0251] Step 3:

[0252] The device encrypts the image, voice, and facial expression data it captures to protect privacy and prevent unauthorized access to the data.

[0253] Step 4:

[0254] The device sends the encrypted data to the server, which transmits it using a secure communication protocol.

[0255] Step 5:

[0256] The server decrypts the received encrypted data and converts it into a data format for analysis.

[0257] Step 6:

[0258] The server analyzes the image data using computer vision technology, which evaluates the user's complexion, facial expressions, and physical features.

[0259] Step 7:

[0260] The server analyzes the voice data using natural language processing (NLP) algorithms, which extracts the user's complaints and subjective symptoms as text data.

[0261] Step 8:

[0262] The server uses an emotion engine to analyze the user's emotions from facial expressions and voice. Emotion recognition determines whether the user is stressed, relaxed, or in other emotional states.

[0263] Step 9:

[0264] The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the condition.

[0265] Step 10:

[0266] Based on the analysis results, the server generates appropriate advice and instructions for first-line treatment to provide to the user. Emotional data is also taken into account in this process. For example, if the user is feeling stressed, advice on how to relax will be added.

[0267] Step 11:

[0268] The server sends generated advice and instructions to the terminal.

[0269] Step 12:

[0270] The advice and instructions received by the terminal are visually displayed on the display and also notified to the user audibly.

[0271] Step 13:

[0272] The user follows the displayed advice and instructions and takes the necessary steps or actions, such as drinking more fluids or taking emergency care.

[0273] Step 14:

[0274] The server records the analysis results, generated advice, and the user's reactions in a database, which will be used for future analysis and health management.

[0275] Step 15:

[0276] If the server obtains the user's consent, it will send the analysis results and important health information to the affiliated medical institution.

[0277] Step 16:

[0278] When users provide feedback after using the system, the server receives that information and uses it to improve the accuracy of the generative artificial intelligence model.

[0279] This series of steps will create a system that, through the collaboration between the AI ​​glasses and a server, will analyze the user's health and emotional data in real time and immediately provide appropriate advice and instructions for primary treatment.

[0280] Example 2

[0281] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0282] Conventional health management systems lack analysis and advice that take into account the user's emotional state, making it difficult to accurately assess health status and provide appropriate treatment. They also lack real-time data capture and analysis, making it difficult to respond quickly to user conditions. Furthermore, they lack the ability to compare health data with past data or continuously train generative artificial intelligence algorithms, making it difficult to improve the system's accuracy.

[0283] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information terminal equipped with an image capture device and a biometric sensor for capturing health data and emotional data in real time, an information terminal that encrypts the captured image data and audio data and transmits them to a central processing unit, a central processing unit that analyzes the received data using a generative artificial intelligence algorithm to determine the user's health status, a central processing unit that generates appropriate advice and instructions for primary treatment for the user based on the analysis results, an information terminal that transmits the generated advice and instructions to the information terminal and notifies the user, and a data management device used to link the recorded data with a medical institution. This enables accurate health assessments that take the user's emotional state into consideration, provision of appropriate advice, and prompt real-time responses. Furthermore, the accuracy of the system can be improved by comparing with past health data and continuously training the generative artificial intelligence algorithm.

[0284] "Image capture device" refers to a device such as a camera for capturing a user's facial expression.

[0285] A "biometric sensor" refers to a sensor used to measure a user's health data, such as body temperature or heart rate.

[0286] "Information terminal" refers to a device that captures health and emotional data, encrypts it, and transmits it to a central processing unit.

[0287] "Central Processing Unit" refers to the electronic device that analyzes the received data, determines the health status, and generates appropriate advice and instructions for primary treatment.

[0288] "Generative artificial intelligence algorithm" refers to an advanced computational method that analyzes health and emotional data to determine medical conditions and generate advice.

[0289] "Data management device" refers to the system used to store recorded data and to communicate with medical institutions as needed.

[0290] "Real-time" refers to a processing format in which data is captured immediately and analyzed and notified.

[0291] "Encryption" refers to the technology of converting data to protect the privacy of that information, making it unreadable to third parties.

[0292] "Emotional data" refers to information about the emotional state obtained from the user's facial expressions and voice.

[0293] "Health status" refers to the user's physical and psychological state as assessed by body temperature, heart rate, facial expression, etc.

[0294] "Advice" refers to instructions or recommendations provided to the user based on the analysis results.

[0295] "First steps" refers to instructions that indicate the initial response that a user should take in an emergency.

[0296] This invention is a system that captures and analyzes a user's health and emotional state in real time and provides advice and primary treatment instructions based on the analysis results. This system consists of an information terminal worn by the user, a central processing unit that analyzes the data, a means for providing the generated advice, and a device that manages the recorded data and connects with medical institutions.

[0297] Hardware and software used

[0298] Information terminal: A wearable device worn by the user, equipped with a camera (image capture device) and multiple biometric sensors (temperature sensor, heart rate sensor, etc.). This terminal captures health and emotional data in real time, encrypts the data, and transmits it to a central processing unit.

[0299] Central Processing Unit: A server on the cloud or a local high-performance computing device that analyzes data using generative artificial intelligence algorithms and sentiment analysis engines.

[0300] Data management device: A cloud storage or on-premise database system that records and stores analysis results, advice, past health data, and medical history data.

[0301] Data processing and calculation

[0302] Once worn by the user, the device begins collecting data using its camera and sensors. The camera captures the user's facial expression and complexion, the temperature sensor measures body temperature, the heart rate sensor measures heart rate, and the voice input function records the user's description of symptoms. This collected data is encrypted in real time and sent to a server.

[0303] The server decrypts the received encrypted data and converts it into the appropriate data format. Image data is analyzed using computer vision technology to evaluate the user's facial color and facial expressions. Voice data is analyzed using natural language processing (NLP) algorithms to extract the user's complaints and subjective symptoms as text data. Furthermore, an emotion analysis engine analyzes the user's emotional state from facial expressions and voice.

[0304] The generative AI algorithm analyzes the integrated health and emotional data to assess the user's health status. Based on this assessment, advice and primary treatment instructions are generated. The accuracy of the analysis results is improved by comparing them with past health data and medical history to ensure the appropriateness of advice for the user.

[0305] The generated advice and instructions are then encrypted and sent to the terminal, where the user can receive the information visually or audibly.

[0306] Specific examples

[0307] Example 1: Early symptoms of a cold

[0308] 1. The user feels unwell in the morning and puts on the AI ​​glasses.

[0309] 2. The device captures the user's complexion, body temperature, and facial expressions, and records symptoms such as "sore throat" and "headache" through voice input.

[0310] 3. The server analyzes the captured data and determines whether the user is showing early symptoms of a cold. At the same time, the server recognizes the user's fatigue from their facial expression.

[0311] 4. The server generates the advice, "You have a sore throat and a headache, so drink plenty of fluids and rest. You also feel fatigued, so it's recommended that you get plenty of rest."

[0312] 5. The device will display advice on the screen and also provide audio notifications.

[0313] Example 2: Acute abdominal pain and stress

[0314] 1. The user experiences severe stomach pain in the middle of the night and puts on the AI ​​glasses.

[0315] 2. The device captures facial expression and facial color with a camera and records the user's voice input, such as "I suddenly have a stomachache." It also determines whether the user is feeling stressed based on their facial expression.

[0316] 3. The server analyzes the data, detects acute severe abdominal pain and high stress levels, and determines that the condition is urgent.

[0317] 4. The server generates the advice, "You are experiencing severe abdominal pain, so please call an ambulance immediately. Also, high stress levels have been detected, so please adopt relaxation techniques once safety is assured."

[0318] 5. The device will display advice on the screen and also provide audio notifications.

[0319] Prompt Sentence Examples

[0320] "My temperature is rising, my throat is sore, and I also have a slight headache this morning."

[0321] "I suddenly get a very strong stomach ache."

[0322] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0323] Step 1: Booting the device

[0324] The user puts on the AI ​​glasses and turns on the device. Once the device is successfully booted and ready, the camera and sensors are initialized and ready to collect data. The input is the user's actions, and the output is the device's ready state.

[0325] Specific example of operation: When the user presses the power button on the AI ​​Glasses, the device starts up with a startup sound and displays a notification that it is ready.

[0326] Step 2: Data collection

[0327] The device activates the camera and sensors to capture facial color, facial expressions, body temperature, and heart rate. The voice input function is enabled to record what the user says. The input is the user's health status and voice, and the output is these data.

[0328] Specific operation example: The camera captures the user's face, the temperature sensor measures body temperature, and the heart rate sensor measures heart rate. When the user says something like "I have a headache," the voice is recorded.

[0329] Step 3: Encrypt and send data

[0330] The device encrypts the captured image, audio, and biometric data using the AES-256 encryption algorithm. The encrypted data is sent to the server. The input is raw data, and the output is encrypted data.

[0331] Specific operation example: Collected data (e.g., facial color images, body temperature, heart rate, and voice data) is encrypted and sent to a server via high-speed wireless communication.

[0332] Step 4: Data analysis

[0333] The server decrypts the received encrypted data and converts it into the appropriate data format. Image data is analyzed using computer vision technology to evaluate the user's facial color and facial expression. Voice data is analyzed using natural language processing (NLP) technology to extract the user's complaints and subjective symptoms as text data. The input is encrypted data, and the output is the analysis results.

[0334] Specific example: Analyze image data using OpenCV and extract text from audio data using the Google Cloud Speech-to-Text API.

[0335] Step 5: Emotion Recognition

[0336] The server uses an emotion engine to analyze the user's emotions from facial expressions and voice, for example, to determine stress levels and relaxation states. The input is the analyzed health data and voice data, and the output is an assessment of the user's emotional state.

[0337] Specific example of operation: Using the Emotion AI API to analyze facial expression data and evaluate it as "high stress level."

[0338] Step 6: Integrate and score the data

[0339] The server integrates the acquired health data, emotional data, past health data, and medical history, and uses a generative artificial intelligence algorithm to assess health status. Each data point is assigned a score to determine overall health status. The input is the integrated data, and the output is an overall health score.

[0340] Specific example of operation: Using TENSORFLOW (registered trademark) to score health status using a generative AI model.

[0341] Step 7: Advice Generation

[0342] The server generates advice and instructions for first-line treatment for the user based on the analysis results. It also takes into account emotional data and adds appropriate recommendations. The input is the analysis results of health data and emotional data, and the output is advice and instructions for first-line treatment.

[0343] Example of specific operation: Advice such as "Drink plenty of fluids and get plenty of rest" is generated in text format.

[0344] Step 8: Notification and enforcement

[0345] The server encrypts the generated advice and instructions and sends them to the terminal. The terminal notifies the user of this information visually or audibly. The input is the encrypted advice data, and the output is the notification to the user.

[0346] Specific example of operation: The AI ​​glasses display will show "Drink plenty of fluids and get plenty of rest," and a voice notification will also be given.

[0347] This detailed processing step allows the user to receive an accurate assessment of their health status in real time and receive appropriate advice and primary treatment instructions.

[0348] (Application example 2)

[0349] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0350] Current security services lack a means to monitor the health and emotional state of security guards in real time, making it difficult to respond quickly when abnormalities in the guard's health or mental state occur. In addition, systems for accurately detecting suspicious individuals and abnormal behavior are still insufficient. This places a heavy burden on security guards and risks reducing overall safety. To address these issues, the present invention aims to provide a comprehensive system that captures health and emotional data in real time and provides appropriate advice and first-line treatment instructions.

[0351] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means equipped with a camera and sensor for capturing health data and emotional data in real time, means for encrypting the captured image data and audio data and transmitting them to the server, server means for analyzing the received data using a generative artificial intelligence model and an emotion engine to determine the medical condition and emotional state, means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, means for transmitting the generated advice and instructions to the terminal and notifying the user, means for using the recorded data to apply to security management, and means for detecting anomalies and taking emergency measures. This makes it possible to monitor the health and emotional state of security guards in real time and to take appropriate action immediately if an abnormality occurs.

[0352] "Terminal means" refers to a device equipped with a camera and sensors for capturing health and emotional data in real time.

[0353] "Encryption means" refers to the technology and method for encrypting captured image and audio data for secure transmission.

[0354] "Server means" is a computer system that analyzes the received data, determines the medical and emotional state, and generates necessary advice and primary treatment instructions.

[0355] A "generative artificial intelligence model" is a machine learning algorithm that analyzes health and emotional data to generate appropriate advice and first-line treatment instructions.

[0356] An "emotion engine" is an algorithm and technology for analyzing a user's facial expressions and voice data to determine their emotional state.

[0357] The "notification means" is a method for transmitting the generated advice and instructions to the terminal and notifying the user visually or audibly.

[0358] "Security management measures" are techniques and methods for utilizing recorded data for security management.

[0359] "Abnormality detection means" refers to techniques and methods for detecting abnormalities in the health or emotional state of security guards.

[0360] "Emergency response measures" are techniques and methods for taking immediate and appropriate action when an abnormality is detected.

[0361] This invention is a system that captures health and emotional data of security guards in real time and provides appropriate advice and instructions for primary treatment based on the analysis results. This system is composed of terminal means, encryption means, server means, generative artificial intelligence model, emotion engine, notification means, security management means, anomaly detection means, and emergency response means.

[0362] 1. Hardware and Software Used

[0363] Terminal means

[0364] A wearable device worn by a user, equipped with a camera and sensors (such as a temperature sensor and a heart rate sensor) to capture health and emotional data. Examples include smart glasses and head-mounted displays.

[0365] Encryption method

[0366] Used to encrypt the captured data. The encryption technology used is a cryptographic library such as OpenSSL.

[0367] Server Means

[0368] It is a computer system that analyzes the received data and determines the patient's medical condition and emotional state. It mainly uses server-side frameworks such as Flask or Django as its software.

[0369] Generative AI model and emotion engine

[0370] Health and emotion data are analyzed using deep learning frameworks such as TensorFlow and Keras, as well as Hugging Face's Transformers.

[0371] Notification means

[0372] This is a mechanism for transmitting advice and instructions for primary measures generated by the server to the terminal means and notifying the user visually and audibly.

[0373] Security Control Measures

[0374] It includes techniques and methods for applying recorded data to security management.

[0375] Anomaly detection and emergency response measures

[0376] This includes techniques and methods for detecting abnormalities in the health or emotional state of security guards and for immediately taking appropriate action.

[0377] 2. Data processing and calculation

[0378] Data collection

[0379] The user wears a device, and cameras and sensors capture data in real time. The collected data is then organized in real time and sent to a server using encryption technology.

[0380] Data analysis

[0381] The server decodes the received data and analyzes it using computer vision techniques and natural language processing algorithms, mobilizing generative artificial intelligence models and emotion engines to determine health and emotional states.

[0382] Advice Generation and Notification

[0383] Based on the analysis results, the server generates the necessary advice and instructions for the first action, which are sent to the terminal means and notified to the user visually and audibly.

[0384] 3. Specific Examples

[0385] Example 1: Suspicious person detection

[0386] If a user (security guard) detects a suspicious person while patrolling, the AI ​​glasses capture the scene and send it to the server, which analyzes it and provides advice such as, "A suspicious person has been spotted. Ensure safety and contact the police."

[0387] Example 2: Detecting health abnormalities

[0388] If a security guard experiences chest pain while on patrol, the AI ​​glasses will capture the image and send it to a server, which will then analyze it and provide advice such as, "Chest pain detected. Please rest immediately."

[0389] Prompt Sentence Examples

[0390] Here are some example prompts to input to the generative AI model:

[0391] Suspicious person detection

[0392] Security Guard A detects a suspicious person while on patrol. Please analyze the data captured by the AI ​​Glasses and provide appropriate countermeasures.

[0393] Security guard's health condition

[0394] Security Guard B experiences chest pains while on patrol. Please provide appropriate first aid advice from captured data.

[0395] As described above, by using the system of the present invention, it is possible to monitor the health and emotional state of security guards in real time, and to take appropriate action immediately if an abnormality occurs.

[0396] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0397] Step 1: Data collection

[0398] The user puts on the terminal (wearable device) and turns it on. The camera and sensors on the terminal capture the user's health and emotional data, such as complexion, facial expression, body temperature, and heart rate, in real time. At the same time, the voice input function records the user's current symptoms and sensations. The input is camera footage, audio, and health data from the sensors, and the output is the captured data.

[0399] Step 2: Data Encryption

[0400] The terminal means organizes the captured image data, voice data, and facial expression data and encrypts them for privacy protection. Specifically, the data is encrypted using the OpenSSL library. The input is the captured data, and the output is the encrypted data.

[0401] Step 3: Send data

[0402] Send encrypted data to a server using a secure communication protocol (e.g., HTTPS). The input is the encrypted data, and the output is a notification to the server that the data has been sent.

[0403] Step 4: Receive and decrypt data

[0404] The server decrypts the encrypted data it receives and converts it into a data format for analysis. Specifically, it uses the OpenSSL library to decrypt the data. The input is the encrypted data received by the server, and the output is the decrypted data.

[0405] Step 5: Data analysis

[0406] The server analyzes image data and audio data. Image data is analyzed for facial color and facial expressions using OpenCV, and audio data is converted to text data using the Transformers library. Health data is analyzed using TensorFlow and Keras. The input is the decoded data, and the output is the analysis result.

[0407] Step 6: Emotion Recognition

[0408] The server uses an emotion engine to analyze emotions from facial expressions and voice data, for example, to determine whether the user is stressed or relaxed. The input is the analyzed facial expression and voice data, and the output is the emotion recognition result.

[0409] Step 7: Data integration and analysis

[0410] The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the patient's condition. The input is the analyzed health data and emotional data, and the output is the integrated analysis result.

[0411] Step 8: Advice Generation

[0412] Based on the analysis results, the server generates advice and instructions for the user, taking into account emotional data. For example, if the user feels stressed, it adds advice on how to relax. The input is the integrated analysis results, and the output is the generated advice and instructions.

[0413] Step 9: Advice Notification

[0414] The server sends the generated advice and instructions to the terminal means, which then notifies the user of the advice and instructions visually or audibly. Specifically, the terminal means uses a display or speaker. The input is the generated advice and instructions, and the output is the notification to the user.

[0415] Step 10: Security Management and Emergency Response

[0416] The server applies the recorded data to security management, detects suspicious individuals, and takes emergency action when a security guard has a health abnormality. The input is user feedback and security data after notification, and the output is improvements to security management and the results of emergency responses.

[0417] These are the specific processing steps for the system that realizes this application example. It is possible to monitor the user's health condition and emotional state in real time and take appropriate action immediately if an abnormality occurs.

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0419] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0421] [Second embodiment]

[0422] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0423] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0424] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0426] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0428] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0429] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0430] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0433] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0434] This invention is a system that captures and analyzes a user's health data in real time and provides appropriate advice and primary treatment instructions. To implement this system, the following major components are required:

[0435] 1. Device and System Configuration

[0436] Device: AI glasses are a wearable device worn by the user. Equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.), they capture health data in real time. They also have the ability to collect image and audio data, encrypt it, and send it to a server.

[0437] Server: Receives the captured data and analyzes the medical condition using a generative artificial intelligence model. Based on the analysis results, it generates appropriate advice and instructions for primary treatment and sends them to the device.

[0438] Generative artificial intelligence model: An advanced AI algorithm that analyzes health data, determines medical conditions, and generates advice.

[0439] 2. System operation explanation

[0440] The system operates as follows.

[0441] Data collection: The user puts on the AI ​​glasses and activates them. The device's camera and sensors capture real-time health data, such as the user's complexion, facial expressions, body temperature, and heart rate. Using the voice input function, the user records their current symptoms and feelings.

[0442] Data encryption and transmission: The device organizes the captured image and audio data in real time, encrypts it for privacy purposes, and then transmits the data to the server.

[0443] Data analysis: The server decrypts the received encrypted data and inputs it into a generative AI model. The model analyzes facial color, facial expressions, and physical characteristics from the image data, and uses natural language processing to extract the user's complaints and subjective symptoms from the voice data. Past health data and medical history are also referenced, and the data is compared with current data.

[0444] Advice generation: Based on the analysis results, the generative AI model generates advice and instructions for first-line treatment for the user. For example, if cold symptoms are observed, basic advice such as "drink plenty of fluids and rest" is provided. If more urgent symptoms are observed, emergency response instructions such as "call an ambulance immediately" are issued.

[0445] Notification and implementation: The server sends the generated advice and instructions to the terminal, which notifies the user visually or audibly. The user follows the displayed advice and instructions and takes the necessary steps or actions.

[0446] Specific examples

[0447] Example 1: Early symptoms of a cold

[0448] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[0449] 2. Device: Captures the user's complexion and body temperature, and records symptoms such as "sore throat" or "headache" via voice input.

[0450] 3. Server: Analyzes the captured data and determines whether early symptoms of a cold are present.

[0451] 4. Server: Generate the advice "You have a sore throat and headache, so drink plenty of fluids and get plenty of rest."

[0452] 5. Device: Advice is displayed visually on the screen and also provided as an audio notification.

[0453] Example 2: Acute abdominal pain

[0454] 1. User: A user who suddenly experiences severe abdominal pain in the middle of the night puts on the AI ​​glasses.

[0455] 2. Device: Capture your facial color with the camera and record "I suddenly have a stomachache" through voice input.

[0456] 3. Server: Analyzes the data and determines that the patient has acute, severe abdominal pain and is in a high state of urgency.

[0457] 4. Server: Generate the advice "The patient is experiencing severe abdominal pain, please call an ambulance immediately."

[0458] 5. Device: Advice is displayed on the screen and also notified by voice.

[0459] In this way, the present invention is a system that provides prompt and appropriate medical care even in remote locations or at night by linking terminals and servers, which is expected to improve users' self-management abilities, reduce unnecessary medical expenses, and ease the burden on medical facilities.

[0460] The processing flow will be explained below.

[0461] Step 1:

[0462] The user puts on the AI ​​glasses and turns them on. The AI ​​glasses are equipped with a camera and multiple sensors, which then become ready.

[0463] Step 2:

[0464] The device uses cameras and sensors to capture real-time health data such as the user's complexion, facial expressions, body temperature, and heart rate, while also recording current symptoms and sensations via voice input.

[0465] Step 3:

[0466] The device encrypts the image and audio data it captures. Encryption is for privacy reasons and to prevent unauthorized access to the data.

[0467] Step 4:

[0468] The device sends the encrypted data to the server, which transmits it using a secure communication protocol.

[0469] Step 5:

[0470] The server decrypts the received encrypted data and converts it into a data format for analysis.

[0471] Step 6:

[0472] The server analyzes the image data using computer vision technology, which evaluates the user's complexion, facial expressions, and physical features.

[0473] Step 7:

[0474] The server analyzes the voice data using natural language processing (NLP) algorithms, which extracts the user's complaints and subjective symptoms as text data.

[0475] Step 8:

[0476] The server references past health data and medical history and compares it with current data to more accurately assess the user's current health status.

[0477] Step 9:

[0478] The server uses a generative artificial intelligence model to analyze the data and determine the condition.

[0479] Step 10:

[0480] Based on the analysis results, the server generates appropriate advice and first-line treatment instructions to provide to the user, such as basic home care advice and instructions for more urgent situations.

[0481] Step 11:

[0482] The server sends generated advice and instructions to the terminal.

[0483] Step 12:

[0484] The advice and instructions received by the terminal are visually displayed on the display and also notified to the user audibly.

[0485] Step 13:

[0486] The user follows the displayed advice and instructions and takes the necessary steps or actions, such as drinking more fluids or taking emergency care.

[0487] Step 14:

[0488] The server records the analysis results, generated advice, and the user's reactions in a database, which will be used for future analysis and health management.

[0489] Step 15:

[0490] If the server obtains the user's consent, it will send the analysis results and important health information to the affiliated medical institution.

[0491] Step 16:

[0492] When users provide feedback after using the system, the server receives that information and uses it to improve the accuracy of the generative artificial intelligence model.

[0493] This series of steps will create a system that, through the collaboration between the AI ​​glasses and a server, will analyze the user's health data in real time and immediately provide appropriate advice and instructions for primary treatment.

[0494] Example 1

[0495] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0496] Health management has become increasingly important in recent years, and there is a particular need for systems that can monitor individual health conditions in real time. However, current health management systems have difficulty collecting and analyzing data in real time, and providing appropriate advice and treatment. Furthermore, they lack a means of linking with medical institutions and other information processing systems while adequately protecting the privacy of collected data. This poses a challenge, preventing users from receiving prompt and appropriate medical care.

[0497] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0498] In this invention, the server includes terminal means equipped with a camera and multiple sensors and capturing the user's health data in real time, means for encrypting the captured image and audio data and transmitting them to the server, server means for decrypting the received encrypted data, analyzing the data using a generative artificial intelligence model, and determining the medical condition, means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, means for transmitting the generated advice and instructions to the terminal and notifying the user visually and audibly, means for using the recorded data to link with other information processing systems, means for referencing past health data and medical history and comparing them with current health data, means for recording the analysis results and advice content in a recording device, and means for continuously training the generative artificial intelligence model to improve accuracy based on user feedback and diagnostic results from other information processing systems. This allows the user's health condition to be understood in real time, enabling prompt and appropriate medical treatment.

[0499] The "terminal means" is a device worn by a user and equipped with a camera and multiple sensors to capture health data in real time.

[0500] A "data encryption means" is a device that has the function of encrypting captured image and audio data to protect the user's privacy.

[0501] The "server means" is a system that has the function of receiving and decrypting encrypted data, and analyzes the data using a generative artificial intelligence model to determine the condition of the disease.

[0502] A "generative artificial intelligence model" is an artificial intelligence algorithm used to analyze captured health and audio data and generate appropriate advice and primary treatment instructions for the user.

[0503] The "advice generation means" is a function that automatically generates appropriate advice and instructions for primary treatment for the user based on the analysis results.

[0504] The "notification means" is a device that transmits the generated advice and instructions for primary action to the terminal and notifies the user visually and audibly.

[0505] The "linking means" is a function for linking the recorded health data of the user with other information processing systems.

[0506] "Comparison means" refers to the function of referencing past health data and medical history and comparing it with current health data.

[0507] The "recording device" is a memory device or database for storing and managing the analysis results and the generated advice content.

[0508] The "training means" is a function that continuously trains the generative artificial intelligence model based on feedback from users and diagnostic results from other information processing systems, thereby improving its accuracy.

[0509] This invention is a system that captures a user's health data in real time, analyzes it using a generative artificial intelligence model, and provides appropriate advice and primary treatment instructions based on the results. To implement this system, the following major components are required:

[0510] 1. Device and System Configuration

[0511] The wearable device (AI glasses) worn by the user is equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.). This device captures the user's health data such as complexion, facial expression, body temperature, and heart rate in real time, and records the user's subjective symptoms using a voice input function.

[0512] The device organizes the captured image and audio data, encrypts the data using an advanced encryption algorithm (e.g., AES 256), and transmits it over the Internet to a server.

[0513] The server decrypts the received encrypted data and inputs it into a generative artificial intelligence model. Analysis is performed using tools such as OpenCV (image processing) and Google Cloud Speech-to-Text (voice analysis). The generative artificial intelligence model analyzes the captured data and determines the patient's condition based on factors such as facial color, facial expression, body temperature, and heart rate.

[0514] Based on the analysis results, the generative AI model generates appropriate advice and instructions for first-line treatment for the user. For example, if cold symptoms are observed, the model generates advice such as "Drink plenty of fluids and rest," and if symptoms require urgent treatment, the model generates instructions such as "Call an ambulance immediately."

[0515] The server sends the generated advice and instructions to the terminal, which then notifies the user visually or audibly, allowing the user to take the necessary steps or actions based on the information.

[0516] Specific examples

[0517] Example 1: Early symptoms of a cold

[0518] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[0519] 2. Device: Captures the user's complexion and body temperature, and records voice input such as "I have a sore throat" or "I have a headache."

[0520] 3. Terminal: Encrypts the captured data and sends it to the server.

[0521] 4. Server: Decrypts the received data and inputs it into the generative artificial intelligence model.

[0522] 5. Server: The AI ​​model identifies the early symptoms of a cold and generates advice such as "drink plenty of fluids and get plenty of rest."

[0523] 6. Device: Advice is displayed visually and also given as an audio notification.

[0524] Example prompt sentence:

[0525] "The user's complexion is pale and their body temperature is 38.0 degrees. Their voice input has been recorded as 'sore throat' and 'headache'. Based on this information, please analyze their current health condition and generate appropriate advice."

[0526] Example 2: Acute abdominal pain

[0527] 1. User: A user who suddenly experiences severe abdominal pain in the middle of the night puts on the AI ​​glasses.

[0528] 2. Device: Capture facial color with the camera and record voice input such as "I suddenly have a stomachache."

[0529] 3. Terminal: Encrypts the captured data and sends it to the server.

[0530] 4. Server: Decrypts the data and feeds it into the generative artificial intelligence model.

[0531] 5. Server: The AI ​​model determines that the patient has acute, severe abdominal pain and is highly urgent, and generates instructions such as "Call an ambulance immediately."

[0532] 6. Terminal: Provides visual and audio instructions.

[0533] Example prompt sentence:

[0534] "We have audio data that indicates the user looks pale and has sudden, severe stomach pain. Based on this information, please analyze their health condition and generate urgent instructions."

[0535] This system is expected to improve users' self-management capabilities and reduce the burden on medical facilities by monitoring users' health conditions in real time and quickly providing appropriate advice and instructions for primary treatment.

[0536] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0537] Step 1:

[0538] The user puts on the AI ​​glasses and turns them on. The device's camera and multiple sensors capture the user's health data in real time. Input data includes the user's complexion, facial expression, body temperature, heart rate, etc. Specifically, the camera captures the user's complexion and facial expression, the temperature sensor measures body temperature, and the heart rate sensor measures heart rate. The output data is the captured image data and sensor data.

[0539] Step 2:

[0540] The terminal encrypts the captured image data and audio data. The input data is the image data and audio data captured in step 1. Specifically, the terminal encrypts the data using an encryption algorithm such as AES 256. The encrypted data becomes the output data.

[0541] Step 3:

[0542] The terminal sends the encrypted data to the server. The input data is the data encrypted in step 2. In concrete terms, the terminal sends data to the server via the Internet. The encrypted data received by the server becomes the output data.

[0543] Step 4:

[0544] The server receives and decrypts encrypted data. The input data is the encrypted data sent from the terminal. Specifically, the server decrypts the encrypted data and obtains the original image data and audio data. The decrypted data becomes the output data.

[0545] Step 5:

[0546] The server inputs the decoded data into a generative artificial intelligence model for analysis. The input data consists of decoded image data and audio data. Specifically, the server uses an image processing tool (e.g., OpenCV) to analyze facial color and facial expressions, and uses an audio analysis tool (e.g., Google Cloud Speech-to-Text) to convert the audio data into natural language. The generative artificial intelligence model uses this data to determine the patient's condition and outputs the analysis results for generating advice.

[0547] Step 6:

[0548] Based on the analysis results, the generative AI model generates appropriate advice and instructions for primary treatment. The input data is the analysis results obtained in step 5. Specifically, the model refers to past health data and medical history and compares it with current data. For example, if cold symptoms are observed, the model generates advice such as "drink plenty of fluids and take plenty of rest." The generated advice becomes the output data.

[0549] Step 7:

[0550] The server sends the generated advice and instructions to the terminal. The input data is the advice and instructions generated in step 6. In concrete terms, the server sends the generated advice and instructions to the terminal, and the terminal receives them. The data sent to the terminal becomes the output data.

[0551] Step 8:

[0552] The device notifies the user of the received advice and instructions visually or audibly. The input data is the advice and instructions received from the server. Specifically, the device's transparent display or bone conduction speaker is used to notify the user of the advice and instructions for primary treatment visually and audibly. The user can receive the notification and take the necessary action.

[0553] (Application example 1)

[0554] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0555] There is a need for a system that can monitor users' health status in real time and provide not only appropriate advice and primary treatment instructions, but also optimal diet and nutritional support. However, conventional systems have difficulty meeting these diverse needs, and one issue is the lack of specific advice on diet and nutrition.

[0556] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0557] In this invention, the server includes a terminal means equipped with a camera and a sensor for capturing health data in real time, a means for encrypting the captured image data and audio data and transmitting it to the server, a server means for analyzing the received data using a generative artificial intelligence model to determine the condition of the patient, a means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, a means for generating optimal diet and nutritional support for the user based on the analysis results, a means for transmitting the generated advice and instructions to the terminal and notifying the user, and a means for using the recorded data to connect with medical institutions. This makes it possible to comprehensively manage the user's health condition and quickly provide appropriate advice and nutritional support.

[0558] The "terminal means" is a wearable device worn by a user, and is a device equipped with a camera and sensors for capturing health data in real time.

[0559] The "means for encrypting captured image data and audio data" is a technology for encrypting image data and audio data acquired from a terminal to protect privacy and transmitting the data safely to a server.

[0560] The "server means" is a device that analyzes the received encrypted data, determines the condition using a generative artificial intelligence model, and generates advice and instructions for primary treatment for the user based on the analysis results.

[0561] A "generative artificial intelligence model" is an advanced AI algorithm that analyzes captured health data, determines medical conditions, and generates advice.

[0562] The "means for generating advice and instructions for primary treatment" is a technology that generates appropriate advice and instructions for primary treatment for the user based on the results of analysis using a generative artificial intelligence model.

[0563] The "means for generating meals and nutritional support" is a technology that generates meals and nutritional support that are optimal for the user based on the analysis results.

[0564] The "means for notifying advice and instructions" is a technique for transmitting the generated advice and instructions to the terminal and notifying the user visually or audibly.

[0565] "Means for collaboration with medical institutions" refers to technology for sharing recorded data with medical institutions and collaborating with them.

[0566] A "database" is a data storage device for recording and storing analysis results and advice content.

[0567] "Feedback from users and diagnostic results from medical institutions" refers to information provided by users and medical institutions that contributes to the continuous training and improvement of generative artificial intelligence models.

[0568] The present invention is a system that monitors the user's health condition in real time and provides optimal advice and instructions for primary treatment, as well as appropriate diet and nutritional support. Hereinafter, specific embodiments of the present invention will be described.

[0569] System Configuration

[0570] 1. Terminal means

[0571] The wearable device is equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.). The device captures real-time health data and collects image and audio data. Using the voice input function, users can record their current symptoms and feelings by voice.

[0572] 2. Data Encryption and Transmission

[0573] The device organizes the captured image and audio data in real time and encrypts them for privacy reasons (using a data encryption library such as OpenSSL), then sends the encrypted data to a cloud server.

[0574] 3. Server Means

[0575] The cloud server decrypts the received encrypted data and inputs it into a generative artificial intelligence model (e.g., GPT-4). The model analyzes facial color, facial expressions, and physical features from the image data, and uses natural language processing to extract the user's complaints and subjective symptoms from the voice data. It also references past health data and medical history, and compares this with current data to determine the condition of the patient.

[0576] 4. Advice Generation

[0577] Based on the analysis results, the generative AI model generates advice and instructions for first-line treatment for the user. Furthermore, based on the analysis results, it also generates optimal dietary and nutritional support for the user. For example, if the user shows early symptoms of a cold, it generates advice such as "It would be good to eat a diet rich in vitamin C."

[0578] 5. Notice and Enforcement

[0579] The generated advice and instructions are sent from the server to the device. The device displays them visually and also notifies the user by voice. The user can follow the displayed advice and instructions and take the necessary measures or actions. The corresponding ingredients are also displayed preferentially on the food delivery selection screen.

[0580] 6. Collaboration with medical institutions

[0581] The recorded data will be used to connect with medical institutions as needed, allowing medical institutions to refer to the user's health data and provide appropriate diagnosis and treatment.

[0582] Specific examples

[0583] Example 1: Suggested diet for preventing colds

[0584] User: A user who feels a slight cold puts on the smart glasses.

[0585] Device: Captures heart rate and facial color, and records voice messages such as "My throat has been a bit sore lately."

[0586] Server: Analyzes the captured data and determines whether it is an early symptom of a cold.

[0587] Server: The advice is to "consider a diet that includes citrus fruits and vegetables that are rich in vitamin C."

[0588] Device: Advice is displayed and voice notification is given. The relevant ingredients are also displayed preferentially on the food delivery selection screen.

[0589] Prompt Sentence Examples

[0590] "My throat has been a bit sore lately. What foods would be good to prevent a cold?"

[0591] In this way, the system of the present invention comprehensively manages the user's health condition and quickly provides appropriate advice and nutritional support.

[0592] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0593] Step 1:

[0594] The user puts on the smart glasses (terminal means) and activates the device. Data is collected as input from cameras, temperature sensors, heart rate sensors, etc. to capture the user's health status in real time. The terminal acquires biometric information (complexion, heart rate, body temperature, etc.) from these sensors and voice input, and organizes it as primary data.

[0595] Step 2:

[0596] Encrypt the organized primary data. The device uses a data encryption library (e.g., OpenSSL) to encrypt the collected health data for privacy purposes. It receives the primary data as input and generates encrypted data as output.

[0597] Step 3:

[0598] The encrypted data is sent to the cloud server. The device sends the encrypted data to the server via the Internet. The device receives the encrypted data as input and sends the data to the cloud server as output.

[0599] Step 4:

[0600] The cloud server decrypts the received encrypted data. When the server receives the data, it uses a decryption library to decrypt it. It takes the encrypted data as input and gets the decrypted data as output.

[0601] Step 5:

[0602] The decoded data is input into a generative AI model for analysis. The server uses a generative AI model (e.g., GPT-4) to receive the decoded data as input, analyze facial color, facial expressions, and physical features from the image data, and extract the user's complaints and subjective symptoms from the voice data using natural language processing. The output is a diagnosis of the disease condition and the user's current health condition.

[0603] Step 6:

[0604] Based on the analysis results, the server generates advice, initial treatment instructions, and optimal diet and nutritional support recommendations. Utilizing a generative AI model, the server receives the condition assessment results as input and generates advice, diet, and nutritional support as output. For example, if foods rich in vitamin C are needed, the server creates a specific recommendation such as, "A diet containing foods rich in vitamin C would be good."

[0605] Step 7:

[0606] The generated advice and instructions are sent to the terminal. The server sends the generated advice and instructions to the terminal via the Internet. It receives advice and instructions as input and sends them to the terminal as output.

[0607] Step 8:

[0608] The device notifies the user of the received advice or instructions. The device displays the advice or instructions on the screen or notifies the user by voice. The device receives advice or instructions from the server as input and notifies the user visually or audibly as output. For example, the device may display advice such as "It would be good to eat meals that include ingredients rich in vitamin C," and prioritize the corresponding ingredients on the food delivery selection screen.

[0609] Step 9:

[0610] The user acts according to the advice and instructions received. The user selects the necessary diet and nutritional support based on the displayed advice examples and takes action accordingly. Specifically, the user performs an action such as ordering a meal using the recommended ingredients.

[0611] Through this series of steps, the user is provided with real-time advice and dietary suggestions based on their health status.

[0612] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0613] This invention combines a system that captures a user's health data in real time and provides advice and primary treatment instructions based on the analysis results with an emotion engine that recognizes the user's emotions. To implement this system, the following main components are required:

[0614] 1. Device and System Configuration

[0615] Device: AI glasses are a wearable device worn by the user. Equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.), they capture emotions along with health data. They also have the ability to collect image and audio data, encrypt it, and send it to a server.

[0616] Server: Receives the captured data and analyzes the medical condition using a generative artificial intelligence model and emotion engine. Based on the analysis results, it generates advice and primary treatment instructions and sends them to the device.

[0617] Generative artificial intelligence model: An advanced AI algorithm that analyzes health data, determines medical conditions, and generates advice.

[0618] Emotion engine: An AI algorithm that recognizes emotions from the user's facial expressions and voice data and integrates that data into the analysis results.

[0619] 2. System operation explanation

[0620] The system operates as follows.

[0621] Data collection: The user puts on the AI ​​glasses and turns them on. The device's camera and sensors capture real-time health data, such as the user's complexion, facial expressions, body temperature, and heart rate. At the same time, the voice input function records the user's current symptoms and sensations, and the camera records the user's facial expressions.

[0622] Data encryption and transmission: The device organizes the captured image data, voice data, and facial expression data in real time, encrypts them to protect privacy, and then transmits the data to the server.

[0623] Data analysis: The server decrypts the received encrypted data and converts it into a data format for analysis. Image data is analyzed using computer vision technology to evaluate facial color, facial expressions, and physical characteristics. Voice data is analyzed using natural language processing (NLP) algorithms to extract the user's complaints and subjective symptoms as text data.

[0624] Emotion Recognition: The server uses an emotion engine to analyze the user's emotions from facial expressions and voice, for example, to determine whether the user is stressed or relaxed.

[0625] Data integration and analysis result generation: The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the condition.

[0626] Advice generation: Based on the analysis results, advice and instructions for first-line treatment are generated for the user. Emotional data is also taken into consideration at this time. For example, if the user is feeling stressed, advice on how to relax is added.

[0627] Notification and implementation: The server sends the generated advice and instructions to the terminal, which notifies the user visually or audibly. The user follows the displayed advice and instructions and takes the necessary steps or actions.

[0628] Specific examples

[0629] Example 1: Early symptoms of a cold

[0630] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[0631] 2. Device: Captures the user's complexion, body temperature, and facial expressions, and records symptoms such as "sore throat" or "headache" via voice input.

[0632] 3. Server: Analyzes the captured data and determines whether the user is showing early symptoms of a cold. At the same time, it recognizes that the user's facial expression indicates fatigue.

[0633] 4. Server: Generate the following advice: "You have a sore throat and headache, so drink plenty of fluids and get plenty of rest. You also feel fatigued, so it's recommended that you get plenty of rest."

[0634] 5. Device: Advice is displayed visually on the screen and also provided as an audio notification.

[0635] Example 2: Acute abdominal pain and stress

[0636] 1. User: Feels severe abdominal pain in the middle of the night and puts on the AI ​​glasses.

[0637] 2. Device: The camera captures the user's facial expression and facial color, and the user is given a voice command such as "I suddenly have a stomachache." The device also determines whether the user is feeling stressed based on the user's facial expression.

[0638] 3. Server: Analyzes the data, detects acute severe abdominal pain and high stress levels, and determines that the situation is urgent.

[0639] 4. Server: Generate the following advice: "Severe abdominal pain has been detected, so please call an ambulance immediately. High stress levels have also been detected, so please adopt relaxation techniques once safety is assured."

[0640] 5. Device: Advice is displayed on the screen and also notified by voice.

[0641] In this way, the present invention, which combines an emotion engine, is a system that provides more accurate and personalized medical support while taking into account the user's emotional state. This is expected to improve users' self-management abilities, reduce unnecessary medical expenses, and ease the burden on medical facilities.

[0642] The processing flow will be explained below.

[0643] Step 1:

[0644] The user puts on the AI ​​glasses and turns them on. The AI ​​glasses are equipped with a camera and multiple sensors, which then become ready.

[0645] Step 2:

[0646] The device uses cameras and sensors to capture real-time health data such as the user's complexion, facial expressions, body temperature, and heart rate, while also recording current symptoms and sensations via voice input.

[0647] Step 3:

[0648] The device encrypts the image, voice, and facial expression data it captures to protect privacy and prevent unauthorized access to the data.

[0649] Step 4:

[0650] The device sends the encrypted data to the server, which transmits it using a secure communication protocol.

[0651] Step 5:

[0652] The server decrypts the received encrypted data and converts it into a data format for analysis.

[0653] Step 6:

[0654] The server analyzes the image data using computer vision technology, which evaluates the user's complexion, facial expressions, and physical features.

[0655] Step 7:

[0656] The server analyzes the voice data using natural language processing (NLP) algorithms, which extracts the user's complaints and subjective symptoms as text data.

[0657] Step 8:

[0658] The server uses an emotion engine to analyze the user's emotions from facial expressions and voice. Emotion recognition determines whether the user is stressed, relaxed, or in other emotional states.

[0659] Step 9:

[0660] The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the condition.

[0661] Step 10:

[0662] Based on the analysis results, the server generates appropriate advice and instructions for first-line treatment to provide to the user. Emotional data is also taken into account in this process. For example, if the user is feeling stressed, advice on how to relax will be added.

[0663] Step 11:

[0664] The server sends generated advice and instructions to the terminal.

[0665] Step 12:

[0666] The advice and instructions received by the terminal are visually displayed on the display and also notified to the user audibly.

[0667] Step 13:

[0668] The user follows the displayed advice and instructions and takes the necessary steps or actions, such as drinking more fluids or taking emergency care.

[0669] Step 14:

[0670] The server records the analysis results, generated advice, and the user's reactions in a database, which will be used for future analysis and health management.

[0671] Step 15:

[0672] If the server obtains the user's consent, it will send the analysis results and important health information to the affiliated medical institution.

[0673] Step 16:

[0674] When users provide feedback after using the system, the server receives that information and uses it to improve the accuracy of the generative artificial intelligence model.

[0675] This series of steps will create a system that, through the collaboration between the AI ​​glasses and a server, will analyze the user's health and emotional data in real time and immediately provide appropriate advice and instructions for primary treatment.

[0676] Example 2

[0677] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0678] Conventional health management systems lack analysis and advice that take into account the user's emotional state, making it difficult to accurately assess health status and provide appropriate treatment. They also lack real-time data capture and analysis, making it difficult to respond quickly to user conditions. Furthermore, they lack the ability to compare health data with past data or continuously train generative artificial intelligence algorithms, making it difficult to improve the system's accuracy.

[0679] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information terminal equipped with an image capture device and a biometric sensor for capturing health data and emotional data in real time, an information terminal that encrypts the captured image data and audio data and transmits them to a central processing unit, a central processing unit that analyzes the received data using a generative artificial intelligence algorithm to determine the user's health status, a central processing unit that generates appropriate advice and instructions for primary treatment for the user based on the analysis results, an information terminal that transmits the generated advice and instructions to the information terminal and notifies the user, and a data management device used to link the recorded data with a medical institution. This enables accurate health assessments that take the user's emotional state into consideration, provision of appropriate advice, and prompt real-time responses. Furthermore, the accuracy of the system can be improved by comparing with past health data and continuously training the generative artificial intelligence algorithm.

[0680] "Image capture device" refers to a device such as a camera for capturing a user's facial expression.

[0681] A "biometric sensor" refers to a sensor used to measure a user's health data, such as body temperature or heart rate.

[0682] "Information terminal" refers to a device that captures health and emotional data, encrypts it, and transmits it to a central processing unit.

[0683] "Central Processing Unit" refers to the electronic device that analyzes the received data, determines the health status, and generates appropriate advice and instructions for primary treatment.

[0684] "Generative artificial intelligence algorithm" refers to an advanced computational method that analyzes health and emotional data to determine medical conditions and generate advice.

[0685] "Data management device" refers to the system used to store recorded data and to communicate with medical institutions as needed.

[0686] "Real-time" refers to a processing format in which data is captured immediately and analyzed and notified.

[0687] "Encryption" refers to the technology of converting data to protect the privacy of that information, making it unreadable to third parties.

[0688] "Emotional data" refers to information about the emotional state obtained from the user's facial expressions and voice.

[0689] "Health status" refers to the user's physical and psychological state as assessed by body temperature, heart rate, facial expression, etc.

[0690] "Advice" refers to instructions or recommendations provided to the user based on the analysis results.

[0691] "First steps" refers to instructions that indicate the initial response that a user should take in an emergency.

[0692] This invention is a system that captures and analyzes a user's health and emotional state in real time and provides advice and primary treatment instructions based on the analysis results. This system consists of an information terminal worn by the user, a central processing unit that analyzes the data, a means for providing the generated advice, and a device that manages the recorded data and connects with medical institutions.

[0693] Hardware and software used

[0694] Information terminal: A wearable device worn by the user, equipped with a camera (image capture device) and multiple biometric sensors (temperature sensor, heart rate sensor, etc.). This terminal captures health and emotional data in real time, encrypts the data, and transmits it to a central processing unit.

[0695] Central Processing Unit: A server on the cloud or a local high-performance computing device that analyzes data using generative artificial intelligence algorithms and sentiment analysis engines.

[0696] Data management device: A cloud storage or on-premise database system that records and stores analysis results, advice, past health data, and medical history data.

[0697] Data processing and calculation

[0698] Once worn by the user, the device begins collecting data using its camera and sensors. The camera captures the user's facial expression and complexion, the temperature sensor measures body temperature, the heart rate sensor measures heart rate, and the voice input function records the user's description of symptoms. This collected data is encrypted in real time and sent to a server.

[0699] The server decrypts the received encrypted data and converts it into the appropriate data format. Image data is analyzed using computer vision technology to evaluate the user's facial color and facial expressions. Voice data is analyzed using natural language processing (NLP) algorithms to extract the user's complaints and subjective symptoms as text data. Furthermore, an emotion analysis engine analyzes the user's emotional state from facial expressions and voice.

[0700] The generative AI algorithm analyzes the integrated health and emotional data to assess the user's health status. Based on this assessment, advice and primary treatment instructions are generated. The accuracy of the analysis results is improved by comparing them with past health data and medical history to ensure the appropriateness of advice for the user.

[0701] The generated advice and instructions are then encrypted and sent to the terminal, where the user can receive the information visually or audibly.

[0702] Specific examples

[0703] Example 1: Early symptoms of a cold

[0704] 1. The user feels unwell in the morning and puts on the AI ​​glasses.

[0705] 2. The device captures the user's complexion, body temperature, and facial expressions, and records symptoms such as "sore throat" and "headache" through voice input.

[0706] 3. The server analyzes the captured data and determines whether the user is showing early symptoms of a cold. At the same time, the server recognizes the user's fatigue from their facial expression.

[0707] 4. The server generates the advice, "You have a sore throat and a headache, so drink plenty of fluids and rest. You also feel fatigued, so it's recommended that you get plenty of rest."

[0708] 5. The device will display advice on the screen and also provide audio notifications.

[0709] Example 2: Acute abdominal pain and stress

[0710] 1. The user experiences severe stomach pain in the middle of the night and puts on the AI ​​glasses.

[0711] 2. The device captures facial expression and facial color with a camera and records the user's voice input, such as "I suddenly have a stomachache." It also determines whether the user is feeling stressed based on their facial expression.

[0712] 3. The server analyzes the data, detects acute severe abdominal pain and high stress levels, and determines that the condition is urgent.

[0713] 4. The server generates the advice, "You are experiencing severe abdominal pain, so please call an ambulance immediately. Also, high stress levels have been detected, so please adopt relaxation techniques once safety is assured."

[0714] 5. The device will display advice on the screen and also provide audio notifications.

[0715] Prompt Sentence Examples

[0716] "My temperature is rising, my throat is sore, and I also have a slight headache this morning."

[0717] "I suddenly get a very strong stomach ache."

[0718] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0719] Step 1: Booting the device

[0720] The user puts on the AI ​​glasses and turns on the device. Once the device is successfully booted and ready, the camera and sensors are initialized and ready to collect data. The input is the user's actions, and the output is the device's ready state.

[0721] Specific example of operation: When the user presses the power button on the AI ​​Glasses, the device starts up with a startup sound and displays a notification that it is ready.

[0722] Step 2: Data collection

[0723] The device activates the camera and sensors to capture facial color, facial expressions, body temperature, and heart rate. The voice input function is enabled to record what the user says. The input is the user's health status and voice, and the output is these data.

[0724] Specific operation example: The camera captures the user's face, the temperature sensor measures body temperature, and the heart rate sensor measures heart rate. When the user says something like "I have a headache," the voice is recorded.

[0725] Step 3: Encrypt and send data

[0726] The device encrypts the captured image, audio, and biometric data using the AES-256 encryption algorithm. The encrypted data is sent to the server. The input is raw data, and the output is encrypted data.

[0727] Specific operation example: Collected data (e.g., facial color images, body temperature, heart rate, and voice data) is encrypted and sent to a server via high-speed wireless communication.

[0728] Step 4: Data analysis

[0729] The server decrypts the received encrypted data and converts it into the appropriate data format. Image data is analyzed using computer vision technology to evaluate the user's facial color and facial expression. Voice data is analyzed using natural language processing (NLP) technology to extract the user's complaints and subjective symptoms as text data. The input is encrypted data, and the output is the analysis results.

[0730] Specific example: Analyze image data using OpenCV and extract text from audio data using the Google Cloud Speech-to-Text API.

[0731] Step 5: Emotion Recognition

[0732] The server uses an emotion engine to analyze the user's emotions from facial expressions and voice, for example, to determine stress levels and relaxation states. The input is the analyzed health data and voice data, and the output is an assessment of the user's emotional state.

[0733] Specific example of operation: Using the Emotion AI API to analyze facial expression data and evaluate it as "high stress level."

[0734] Step 6: Integrate and score the data

[0735] The server integrates the acquired health data, emotional data, past health data, and medical history, and uses a generative artificial intelligence algorithm to assess health status. Each data point is assigned a score to determine overall health status. The input is the integrated data, and the output is an overall health score.

[0736] Specific example of how it works: Using TensorFlow to score health status using a generative AI model.

[0737] Step 7: Advice Generation

[0738] The server generates advice and instructions for first-line treatment for the user based on the analysis results. It also takes into account emotional data and adds appropriate recommendations. The input is the analysis results of health data and emotional data, and the output is advice and instructions for first-line treatment.

[0739] Example of specific operation: Advice such as "Drink plenty of fluids and get plenty of rest" is generated in text format.

[0740] Step 8: Notification and enforcement

[0741] The server encrypts the generated advice and instructions and sends them to the terminal. The terminal notifies the user of this information visually or audibly. The input is the encrypted advice data, and the output is the notification to the user.

[0742] Specific example of operation: The AI ​​glasses display will show "Drink plenty of fluids and get plenty of rest," and a voice notification will also be given.

[0743] This detailed processing step allows the user to receive an accurate assessment of their health status in real time and receive appropriate advice and primary treatment instructions.

[0744] (Application example 2)

[0745] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0746] Current security services lack a means to monitor the health and emotional state of security guards in real time, making it difficult to respond quickly when abnormalities in the guard's health or mental state occur. In addition, systems for accurately detecting suspicious individuals and abnormal behavior are still insufficient. This places a heavy burden on security guards and risks reducing overall safety. To address these issues, the present invention aims to provide a comprehensive system that captures health and emotional data in real time and provides appropriate advice and first-line treatment instructions.

[0747] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means equipped with a camera and sensor for capturing health data and emotional data in real time, means for encrypting the captured image data and audio data and transmitting them to the server, server means for analyzing the received data using a generative artificial intelligence model and an emotion engine to determine the medical condition and emotional state, means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, means for transmitting the generated advice and instructions to the terminal and notifying the user, means for using the recorded data to apply to security management, and means for detecting anomalies and taking emergency measures. This makes it possible to monitor the health and emotional state of security guards in real time and to take appropriate action immediately if an abnormality occurs.

[0748] "Terminal means" refers to a device equipped with a camera and sensors for capturing health and emotional data in real time.

[0749] "Encryption means" refers to the technology and method for encrypting captured image and audio data for secure transmission.

[0750] "Server means" is a computer system that analyzes the received data, determines the medical and emotional state, and generates necessary advice and primary treatment instructions.

[0751] A "generative artificial intelligence model" is a machine learning algorithm that analyzes health and emotional data to generate appropriate advice and first-line treatment instructions.

[0752] An "emotion engine" is an algorithm and technology for analyzing a user's facial expressions and voice data to determine their emotional state.

[0753] The "notification means" is a method for transmitting the generated advice and instructions to the terminal and notifying the user visually or audibly.

[0754] "Security management measures" are techniques and methods for utilizing recorded data for security management.

[0755] "Abnormality detection means" refers to techniques and methods for detecting abnormalities in the health or emotional state of security guards.

[0756] "Emergency response measures" are techniques and methods for taking immediate and appropriate action when an abnormality is detected.

[0757] This invention is a system that captures health and emotional data of security guards in real time and provides appropriate advice and instructions for primary treatment based on the analysis results. This system is composed of terminal means, encryption means, server means, generative artificial intelligence model, emotion engine, notification means, security management means, anomaly detection means, and emergency response means.

[0758] 1. Hardware and Software Used

[0759] Terminal means

[0760] A wearable device worn by a user, equipped with a camera and sensors (such as a temperature sensor and a heart rate sensor) to capture health and emotional data. Examples include smart glasses and head-mounted displays.

[0761] Encryption method

[0762] Used to encrypt the captured data. The encryption technology used is a cryptographic library such as OpenSSL.

[0763] Server Means

[0764] It is a computer system that analyzes the received data and determines the patient's medical condition and emotional state. It mainly uses server-side frameworks such as Flask or Django as its software.

[0765] Generative AI model and emotion engine

[0766] Health and emotion data are analyzed using deep learning frameworks such as TensorFlow and Keras, as well as Hugging Face's Transformers.

[0767] Notification means

[0768] This is a mechanism for transmitting advice and instructions for primary measures generated by the server to the terminal means and notifying the user visually and audibly.

[0769] Security Control Measures

[0770] It includes techniques and methods for applying recorded data to security management.

[0771] Anomaly detection and emergency response measures

[0772] This includes techniques and methods for detecting abnormalities in the health or emotional state of security guards and for immediately taking appropriate action.

[0773] 2. Data processing and calculation

[0774] Data collection

[0775] The user wears a device, and cameras and sensors capture data in real time. The collected data is then organized in real time and sent to a server using encryption technology.

[0776] Data analysis

[0777] The server decodes the received data and analyzes it using computer vision techniques and natural language processing algorithms, mobilizing generative artificial intelligence models and emotion engines to determine health and emotional states.

[0778] Advice Generation and Notification

[0779] Based on the analysis results, the server generates the necessary advice and instructions for the first action, which are sent to the terminal means and notified to the user visually and audibly.

[0780] 3. Specific Examples

[0781] Example 1: Suspicious person detection

[0782] If a user (security guard) detects a suspicious person while patrolling, the AI ​​glasses capture the scene and send it to the server, which analyzes it and provides advice such as, "A suspicious person has been spotted. Ensure safety and contact the police."

[0783] Example 2: Detecting health abnormalities

[0784] If a security guard experiences chest pain while on patrol, the AI ​​glasses will capture the image and send it to a server, which will then analyze it and provide advice such as, "Chest pain detected. Please rest immediately."

[0785] Prompt Sentence Examples

[0786] Here are some example prompts to input to the generative AI model:

[0787] Suspicious person detection

[0788] Security Guard A detects a suspicious person while on patrol. Please analyze the data captured by the AI ​​Glasses and provide appropriate countermeasures.

[0789] Security guard's health condition

[0790] Security Guard B experiences chest pains while on patrol. Please provide appropriate first aid advice from captured data.

[0791] As described above, by using the system of the present invention, it is possible to monitor the health and emotional state of security guards in real time, and to take appropriate action immediately if an abnormality occurs.

[0792] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0793] Step 1: Data collection

[0794] The user puts on the terminal (wearable device) and turns it on. The camera and sensors on the terminal capture the user's health and emotional data, such as complexion, facial expression, body temperature, and heart rate, in real time. At the same time, the voice input function records the user's current symptoms and sensations. The input is camera footage, audio, and health data from the sensors, and the output is the captured data.

[0795] Step 2: Data Encryption

[0796] The terminal means organizes the captured image data, voice data, and facial expression data and encrypts them for privacy protection. Specifically, the data is encrypted using the OpenSSL library. The input is the captured data, and the output is the encrypted data.

[0797] Step 3: Send data

[0798] Send encrypted data to a server using a secure communication protocol (e.g., HTTPS). The input is the encrypted data, and the output is a notification to the server that the data has been sent.

[0799] Step 4: Receive and decrypt data

[0800] The server decrypts the encrypted data it receives and converts it into a data format for analysis. Specifically, it uses the OpenSSL library to decrypt the data. The input is the encrypted data received by the server, and the output is the decrypted data.

[0801] Step 5: Data analysis

[0802] The server analyzes image data and audio data. Image data is analyzed for facial color and facial expressions using OpenCV, and audio data is converted to text data using the Transformers library. Health data is analyzed using TensorFlow and Keras. The input is the decoded data, and the output is the analysis result.

[0803] Step 6: Emotion Recognition

[0804] The server uses an emotion engine to analyze emotions from facial expressions and voice data, for example, to determine whether the user is stressed or relaxed. The input is the analyzed facial expression and voice data, and the output is the emotion recognition result.

[0805] Step 7: Data integration and analysis

[0806] The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the patient's condition. The input is the analyzed health data and emotional data, and the output is the integrated analysis result.

[0807] Step 8: Advice Generation

[0808] Based on the analysis results, the server generates advice and instructions for the user, taking into account emotional data. For example, if the user feels stressed, it adds advice on how to relax. The input is the integrated analysis results, and the output is the generated advice and instructions.

[0809] Step 9: Advice Notification

[0810] The server sends the generated advice and instructions to the terminal means, which then notifies the user of the advice and instructions visually or audibly. Specifically, the terminal means uses a display or speaker. The input is the generated advice and instructions, and the output is the notification to the user.

[0811] Step 10: Security Management and Emergency Response

[0812] The server applies the recorded data to security management, detects suspicious individuals, and takes emergency action when a security guard has a health abnormality. The input is user feedback and security data after notification, and the output is improvements to security management and the results of emergency responses.

[0813] These are the specific processing steps for the system that realizes this application example. It is possible to monitor the user's health condition and emotional state in real time and take appropriate action immediately if an abnormality occurs.

[0814] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0815] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0816] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0817] [Third embodiment]

[0818] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0819] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0820] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0821] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0822] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0823] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0824] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0825] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0826] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0827] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0828] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0829] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0830] This invention is a system that captures and analyzes a user's health data in real time and provides appropriate advice and primary treatment instructions. To implement this system, the following major components are required:

[0831] 1. Device and System Configuration

[0832] Device: AI glasses are a wearable device worn by the user. Equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.), they capture health data in real time. They also have the ability to collect image and audio data, encrypt it, and send it to a server.

[0833] Server: Receives the captured data and analyzes the medical condition using a generative artificial intelligence model. Based on the analysis results, it generates appropriate advice and instructions for primary treatment and sends them to the device.

[0834] Generative artificial intelligence model: An advanced AI algorithm that analyzes health data, determines medical conditions, and generates advice.

[0835] 2. System operation explanation

[0836] The system operates as follows.

[0837] Data collection: The user puts on the AI ​​glasses and activates them. The device's camera and sensors capture real-time health data, such as the user's complexion, facial expressions, body temperature, and heart rate. Using the voice input function, the user records their current symptoms and feelings.

[0838] Data encryption and transmission: The device organizes the captured image and audio data in real time, encrypts it for privacy purposes, and then transmits the data to the server.

[0839] Data analysis: The server decrypts the received encrypted data and inputs it into a generative AI model. The model analyzes facial color, facial expressions, and physical characteristics from the image data, and uses natural language processing to extract the user's complaints and subjective symptoms from the voice data. Past health data and medical history are also referenced, and the data is compared with current data.

[0840] Advice generation: Based on the analysis results, the generative AI model generates advice and instructions for first-line treatment for the user. For example, if cold symptoms are observed, basic advice such as "drink plenty of fluids and rest" is provided. If more urgent symptoms are observed, emergency response instructions such as "call an ambulance immediately" are issued.

[0841] Notification and implementation: The server sends the generated advice and instructions to the terminal, which notifies the user visually or audibly. The user follows the displayed advice and instructions and takes the necessary steps or actions.

[0842] Specific examples

[0843] Example 1: Early symptoms of a cold

[0844] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[0845] 2. Device: Captures the user's complexion and body temperature, and records symptoms such as "sore throat" or "headache" via voice input.

[0846] 3. Server: Analyzes the captured data and determines whether early symptoms of a cold are present.

[0847] 4. Server: Generate the advice "You have a sore throat and headache, so drink plenty of fluids and get plenty of rest."

[0848] 5. Device: Advice is displayed visually on the screen and also provided as an audio notification.

[0849] Example 2: Acute abdominal pain

[0850] 1. User: A user who suddenly experiences severe abdominal pain in the middle of the night puts on the AI ​​glasses.

[0851] 2. Device: Capture your facial color with the camera and record "I suddenly have a stomachache" through voice input.

[0852] 3. Server: Analyzes the data and determines that the patient has acute, severe abdominal pain and is in a high state of urgency.

[0853] 4. Server: Generate the advice "The patient is experiencing severe abdominal pain, please call an ambulance immediately."

[0854] 5. Device: Advice is displayed on the screen and also notified by voice.

[0855] In this way, the present invention is a system that provides prompt and appropriate medical care even in remote locations or at night by linking terminals and servers, which is expected to improve users' self-management abilities, reduce unnecessary medical expenses, and ease the burden on medical facilities.

[0856] The processing flow will be explained below.

[0857] Step 1:

[0858] The user puts on the AI ​​glasses and turns them on. The AI ​​glasses are equipped with a camera and multiple sensors, which then become ready.

[0859] Step 2:

[0860] The device uses cameras and sensors to capture real-time health data such as the user's complexion, facial expressions, body temperature, and heart rate, while also recording current symptoms and sensations via voice input.

[0861] Step 3:

[0862] The device encrypts the image and audio data it captures. Encryption is for privacy reasons and to prevent unauthorized access to the data.

[0863] Step 4:

[0864] The device sends the encrypted data to the server, which transmits it using a secure communication protocol.

[0865] Step 5:

[0866] The server decrypts the received encrypted data and converts it into a data format for analysis.

[0867] Step 6:

[0868] The server analyzes the image data using computer vision technology, which evaluates the user's complexion, facial expressions, and physical features.

[0869] Step 7:

[0870] The server analyzes the voice data using natural language processing (NLP) algorithms, which extracts the user's complaints and subjective symptoms as text data.

[0871] Step 8:

[0872] The server references past health data and medical history and compares it with current data to more accurately assess the user's current health status.

[0873] Step 9:

[0874] The server uses a generative artificial intelligence model to analyze the data and determine the condition.

[0875] Step 10:

[0876] Based on the analysis results, the server generates appropriate advice and first-line treatment instructions to provide to the user, such as basic home care advice and instructions for more urgent situations.

[0877] Step 11:

[0878] The server sends generated advice and instructions to the terminal.

[0879] Step 12:

[0880] The advice and instructions received by the terminal are visually displayed on the display and also notified to the user audibly.

[0881] Step 13:

[0882] The user follows the displayed advice and instructions and takes the necessary steps or actions, such as drinking more fluids or taking emergency care.

[0883] Step 14:

[0884] The server records the analysis results, generated advice, and the user's reactions in a database, which will be used for future analysis and health management.

[0885] Step 15:

[0886] If the server obtains the user's consent, it will send the analysis results and important health information to the affiliated medical institution.

[0887] Step 16:

[0888] When users provide feedback after using the system, the server receives that information and uses it to improve the accuracy of the generative artificial intelligence model.

[0889] This series of steps will create a system that, through the collaboration between the AI ​​glasses and a server, will analyze the user's health data in real time and immediately provide appropriate advice and instructions for primary treatment.

[0890] Example 1

[0891] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0892] Health management has become increasingly important in recent years, and there is a particular need for systems that can monitor individual health conditions in real time. However, current health management systems have difficulty collecting and analyzing data in real time, and providing appropriate advice and treatment. Furthermore, they lack a means of linking with medical institutions and other information processing systems while adequately protecting the privacy of collected data. This poses a challenge, preventing users from receiving prompt and appropriate medical care.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0894] In this invention, the server includes terminal means equipped with a camera and multiple sensors and capturing the user's health data in real time, means for encrypting the captured image and audio data and transmitting them to the server, server means for decrypting the received encrypted data, analyzing the data using a generative artificial intelligence model, and determining the medical condition, means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, means for transmitting the generated advice and instructions to the terminal and notifying the user visually and audibly, means for using the recorded data to link with other information processing systems, means for referencing past health data and medical history and comparing them with current health data, means for recording the analysis results and advice content in a recording device, and means for continuously training the generative artificial intelligence model to improve accuracy based on user feedback and diagnostic results from other information processing systems. This allows the user's health condition to be understood in real time, enabling prompt and appropriate medical treatment.

[0895] The "terminal means" is a device worn by a user and equipped with a camera and multiple sensors to capture health data in real time.

[0896] A "data encryption means" is a device that has the function of encrypting captured image and audio data to protect the user's privacy.

[0897] The "server means" is a system that has the function of receiving and decrypting encrypted data, and analyzes the data using a generative artificial intelligence model to determine the condition of the disease.

[0898] A "generative artificial intelligence model" is an artificial intelligence algorithm used to analyze captured health and audio data and generate appropriate advice and primary treatment instructions for the user.

[0899] The "advice generation means" is a function that automatically generates appropriate advice and instructions for primary treatment for the user based on the analysis results.

[0900] The "notification means" is a device that transmits the generated advice and instructions for primary action to the terminal and notifies the user visually and audibly.

[0901] The "linking means" is a function for linking the recorded health data of the user with other information processing systems.

[0902] "Comparison means" refers to the function of referencing past health data and medical history and comparing it with current health data.

[0903] The "recording device" is a memory device or database for storing and managing the analysis results and the generated advice content.

[0904] The "training means" is a function that continuously trains the generative artificial intelligence model based on feedback from users and diagnostic results from other information processing systems, thereby improving its accuracy.

[0905] This invention is a system that captures a user's health data in real time, analyzes it using a generative artificial intelligence model, and provides appropriate advice and primary treatment instructions based on the results. To implement this system, the following major components are required:

[0906] 1. Device and System Configuration

[0907] The wearable device (AI glasses) worn by the user is equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.). This device captures the user's health data such as complexion, facial expression, body temperature, and heart rate in real time, and records the user's subjective symptoms using a voice input function.

[0908] The device organizes the captured image and audio data, encrypts the data using an advanced encryption algorithm (e.g., AES 256), and transmits it over the Internet to a server.

[0909] The server decrypts the received encrypted data and inputs it into a generative artificial intelligence model. Analysis is performed using tools such as OpenCV (image processing) and Google Cloud Speech-to-Text (voice analysis). The generative artificial intelligence model analyzes the captured data and determines the patient's condition based on factors such as facial color, facial expression, body temperature, and heart rate.

[0910] Based on the analysis results, the generative AI model generates appropriate advice and instructions for first-line treatment for the user. For example, if cold symptoms are observed, the model generates advice such as "Drink plenty of fluids and rest," and if symptoms require urgent treatment, the model generates instructions such as "Call an ambulance immediately."

[0911] The server sends the generated advice and instructions to the terminal, which then notifies the user visually or audibly, allowing the user to take the necessary steps or actions based on the information.

[0912] Specific examples

[0913] Example 1: Early symptoms of a cold

[0914] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[0915] 2. Device: Captures the user's complexion and body temperature, and records voice input such as "I have a sore throat" or "I have a headache."

[0916] 3. Terminal: Encrypts the captured data and sends it to the server.

[0917] 4. Server: Decrypts the received data and inputs it into the generative artificial intelligence model.

[0918] 5. Server: The AI ​​model identifies the early symptoms of a cold and generates advice such as "drink plenty of fluids and get plenty of rest."

[0919] 6. Device: Advice is displayed visually and also given as an audio notification.

[0920] Example prompt sentence:

[0921] "The user's complexion is pale and their body temperature is 38.0 degrees. Their voice input has been recorded as 'sore throat' and 'headache'. Based on this information, please analyze their current health condition and generate appropriate advice."

[0922] Example 2: Acute abdominal pain

[0923] 1. User: A user who suddenly experiences severe abdominal pain in the middle of the night puts on the AI ​​glasses.

[0924] 2. Device: Capture facial color with the camera and record voice input such as "I suddenly have a stomachache."

[0925] 3. Terminal: Encrypts the captured data and sends it to the server.

[0926] 4. Server: Decrypts the data and feeds it into the generative artificial intelligence model.

[0927] 5. Server: The AI ​​model determines that the patient has acute, severe abdominal pain and is highly urgent, and generates instructions such as "Call an ambulance immediately."

[0928] 6. Terminal: Provides visual and audio instructions.

[0929] Example prompt sentence:

[0930] "We have audio data that indicates the user looks pale and has sudden, severe stomach pain. Based on this information, please analyze their health condition and generate urgent instructions."

[0931] This system is expected to improve users' self-management capabilities and reduce the burden on medical facilities by monitoring users' health conditions in real time and quickly providing appropriate advice and instructions for primary treatment.

[0932] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0933] Step 1:

[0934] The user puts on the AI ​​glasses and turns them on. The device's camera and multiple sensors capture the user's health data in real time. Input data includes the user's complexion, facial expression, body temperature, heart rate, etc. Specifically, the camera captures the user's complexion and facial expression, the temperature sensor measures body temperature, and the heart rate sensor measures heart rate. The output data is the captured image data and sensor data.

[0935] Step 2:

[0936] The terminal encrypts the captured image data and audio data. The input data is the image data and audio data captured in step 1. Specifically, the terminal encrypts the data using an encryption algorithm such as AES 256. The encrypted data becomes the output data.

[0937] Step 3:

[0938] The terminal sends the encrypted data to the server. The input data is the data encrypted in step 2. In concrete terms, the terminal sends data to the server via the Internet. The encrypted data received by the server becomes the output data.

[0939] Step 4:

[0940] The server receives and decrypts encrypted data. The input data is the encrypted data sent from the terminal. Specifically, the server decrypts the encrypted data and obtains the original image data and audio data. The decrypted data becomes the output data.

[0941] Step 5:

[0942] The server inputs the decoded data into a generative artificial intelligence model for analysis. The input data consists of decoded image data and audio data. Specifically, the server uses an image processing tool (e.g., OpenCV) to analyze facial color and facial expressions, and uses an audio analysis tool (e.g., Google Cloud Speech-to-Text) to convert the audio data into natural language. The generative artificial intelligence model uses this data to determine the patient's condition and outputs the analysis results for generating advice.

[0943] Step 6:

[0944] Based on the analysis results, the generative AI model generates appropriate advice and instructions for primary treatment. The input data is the analysis results obtained in step 5. Specifically, the model refers to past health data and medical history and compares it with current data. For example, if cold symptoms are observed, the model generates advice such as "drink plenty of fluids and take plenty of rest." The generated advice becomes the output data.

[0945] Step 7:

[0946] The server sends the generated advice and instructions to the terminal. The input data is the advice and instructions generated in step 6. In concrete terms, the server sends the generated advice and instructions to the terminal, and the terminal receives them. The data sent to the terminal becomes the output data.

[0947] Step 8:

[0948] The device notifies the user of the received advice and instructions visually or audibly. The input data is the advice and instructions received from the server. Specifically, the device's transparent display or bone conduction speaker is used to notify the user of the advice and instructions for primary treatment visually and audibly. The user can receive the notification and take the necessary action.

[0949] (Application example 1)

[0950] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0951] There is a need for a system that can monitor users' health status in real time and provide not only appropriate advice and primary treatment instructions, but also optimal diet and nutritional support. However, conventional systems have difficulty meeting these diverse needs, and one issue is the lack of specific advice on diet and nutrition.

[0952] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0953] In this invention, the server includes a terminal means equipped with a camera and a sensor for capturing health data in real time, a means for encrypting the captured image data and audio data and transmitting it to the server, a server means for analyzing the received data using a generative artificial intelligence model to determine the condition of the patient, a means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, a means for generating optimal diet and nutritional support for the user based on the analysis results, a means for transmitting the generated advice and instructions to the terminal and notifying the user, and a means for using the recorded data to connect with medical institutions. This makes it possible to comprehensively manage the user's health condition and quickly provide appropriate advice and nutritional support.

[0954] The "terminal means" is a wearable device worn by a user, and is a device equipped with a camera and sensors for capturing health data in real time.

[0955] The "means for encrypting captured image data and audio data" is a technology for encrypting image data and audio data acquired from a terminal to protect privacy and transmitting the data safely to a server.

[0956] The "server means" is a device that analyzes the received encrypted data, determines the condition using a generative artificial intelligence model, and generates advice and instructions for primary treatment for the user based on the analysis results.

[0957] A "generative artificial intelligence model" is an advanced AI algorithm that analyzes captured health data, determines medical conditions, and generates advice.

[0958] The "means for generating advice and instructions for primary treatment" is a technology that generates appropriate advice and instructions for primary treatment for the user based on the results of analysis using a generative artificial intelligence model.

[0959] The "means for generating meals and nutritional support" is a technology that generates meals and nutritional support that are optimal for the user based on the analysis results.

[0960] The "means for notifying advice and instructions" is a technique for transmitting the generated advice and instructions to the terminal and notifying the user visually or audibly.

[0961] "Means for collaboration with medical institutions" refers to technology for sharing recorded data with medical institutions and collaborating with them.

[0962] A "database" is a data storage device for recording and storing analysis results and advice content.

[0963] "Feedback from users and diagnostic results from medical institutions" refers to information provided by users and medical institutions that contributes to the continuous training and improvement of generative artificial intelligence models.

[0964] The present invention is a system that monitors the user's health condition in real time and provides optimal advice and instructions for primary treatment, as well as appropriate diet and nutritional support. Hereinafter, specific embodiments of the present invention will be described.

[0965] System Configuration

[0966] 1. Terminal means

[0967] The wearable device is equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.). The device captures real-time health data and collects image and audio data. Using the voice input function, users can record their current symptoms and feelings by voice.

[0968] 2. Data Encryption and Transmission

[0969] The device organizes the captured image and audio data in real time and encrypts them for privacy reasons (using a data encryption library such as OpenSSL), then sends the encrypted data to a cloud server.

[0970] 3. Server Means

[0971] The cloud server decrypts the received encrypted data and inputs it into a generative artificial intelligence model (e.g., GPT-4). The model analyzes facial color, facial expressions, and physical features from the image data, and uses natural language processing to extract the user's complaints and subjective symptoms from the voice data. It also references past health data and medical history, and compares this with current data to determine the condition of the patient.

[0972] 4. Advice Generation

[0973] Based on the analysis results, the generative AI model generates advice and instructions for first-line treatment for the user. Furthermore, based on the analysis results, it also generates optimal dietary and nutritional support for the user. For example, if the user shows early symptoms of a cold, it generates advice such as "It would be good to eat a diet rich in vitamin C."

[0974] 5. Notice and Enforcement

[0975] The generated advice and instructions are sent from the server to the device. The device displays them visually and also notifies the user by voice. The user can follow the displayed advice and instructions and take the necessary measures or actions. The corresponding ingredients are also displayed preferentially on the food delivery selection screen.

[0976] 6. Collaboration with medical institutions

[0977] The recorded data will be used to connect with medical institutions as needed, allowing medical institutions to refer to the user's health data and provide appropriate diagnosis and treatment.

[0978] Specific examples

[0979] Example 1: Suggested diet for preventing colds

[0980] User: A user who feels a slight cold puts on the smart glasses.

[0981] Device: Captures heart rate and facial color, and records voice messages such as "My throat has been a bit sore lately."

[0982] Server: Analyzes the captured data and determines whether it is an early symptom of a cold.

[0983] Server: The advice is to "consider a diet that includes citrus fruits and vegetables that are rich in vitamin C."

[0984] Device: Advice is displayed and voice notification is given. The relevant ingredients are also displayed preferentially on the food delivery selection screen.

[0985] Prompt Sentence Examples

[0986] "My throat has been a bit sore lately. What foods would be good to prevent a cold?"

[0987] In this way, the system of the present invention comprehensively manages the user's health condition and quickly provides appropriate advice and nutritional support.

[0988] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0989] Step 1:

[0990] The user puts on the smart glasses (terminal means) and activates the device. Data is collected as input from cameras, temperature sensors, heart rate sensors, etc. to capture the user's health status in real time. The terminal acquires biometric information (complexion, heart rate, body temperature, etc.) from these sensors and voice input, and organizes it as primary data.

[0991] Step 2:

[0992] Encrypt the organized primary data. The device uses a data encryption library (e.g., OpenSSL) to encrypt the collected health data for privacy purposes. It receives the primary data as input and generates encrypted data as output.

[0993] Step 3:

[0994] The encrypted data is sent to the cloud server. The device sends the encrypted data to the server via the Internet. The device receives the encrypted data as input and sends the data to the cloud server as output.

[0995] Step 4:

[0996] The cloud server decrypts the received encrypted data. When the server receives the data, it uses a decryption library to decrypt it. It takes the encrypted data as input and gets the decrypted data as output.

[0997] Step 5:

[0998] The decoded data is input into a generative AI model for analysis. The server uses a generative AI model (e.g., GPT-4) to receive the decoded data as input, analyze facial color, facial expressions, and physical features from the image data, and extract the user's complaints and subjective symptoms from the voice data using natural language processing. The output is a diagnosis of the disease condition and the user's current health condition.

[0999] Step 6:

[1000] Based on the analysis results, the server generates advice, initial treatment instructions, and optimal diet and nutritional support recommendations. Utilizing a generative AI model, the server receives the condition assessment results as input and generates advice, diet, and nutritional support as output. For example, if foods rich in vitamin C are needed, the server creates a specific recommendation such as, "A diet containing foods rich in vitamin C would be good."

[1001] Step 7:

[1002] The generated advice and instructions are sent to the terminal. The server sends the generated advice and instructions to the terminal via the Internet. It receives advice and instructions as input and sends them to the terminal as output.

[1003] Step 8:

[1004] The device notifies the user of the received advice or instructions. The device displays the advice or instructions on the screen or notifies the user by voice. The device receives advice or instructions from the server as input and notifies the user visually or audibly as output. For example, the device may display advice such as "It would be good to eat meals that include ingredients rich in vitamin C," and prioritize the corresponding ingredients on the food delivery selection screen.

[1005] Step 9:

[1006] The user acts according to the advice and instructions received. The user selects the necessary diet and nutritional support based on the displayed advice examples and takes action accordingly. Specifically, the user performs an action such as ordering a meal using the recommended ingredients.

[1007] Through this series of steps, the user is provided with real-time advice and dietary suggestions based on their health status.

[1008] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1009] This invention combines a system that captures a user's health data in real time and provides advice and primary treatment instructions based on the analysis results with an emotion engine that recognizes the user's emotions. To implement this system, the following main components are required:

[1010] 1. Device and System Configuration

[1011] Device: AI glasses are a wearable device worn by the user. Equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.), they capture emotions along with health data. They also have the ability to collect image and audio data, encrypt it, and send it to a server.

[1012] Server: Receives the captured data and analyzes the medical condition using a generative artificial intelligence model and emotion engine. Based on the analysis results, it generates advice and primary treatment instructions and sends them to the device.

[1013] Generative artificial intelligence model: An advanced AI algorithm that analyzes health data, determines medical conditions, and generates advice.

[1014] Emotion engine: An AI algorithm that recognizes emotions from the user's facial expressions and voice data and integrates that data into the analysis results.

[1015] 2. System operation explanation

[1016] The system operates as follows.

[1017] Data collection: The user puts on the AI ​​glasses and turns them on. The device's camera and sensors capture real-time health data, such as the user's complexion, facial expressions, body temperature, and heart rate. At the same time, the voice input function records the user's current symptoms and sensations, and the camera records the user's facial expressions.

[1018] Data encryption and transmission: The device organizes the captured image data, voice data, and facial expression data in real time, encrypts them to protect privacy, and then transmits the data to the server.

[1019] Data analysis: The server decrypts the received encrypted data and converts it into a data format for analysis. Image data is analyzed using computer vision technology to evaluate facial color, facial expressions, and physical characteristics. Voice data is analyzed using natural language processing (NLP) algorithms to extract the user's complaints and subjective symptoms as text data.

[1020] Emotion Recognition: The server uses an emotion engine to analyze the user's emotions from facial expressions and voice, for example, to determine whether the user is stressed or relaxed.

[1021] Data integration and analysis result generation: The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the condition.

[1022] Advice generation: Based on the analysis results, advice and instructions for first-line treatment are generated for the user. Emotional data is also taken into consideration at this time. For example, if the user is feeling stressed, advice on how to relax is added.

[1023] Notification and implementation: The server sends the generated advice and instructions to the terminal, which notifies the user visually or audibly. The user follows the displayed advice and instructions and takes the necessary steps or actions.

[1024] Specific examples

[1025] Example 1: Early symptoms of a cold

[1026] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[1027] 2. Device: Captures the user's complexion, body temperature, and facial expressions, and records symptoms such as "sore throat" or "headache" via voice input.

[1028] 3. Server: Analyzes the captured data and determines whether the user is showing early symptoms of a cold. At the same time, it recognizes that the user's facial expression indicates fatigue.

[1029] 4. Server: Generate the following advice: "You have a sore throat and headache, so drink plenty of fluids and get plenty of rest. You also feel fatigued, so it's recommended that you get plenty of rest."

[1030] 5. Device: Advice is displayed visually on the screen and also provided as an audio notification.

[1031] Example 2: Acute abdominal pain and stress

[1032] 1. User: Feels severe abdominal pain in the middle of the night and puts on the AI ​​glasses.

[1033] 2. Device: The camera captures the user's facial expression and facial color, and the user is given a voice command such as "I suddenly have a stomachache." The device also determines whether the user is feeling stressed based on the user's facial expression.

[1034] 3. Server: Analyzes the data, detects acute severe abdominal pain and high stress levels, and determines that the situation is urgent.

[1035] 4. Server: Generate the following advice: "Severe abdominal pain has been detected, so please call an ambulance immediately. High stress levels have also been detected, so please adopt relaxation techniques once safety is assured."

[1036] 5. Device: Advice is displayed on the screen and also notified by voice.

[1037] In this way, the present invention, which combines an emotion engine, is a system that provides more accurate and personalized medical support while taking into account the user's emotional state. This is expected to improve users' self-management abilities, reduce unnecessary medical expenses, and ease the burden on medical facilities.

[1038] The processing flow will be explained below.

[1039] Step 1:

[1040] The user puts on the AI ​​glasses and turns them on. The AI ​​glasses are equipped with a camera and multiple sensors, which then become ready.

[1041] Step 2:

[1042] The device uses cameras and sensors to capture real-time health data such as the user's complexion, facial expressions, body temperature, and heart rate, while also recording current symptoms and sensations via voice input.

[1043] Step 3:

[1044] The device encrypts the image, voice, and facial expression data it captures to protect privacy and prevent unauthorized access to the data.

[1045] Step 4:

[1046] The device sends the encrypted data to the server, which transmits it using a secure communication protocol.

[1047] Step 5:

[1048] The server decrypts the received encrypted data and converts it into a data format for analysis.

[1049] Step 6:

[1050] The server analyzes the image data using computer vision technology, which evaluates the user's complexion, facial expressions, and physical features.

[1051] Step 7:

[1052] The server analyzes the voice data using natural language processing (NLP) algorithms, which extracts the user's complaints and subjective symptoms as text data.

[1053] Step 8:

[1054] The server uses an emotion engine to analyze the user's emotions from facial expressions and voice. Emotion recognition determines whether the user is stressed, relaxed, or in other emotional states.

[1055] Step 9:

[1056] The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the condition.

[1057] Step 10:

[1058] Based on the analysis results, the server generates appropriate advice and instructions for first-line treatment to provide to the user. Emotional data is also taken into account in this process. For example, if the user is feeling stressed, advice on how to relax will be added.

[1059] Step 11:

[1060] The server sends generated advice and instructions to the terminal.

[1061] Step 12:

[1062] The advice and instructions received by the terminal are visually displayed on the display and also notified to the user audibly.

[1063] Step 13:

[1064] The user follows the displayed advice and instructions and takes the necessary steps or actions, such as drinking more fluids or taking emergency care.

[1065] Step 14:

[1066] The server records the analysis results, generated advice, and the user's reactions in a database, which will be used for future analysis and health management.

[1067] Step 15:

[1068] If the server obtains the user's consent, it will send the analysis results and important health information to the affiliated medical institution.

[1069] Step 16:

[1070] When users provide feedback after using the system, the server receives that information and uses it to improve the accuracy of the generative artificial intelligence model.

[1071] This series of steps will create a system that, through the collaboration between the AI ​​glasses and a server, will analyze the user's health and emotional data in real time and immediately provide appropriate advice and instructions for primary treatment.

[1072] Example 2

[1073] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1074] Conventional health management systems lack analysis and advice that take into account the user's emotional state, making it difficult to accurately assess health status and provide appropriate treatment. They also lack real-time data capture and analysis, making it difficult to respond quickly to user conditions. Furthermore, they lack the ability to compare health data with past data or continuously train generative artificial intelligence algorithms, making it difficult to improve the system's accuracy.

[1075] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information terminal equipped with an image capture device and a biometric sensor for capturing health data and emotional data in real time, an information terminal that encrypts the captured image data and audio data and transmits them to a central processing unit, a central processing unit that analyzes the received data using a generative artificial intelligence algorithm to determine the user's health status, a central processing unit that generates appropriate advice and instructions for primary treatment for the user based on the analysis results, an information terminal that transmits the generated advice and instructions to the information terminal and notifies the user, and a data management device used to link the recorded data with a medical institution. This enables accurate health assessments that take the user's emotional state into consideration, provision of appropriate advice, and prompt real-time responses. Furthermore, the accuracy of the system can be improved by comparing with past health data and continuously training the generative artificial intelligence algorithm.

[1076] "Image capture device" refers to a device such as a camera for capturing a user's facial expression.

[1077] A "biometric sensor" refers to a sensor used to measure a user's health data, such as body temperature or heart rate.

[1078] "Information terminal" refers to a device that captures health and emotional data, encrypts it, and transmits it to a central processing unit.

[1079] "Central Processing Unit" refers to the electronic device that analyzes the received data, determines the health status, and generates appropriate advice and instructions for primary treatment.

[1080] "Generative artificial intelligence algorithm" refers to an advanced computational method that analyzes health and emotional data to determine medical conditions and generate advice.

[1081] "Data management device" refers to the system used to store recorded data and to communicate with medical institutions as needed.

[1082] "Real-time" refers to a processing format in which data is captured immediately and analyzed and notified.

[1083] "Encryption" refers to the technology of converting data to protect the privacy of that information, making it unreadable to third parties.

[1084] "Emotional data" refers to information about the emotional state obtained from the user's facial expressions and voice.

[1085] "Health status" refers to the user's physical and psychological state as assessed by body temperature, heart rate, facial expression, etc.

[1086] "Advice" refers to instructions or recommendations provided to the user based on the analysis results.

[1087] "First steps" refers to instructions that indicate the initial response that a user should take in an emergency.

[1088] This invention is a system that captures and analyzes a user's health and emotional state in real time and provides advice and primary treatment instructions based on the analysis results. This system consists of an information terminal worn by the user, a central processing unit that analyzes the data, a means for providing the generated advice, and a device that manages the recorded data and connects with medical institutions.

[1089] Hardware and software used

[1090] Information terminal: A wearable device worn by the user, equipped with a camera (image capture device) and multiple biometric sensors (temperature sensor, heart rate sensor, etc.). This terminal captures health and emotional data in real time, encrypts the data, and transmits it to a central processing unit.

[1091] Central Processing Unit: A server on the cloud or a local high-performance computing device that analyzes data using generative artificial intelligence algorithms and sentiment analysis engines.

[1092] Data management device: A cloud storage or on-premise database system that records and stores analysis results, advice, past health data, and medical history data.

[1093] Data processing and calculation

[1094] Once worn by the user, the device begins collecting data using its camera and sensors. The camera captures the user's facial expression and complexion, the temperature sensor measures body temperature, the heart rate sensor measures heart rate, and the voice input function records the user's description of symptoms. This collected data is encrypted in real time and sent to a server.

[1095] The server decrypts the received encrypted data and converts it into the appropriate data format. Image data is analyzed using computer vision technology to evaluate the user's facial color and facial expressions. Voice data is analyzed using natural language processing (NLP) algorithms to extract the user's complaints and subjective symptoms as text data. Furthermore, an emotion analysis engine analyzes the user's emotional state from facial expressions and voice.

[1096] The generative AI algorithm analyzes the integrated health and emotional data to assess the user's health status. Based on this assessment, advice and primary treatment instructions are generated. The accuracy of the analysis results is improved by comparing them with past health data and medical history to ensure the appropriateness of advice for the user.

[1097] The generated advice and instructions are then encrypted and sent to the terminal, where the user can receive the information visually or audibly.

[1098] Specific examples

[1099] Example 1: Early symptoms of a cold

[1100] 1. The user feels unwell in the morning and puts on the AI ​​glasses.

[1101] 2. The device captures the user's complexion, body temperature, and facial expressions, and records symptoms such as "sore throat" and "headache" through voice input.

[1102] 3. The server analyzes the captured data and determines whether the user is showing early symptoms of a cold. At the same time, the server recognizes the user's fatigue from their facial expression.

[1103] 4. The server generates the advice, "You have a sore throat and a headache, so drink plenty of fluids and rest. You also feel fatigued, so it's recommended that you get plenty of rest."

[1104] 5. The device will display advice on the screen and also provide audio notifications.

[1105] Example 2: Acute abdominal pain and stress

[1106] 1. The user experiences severe stomach pain in the middle of the night and puts on the AI ​​glasses.

[1107] 2. The device captures facial expression and facial color with a camera and records the user's voice input, such as "I suddenly have a stomachache." It also determines whether the user is feeling stressed based on their facial expression.

[1108] 3. The server analyzes the data, detects acute severe abdominal pain and high stress levels, and determines that the condition is urgent.

[1109] 4. The server generates the advice, "You are experiencing severe abdominal pain, so please call an ambulance immediately. Also, high stress levels have been detected, so please adopt relaxation techniques once safety is assured."

[1110] 5. The device will display advice on the screen and also provide audio notifications.

[1111] Prompt Sentence Examples

[1112] "My temperature is rising, my throat is sore, and I also have a slight headache this morning."

[1113] "I suddenly get a very strong stomach ache."

[1114] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1115] Step 1: Booting the device

[1116] The user puts on the AI ​​glasses and turns on the device. Once the device is successfully booted and ready, the camera and sensors are initialized and ready to collect data. The input is the user's actions, and the output is the device's ready state.

[1117] Specific example of operation: When the user presses the power button on the AI ​​Glasses, the device starts up with a startup sound and displays a notification that it is ready.

[1118] Step 2: Data collection

[1119] The device activates the camera and sensors to capture facial color, facial expressions, body temperature, and heart rate. The voice input function is enabled to record what the user says. The input is the user's health status and voice, and the output is these data.

[1120] Specific operation example: The camera captures the user's face, the temperature sensor measures body temperature, and the heart rate sensor measures heart rate. When the user says something like "I have a headache," the voice is recorded.

[1121] Step 3: Encrypt and send data

[1122] The device encrypts the captured image, audio, and biometric data using the AES-256 encryption algorithm. The encrypted data is sent to the server. The input is raw data, and the output is encrypted data.

[1123] Specific operation example: Collected data (e.g., facial color images, body temperature, heart rate, and voice data) is encrypted and sent to a server via high-speed wireless communication.

[1124] Step 4: Data analysis

[1125] The server decrypts the received encrypted data and converts it into the appropriate data format. Image data is analyzed using computer vision technology to evaluate the user's facial color and facial expression. Voice data is analyzed using natural language processing (NLP) technology to extract the user's complaints and subjective symptoms as text data. The input is encrypted data, and the output is the analysis results.

[1126] Specific example: Analyze image data using OpenCV and extract text from audio data using the Google Cloud Speech-to-Text API.

[1127] Step 5: Emotion Recognition

[1128] The server uses an emotion engine to analyze the user's emotions from facial expressions and voice, for example, to determine stress levels and relaxation states. The input is the analyzed health data and voice data, and the output is an assessment of the user's emotional state.

[1129] Specific example of operation: Using the Emotion AI API to analyze facial expression data and evaluate it as "high stress level."

[1130] Step 6: Integrate and score the data

[1131] The server integrates the acquired health data, emotional data, past health data, and medical history, and uses a generative artificial intelligence algorithm to assess health status. Each data point is assigned a score to determine overall health status. The input is the integrated data, and the output is an overall health score.

[1132] Specific example of how it works: Using TensorFlow to score health status using a generative AI model.

[1133] Step 7: Advice Generation

[1134] The server generates advice and instructions for first-line treatment for the user based on the analysis results. It also takes into account emotional data and adds appropriate recommendations. The input is the analysis results of health data and emotional data, and the output is advice and instructions for first-line treatment.

[1135] Example of specific operation: Advice such as "Drink plenty of fluids and get plenty of rest" is generated in text format.

[1136] Step 8: Notification and enforcement

[1137] The server encrypts the generated advice and instructions and sends them to the terminal. The terminal notifies the user of this information visually or audibly. The input is the encrypted advice data, and the output is the notification to the user.

[1138] Specific example of operation: The AI ​​glasses display will show "Drink plenty of fluids and get plenty of rest," and a voice notification will also be given.

[1139] This detailed processing step allows the user to receive an accurate assessment of their health status in real time and receive appropriate advice and primary treatment instructions.

[1140] (Application example 2)

[1141] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1142] Current security services lack a means to monitor the health and emotional state of security guards in real time, making it difficult to respond quickly when abnormalities in the guard's health or mental state occur. In addition, systems for accurately detecting suspicious individuals and abnormal behavior are still insufficient. This places a heavy burden on security guards and risks reducing overall safety. To address these issues, the present invention aims to provide a comprehensive system that captures health and emotional data in real time and provides appropriate advice and first-line treatment instructions.

[1143] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means equipped with a camera and sensor for capturing health data and emotional data in real time, means for encrypting the captured image data and audio data and transmitting them to the server, server means for analyzing the received data using a generative artificial intelligence model and an emotion engine to determine the medical condition and emotional state, means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, means for transmitting the generated advice and instructions to the terminal and notifying the user, means for using the recorded data to apply to security management, and means for detecting anomalies and taking emergency measures. This makes it possible to monitor the health and emotional state of security guards in real time and to take appropriate action immediately if an abnormality occurs.

[1144] "Terminal means" refers to a device equipped with a camera and sensors for capturing health and emotional data in real time.

[1145] "Encryption means" refers to the technology and method for encrypting captured image and audio data for secure transmission.

[1146] "Server means" is a computer system that analyzes the received data, determines the medical and emotional state, and generates necessary advice and primary treatment instructions.

[1147] A "generative artificial intelligence model" is a machine learning algorithm that analyzes health and emotional data to generate appropriate advice and first-line treatment instructions.

[1148] An "emotion engine" is an algorithm and technology for analyzing a user's facial expressions and voice data to determine their emotional state.

[1149] The "notification means" is a method for transmitting the generated advice and instructions to the terminal and notifying the user visually or audibly.

[1150] "Security management measures" are techniques and methods for utilizing recorded data for security management.

[1151] "Abnormality detection means" refers to techniques and methods for detecting abnormalities in the health or emotional state of security guards.

[1152] "Emergency response measures" are techniques and methods for taking immediate and appropriate action when an abnormality is detected.

[1153] This invention is a system that captures health and emotional data of security guards in real time and provides appropriate advice and instructions for primary treatment based on the analysis results. This system is composed of terminal means, encryption means, server means, generative artificial intelligence model, emotion engine, notification means, security management means, anomaly detection means, and emergency response means.

[1154] 1. Hardware and Software Used

[1155] Terminal means

[1156] A wearable device worn by a user, equipped with a camera and sensors (such as a temperature sensor and a heart rate sensor) to capture health and emotional data. Examples include smart glasses and head-mounted displays.

[1157] Encryption method

[1158] Used to encrypt the captured data. The encryption technology used is a cryptographic library such as OpenSSL.

[1159] Server Means

[1160] It is a computer system that analyzes the received data and determines the patient's medical condition and emotional state. It mainly uses server-side frameworks such as Flask or Django as its software.

[1161] Generative AI model and emotion engine

[1162] Health and emotion data are analyzed using deep learning frameworks such as TensorFlow and Keras, as well as Hugging Face's Transformers.

[1163] Notification means

[1164] This is a mechanism for transmitting advice and instructions for primary measures generated by the server to the terminal means and notifying the user visually and audibly.

[1165] Security Control Measures

[1166] It includes techniques and methods for applying recorded data to security management.

[1167] Anomaly detection and emergency response measures

[1168] This includes techniques and methods for detecting abnormalities in the health or emotional state of security guards and for immediately taking appropriate action.

[1169] 2. Data processing and calculation

[1170] Data collection

[1171] The user wears a device, and cameras and sensors capture data in real time. The collected data is then organized in real time and sent to a server using encryption technology.

[1172] Data analysis

[1173] The server decodes the received data and analyzes it using computer vision techniques and natural language processing algorithms, mobilizing generative artificial intelligence models and emotion engines to determine health and emotional states.

[1174] Advice Generation and Notification

[1175] Based on the analysis results, the server generates the necessary advice and instructions for the first action, which are sent to the terminal means and notified to the user visually and audibly.

[1176] 3. Specific Examples

[1177] Example 1: Suspicious person detection

[1178] If a user (security guard) detects a suspicious person while patrolling, the AI ​​glasses capture the scene and send it to the server, which analyzes it and provides advice such as, "A suspicious person has been spotted. Ensure safety and contact the police."

[1179] Example 2: Detecting health abnormalities

[1180] If a security guard experiences chest pain while on patrol, the AI ​​glasses will capture the image and send it to a server, which will then analyze it and provide advice such as, "Chest pain detected. Please rest immediately."

[1181] Prompt Sentence Examples

[1182] Here are some example prompts to input to the generative AI model:

[1183] Suspicious person detection

[1184] Security Guard A detects a suspicious person while on patrol. Please analyze the data captured by the AI ​​Glasses and provide appropriate countermeasures.

[1185] Security guard's health condition

[1186] Security Guard B experiences chest pains while on patrol. Please provide appropriate first aid advice from captured data.

[1187] As described above, by using the system of the present invention, it is possible to monitor the health and emotional state of security guards in real time, and to take appropriate action immediately if an abnormality occurs.

[1188] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1189] Step 1: Data collection

[1190] The user puts on the terminal (wearable device) and turns it on. The camera and sensors on the terminal capture the user's health and emotional data, such as complexion, facial expression, body temperature, and heart rate, in real time. At the same time, the voice input function records the user's current symptoms and sensations. The input is camera footage, audio, and health data from the sensors, and the output is the captured data.

[1191] Step 2: Data Encryption

[1192] The terminal means organizes the captured image data, voice data, and facial expression data and encrypts them for privacy protection. Specifically, the data is encrypted using the OpenSSL library. The input is the captured data, and the output is the encrypted data.

[1193] Step 3: Send data

[1194] Send encrypted data to a server using a secure communication protocol (e.g., HTTPS). The input is the encrypted data, and the output is a notification to the server that the data has been sent.

[1195] Step 4: Receive and decrypt data

[1196] The server decrypts the encrypted data it receives and converts it into a data format for analysis. Specifically, it uses the OpenSSL library to decrypt the data. The input is the encrypted data received by the server, and the output is the decrypted data.

[1197] Step 5: Data analysis

[1198] The server analyzes image data and audio data. Image data is analyzed for facial color and facial expressions using OpenCV, and audio data is converted to text data using the Transformers library. Health data is analyzed using TensorFlow and Keras. The input is the decoded data, and the output is the analysis result.

[1199] Step 6: Emotion Recognition

[1200] The server uses an emotion engine to analyze emotions from facial expressions and voice data, for example, to determine whether the user is stressed or relaxed. The input is the analyzed facial expression and voice data, and the output is the emotion recognition result.

[1201] Step 7: Data integration and analysis

[1202] The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the patient's condition. The input is the analyzed health data and emotional data, and the output is the integrated analysis result.

[1203] Step 8: Advice Generation

[1204] Based on the analysis results, the server generates advice and instructions for the user, taking into account emotional data. For example, if the user feels stressed, it adds advice on how to relax. The input is the integrated analysis results, and the output is the generated advice and instructions.

[1205] Step 9: Advice Notification

[1206] The server sends the generated advice and instructions to the terminal means, which then notifies the user of the advice and instructions visually or audibly. Specifically, the terminal means uses a display or speaker. The input is the generated advice and instructions, and the output is the notification to the user.

[1207] Step 10: Security Management and Emergency Response

[1208] The server applies the recorded data to security management, detects suspicious individuals, and takes emergency action when a security guard has a health abnormality. The input is user feedback and security data after notification, and the output is improvements to security management and the results of emergency responses.

[1209] These are the specific processing steps for the system that realizes this application example. It is possible to monitor the user's health condition and emotional state in real time and take appropriate action immediately if an abnormality occurs.

[1210] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1211] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1212] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1213] [Fourth embodiment]

[1214] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1215] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1216] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1217] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1218] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1219] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1220] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1221] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1222] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1223] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1224] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1225] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1226] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1227] This invention is a system that captures and analyzes a user's health data in real time and provides appropriate advice and primary treatment instructions. To implement this system, the following major components are required:

[1228] 1. Device and System Configuration

[1229] Device: AI glasses are a wearable device worn by the user. Equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.), they capture health data in real time. They also have the ability to collect image and audio data, encrypt it, and send it to a server.

[1230] Server: Receives the captured data and analyzes the medical condition using a generative artificial intelligence model. Based on the analysis results, it generates appropriate advice and instructions for primary treatment and sends them to the device.

[1231] Generative artificial intelligence model: An advanced AI algorithm that analyzes health data, determines medical conditions, and generates advice.

[1232] 2. System operation explanation

[1233] The system operates as follows.

[1234] Data collection: The user puts on the AI ​​glasses and activates them. The device's camera and sensors capture real-time health data, such as the user's complexion, facial expressions, body temperature, and heart rate. Using the voice input function, the user records their current symptoms and feelings.

[1235] Data encryption and transmission: The device organizes the captured image and audio data in real time, encrypts it for privacy purposes, and then transmits the data to the server.

[1236] Data analysis: The server decrypts the received encrypted data and inputs it into a generative AI model. The model analyzes facial color, facial expressions, and physical characteristics from the image data, and uses natural language processing to extract the user's complaints and subjective symptoms from the voice data. Past health data and medical history are also referenced, and the data is compared with current data.

[1237] Advice generation: Based on the analysis results, the generative AI model generates advice and instructions for first-line treatment for the user. For example, if cold symptoms are observed, basic advice such as "drink plenty of fluids and rest" is provided. If more urgent symptoms are observed, emergency response instructions such as "call an ambulance immediately" are issued.

[1238] Notification and implementation: The server sends the generated advice and instructions to the terminal, which notifies the user visually or audibly. The user follows the displayed advice and instructions and takes the necessary steps or actions.

[1239] Specific examples

[1240] Example 1: Early symptoms of a cold

[1241] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[1242] 2. Device: Captures the user's complexion and body temperature, and records symptoms such as "sore throat" or "headache" via voice input.

[1243] 3. Server: Analyzes the captured data and determines whether early symptoms of a cold are present.

[1244] 4. Server: Generate the advice "You have a sore throat and headache, so drink plenty of fluids and get plenty of rest."

[1245] 5. Device: Advice is displayed visually on the screen and also provided as an audio notification.

[1246] Example 2: Acute abdominal pain

[1247] 1. User: A user who suddenly experiences severe abdominal pain in the middle of the night puts on the AI ​​glasses.

[1248] 2. Device: Capture your facial color with the camera and record "I suddenly have a stomachache" through voice input.

[1249] 3. Server: Analyzes the data and determines that the patient has acute, severe abdominal pain and is in a high state of urgency.

[1250] 4. Server: Generate the advice "The patient is experiencing severe abdominal pain, please call an ambulance immediately."

[1251] 5. Device: Advice is displayed on the screen and also notified by voice.

[1252] In this way, the present invention is a system that provides prompt and appropriate medical care even in remote locations or at night by linking terminals and servers, which is expected to improve users' self-management abilities, reduce unnecessary medical expenses, and ease the burden on medical facilities.

[1253] The processing flow will be explained below.

[1254] Step 1:

[1255] The user puts on the AI ​​glasses and turns them on. The AI ​​glasses are equipped with a camera and multiple sensors, which then become ready.

[1256] Step 2:

[1257] The device uses cameras and sensors to capture real-time health data such as the user's complexion, facial expressions, body temperature, and heart rate, while also recording current symptoms and sensations via voice input.

[1258] Step 3:

[1259] The device encrypts the image and audio data it captures. Encryption is for privacy reasons and to prevent unauthorized access to the data.

[1260] Step 4:

[1261] The device sends the encrypted data to the server, which transmits it using a secure communication protocol.

[1262] Step 5:

[1263] The server decrypts the received encrypted data and converts it into a data format for analysis.

[1264] Step 6:

[1265] The server analyzes the image data using computer vision technology, which evaluates the user's complexion, facial expressions, and physical features.

[1266] Step 7:

[1267] The server analyzes the voice data using natural language processing (NLP) algorithms, which extracts the user's complaints and subjective symptoms as text data.

[1268] Step 8:

[1269] The server references past health data and medical history and compares it with current data to more accurately assess the user's current health status.

[1270] Step 9:

[1271] The server uses a generative artificial intelligence model to analyze the data and determine the condition.

[1272] Step 10:

[1273] Based on the analysis results, the server generates appropriate advice and first-line treatment instructions to provide to the user, such as basic home care advice and instructions for more urgent situations.

[1274] Step 11:

[1275] The server sends generated advice and instructions to the terminal.

[1276] Step 12:

[1277] The advice and instructions received by the terminal are visually displayed on the display and also notified to the user audibly.

[1278] Step 13:

[1279] The user follows the displayed advice and instructions and takes the necessary steps or actions, such as drinking more fluids or taking emergency care.

[1280] Step 14:

[1281] The server records the analysis results, generated advice, and the user's reactions in a database, which will be used for future analysis and health management.

[1282] Step 15:

[1283] If the server obtains the user's consent, it will send the analysis results and important health information to the affiliated medical institution.

[1284] Step 16:

[1285] When users provide feedback after using the system, the server receives that information and uses it to improve the accuracy of the generative artificial intelligence model.

[1286] This series of steps will create a system that, through the collaboration between the AI ​​glasses and a server, will analyze the user's health data in real time and immediately provide appropriate advice and instructions for primary treatment.

[1287] Example 1

[1288] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1289] Health management has become increasingly important in recent years, and there is a particular need for systems that can monitor individual health conditions in real time. However, current health management systems have difficulty collecting and analyzing data in real time, and providing appropriate advice and treatment. Furthermore, they lack a means of linking with medical institutions and other information processing systems while adequately protecting the privacy of collected data. This poses a challenge, preventing users from receiving prompt and appropriate medical care.

[1290] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1291] In this invention, the server includes terminal means equipped with a camera and multiple sensors and capturing the user's health data in real time, means for encrypting the captured image and audio data and transmitting them to the server, server means for decrypting the received encrypted data, analyzing the data using a generative artificial intelligence model, and determining the medical condition, means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, means for transmitting the generated advice and instructions to the terminal and notifying the user visually and audibly, means for using the recorded data to link with other information processing systems, means for referencing past health data and medical history and comparing them with current health data, means for recording the analysis results and advice content in a recording device, and means for continuously training the generative artificial intelligence model to improve accuracy based on user feedback and diagnostic results from other information processing systems. This allows the user's health condition to be understood in real time, enabling prompt and appropriate medical treatment.

[1292] The "terminal means" is a device worn by a user and equipped with a camera and multiple sensors to capture health data in real time.

[1293] A "data encryption means" is a device that has the function of encrypting captured image and audio data to protect the user's privacy.

[1294] The "server means" is a system that has the function of receiving and decrypting encrypted data, and analyzes the data using a generative artificial intelligence model to determine the condition of the disease.

[1295] A "generative artificial intelligence model" is an artificial intelligence algorithm used to analyze captured health and audio data and generate appropriate advice and primary treatment instructions for the user.

[1296] The "advice generation means" is a function that automatically generates appropriate advice and instructions for primary treatment for the user based on the analysis results.

[1297] The "notification means" is a device that transmits the generated advice and instructions for primary action to the terminal and notifies the user visually and audibly.

[1298] The "linking means" is a function for linking the recorded health data of the user with other information processing systems.

[1299] "Comparison means" refers to the function of referencing past health data and medical history and comparing it with current health data.

[1300] The "recording device" is a memory device or database for storing and managing the analysis results and the generated advice content.

[1301] The "training means" is a function that continuously trains the generative artificial intelligence model based on feedback from users and diagnostic results from other information processing systems, thereby improving its accuracy.

[1302] This invention is a system that captures a user's health data in real time, analyzes it using a generative artificial intelligence model, and provides appropriate advice and primary treatment instructions based on the results. To implement this system, the following major components are required:

[1303] 1. Device and System Configuration

[1304] The wearable device (AI glasses) worn by the user is equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.). This device captures the user's health data such as complexion, facial expression, body temperature, and heart rate in real time, and records the user's subjective symptoms using a voice input function.

[1305] The device organizes the captured image and audio data, encrypts the data using an advanced encryption algorithm (e.g., AES 256), and transmits it over the Internet to a server.

[1306] The server decrypts the received encrypted data and inputs it into a generative artificial intelligence model. Analysis is performed using tools such as OpenCV (image processing) and Google Cloud Speech-to-Text (voice analysis). The generative artificial intelligence model analyzes the captured data and determines the patient's condition based on factors such as facial color, facial expression, body temperature, and heart rate.

[1307] Based on the analysis results, the generative AI model generates appropriate advice and instructions for first-line treatment for the user. For example, if cold symptoms are observed, the model generates advice such as "Drink plenty of fluids and rest," and if symptoms require urgent treatment, the model generates instructions such as "Call an ambulance immediately."

[1308] The server sends the generated advice and instructions to the terminal, which then notifies the user visually or audibly, allowing the user to take the necessary steps or actions based on the information.

[1309] Specific examples

[1310] Example 1: Early symptoms of a cold

[1311] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[1312] 2. Device: Captures the user's complexion and body temperature, and records voice input such as "I have a sore throat" or "I have a headache."

[1313] 3. Terminal: Encrypts the captured data and sends it to the server.

[1314] 4. Server: Decrypts the received data and inputs it into the generative artificial intelligence model.

[1315] 5. Server: The AI ​​model identifies the early symptoms of a cold and generates advice such as "drink plenty of fluids and get plenty of rest."

[1316] 6. Device: Advice is displayed visually and also given as an audio notification.

[1317] Example prompt sentence:

[1318] "The user's complexion is pale and their body temperature is 38.0 degrees. Their voice input has been recorded as 'sore throat' and 'headache'. Based on this information, please analyze their current health condition and generate appropriate advice."

[1319] Example 2: Acute abdominal pain

[1320] 1. User: A user who suddenly experiences severe abdominal pain in the middle of the night puts on the AI ​​glasses.

[1321] 2. Device: Capture facial color with the camera and record voice input such as "I suddenly have a stomachache."

[1322] 3. Terminal: Encrypts the captured data and sends it to the server.

[1323] 4. Server: Decrypts the data and feeds it into the generative artificial intelligence model.

[1324] 5. Server: The AI ​​model determines that the patient has acute, severe abdominal pain and is highly urgent, and generates instructions such as "Call an ambulance immediately."

[1325] 6. Terminal: Provides visual and audio instructions.

[1326] Example prompt sentence:

[1327] "We have audio data that indicates the user looks pale and has sudden, severe stomach pain. Based on this information, please analyze their health condition and generate urgent instructions."

[1328] This system is expected to improve users' self-management capabilities and reduce the burden on medical facilities by monitoring users' health conditions in real time and quickly providing appropriate advice and instructions for primary treatment.

[1329] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1330] Step 1:

[1331] The user puts on the AI ​​glasses and turns them on. The device's camera and multiple sensors capture the user's health data in real time. Input data includes the user's complexion, facial expression, body temperature, heart rate, etc. Specifically, the camera captures the user's complexion and facial expression, the temperature sensor measures body temperature, and the heart rate sensor measures heart rate. The output data is the captured image data and sensor data.

[1332] Step 2:

[1333] The terminal encrypts the captured image data and audio data. The input data is the image data and audio data captured in step 1. Specifically, the terminal encrypts the data using an encryption algorithm such as AES 256. The encrypted data becomes the output data.

[1334] Step 3:

[1335] The terminal sends the encrypted data to the server. The input data is the data encrypted in step 2. In concrete terms, the terminal sends data to the server via the Internet. The encrypted data received by the server becomes the output data.

[1336] Step 4:

[1337] The server receives and decrypts encrypted data. The input data is the encrypted data sent from the terminal. Specifically, the server decrypts the encrypted data and obtains the original image data and audio data. The decrypted data becomes the output data.

[1338] Step 5:

[1339] The server inputs the decoded data into a generative artificial intelligence model for analysis. The input data consists of decoded image data and audio data. Specifically, the server uses an image processing tool (e.g., OpenCV) to analyze facial color and facial expressions, and uses an audio analysis tool (e.g., Google Cloud Speech-to-Text) to convert the audio data into natural language. The generative artificial intelligence model uses this data to determine the patient's condition and outputs the analysis results for generating advice.

[1340] Step 6:

[1341] Based on the analysis results, the generative AI model generates appropriate advice and instructions for primary treatment. The input data is the analysis results obtained in step 5. Specifically, the model refers to past health data and medical history and compares it with current data. For example, if cold symptoms are observed, the model generates advice such as "drink plenty of fluids and take plenty of rest." The generated advice becomes the output data.

[1342] Step 7:

[1343] The server sends the generated advice and instructions to the terminal. The input data is the advice and instructions generated in step 6. In concrete terms, the server sends the generated advice and instructions to the terminal, and the terminal receives them. The data sent to the terminal becomes the output data.

[1344] Step 8:

[1345] The device notifies the user of the received advice and instructions visually or audibly. The input data is the advice and instructions received from the server. Specifically, the device's transparent display or bone conduction speaker is used to notify the user of the advice and instructions for primary treatment visually and audibly. The user can receive the notification and take the necessary action.

[1346] (Application example 1)

[1347] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1348] There is a need for a system that can monitor users' health status in real time and provide not only appropriate advice and primary treatment instructions, but also optimal diet and nutritional support. However, conventional systems have difficulty meeting these diverse needs, and one issue is the lack of specific advice on diet and nutrition.

[1349] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1350] In this invention, the server includes a terminal means equipped with a camera and a sensor for capturing health data in real time, a means for encrypting the captured image data and audio data and transmitting it to the server, a server means for analyzing the received data using a generative artificial intelligence model to determine the condition of the patient, a means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, a means for generating optimal diet and nutritional support for the user based on the analysis results, a means for transmitting the generated advice and instructions to the terminal and notifying the user, and a means for using the recorded data to connect with medical institutions. This makes it possible to comprehensively manage the user's health condition and quickly provide appropriate advice and nutritional support.

[1351] The "terminal means" is a wearable device worn by a user, and is a device equipped with a camera and sensors for capturing health data in real time.

[1352] The "means for encrypting captured image data and audio data" is a technology for encrypting image data and audio data acquired from a terminal to protect privacy and transmitting the data safely to a server.

[1353] The "server means" is a device that analyzes the received encrypted data, determines the condition using a generative artificial intelligence model, and generates advice and instructions for primary treatment for the user based on the analysis results.

[1354] A "generative artificial intelligence model" is an advanced AI algorithm that analyzes captured health data, determines medical conditions, and generates advice.

[1355] The "means for generating advice and instructions for primary treatment" is a technology that generates appropriate advice and instructions for primary treatment for the user based on the results of analysis using a generative artificial intelligence model.

[1356] The "means for generating meals and nutritional support" is a technology that generates meals and nutritional support that are optimal for the user based on the analysis results.

[1357] The "means for notifying advice and instructions" is a technique for transmitting the generated advice and instructions to the terminal and notifying the user visually or audibly.

[1358] "Means for collaboration with medical institutions" refers to technology for sharing recorded data with medical institutions and collaborating with them.

[1359] A "database" is a data storage device for recording and storing analysis results and advice content.

[1360] "Feedback from users and diagnostic results from medical institutions" refers to information provided by users and medical institutions that contributes to the continuous training and improvement of generative artificial intelligence models.

[1361] The present invention is a system that monitors the user's health condition in real time and provides optimal advice and instructions for primary treatment, as well as appropriate diet and nutritional support. Hereinafter, specific embodiments of the present invention will be described.

[1362] System Configuration

[1363] 1. Terminal means

[1364] The wearable device is equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.). The device captures real-time health data and collects image and audio data. Using the voice input function, users can record their current symptoms and feelings by voice.

[1365] 2. Data Encryption and Transmission

[1366] The device organizes the captured image and audio data in real time and encrypts them for privacy reasons (using a data encryption library such as OpenSSL), then sends the encrypted data to a cloud server.

[1367] 3. Server Means

[1368] The cloud server decrypts the received encrypted data and inputs it into a generative artificial intelligence model (e.g., GPT-4). The model analyzes facial color, facial expressions, and physical features from the image data, and uses natural language processing to extract the user's complaints and subjective symptoms from the voice data. It also references past health data and medical history, and compares this with current data to determine the condition of the patient.

[1369] 4. Advice Generation

[1370] Based on the analysis results, the generative AI model generates advice and instructions for first-line treatment for the user. Furthermore, based on the analysis results, it also generates optimal dietary and nutritional support for the user. For example, if the user shows early symptoms of a cold, it generates advice such as "It would be good to eat a diet rich in vitamin C."

[1371] 5. Notice and Enforcement

[1372] The generated advice and instructions are sent from the server to the device. The device displays them visually and also notifies the user by voice. The user can follow the displayed advice and instructions and take the necessary measures or actions. The corresponding ingredients are also displayed preferentially on the food delivery selection screen.

[1373] 6. Collaboration with medical institutions

[1374] The recorded data will be used to connect with medical institutions as needed, allowing medical institutions to refer to the user's health data and provide appropriate diagnosis and treatment.

[1375] Specific examples

[1376] Example 1: Suggested diet for preventing colds

[1377] User: A user who feels a slight cold puts on the smart glasses.

[1378] Device: Captures heart rate and facial color, and records voice messages such as "My throat has been a bit sore lately."

[1379] Server: Analyzes the captured data and determines whether it is an early symptom of a cold.

[1380] Server: The advice is to "consider a diet that includes citrus fruits and vegetables that are rich in vitamin C."

[1381] Device: Advice is displayed and voice notification is given. The relevant ingredients are also displayed preferentially on the food delivery selection screen.

[1382] Prompt Sentence Examples

[1383] "My throat has been a bit sore lately. What foods would be good to prevent a cold?"

[1384] In this way, the system of the present invention comprehensively manages the user's health condition and quickly provides appropriate advice and nutritional support.

[1385] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1386] Step 1:

[1387] The user puts on the smart glasses (terminal means) and activates the device. Data is collected as input from cameras, temperature sensors, heart rate sensors, etc. to capture the user's health status in real time. The terminal acquires biometric information (complexion, heart rate, body temperature, etc.) from these sensors and voice input, and organizes it as primary data.

[1388] Step 2:

[1389] Encrypt the organized primary data. The device uses a data encryption library (e.g., OpenSSL) to encrypt the collected health data for privacy purposes. It receives the primary data as input and generates encrypted data as output.

[1390] Step 3:

[1391] The encrypted data is sent to the cloud server. The device sends the encrypted data to the server via the Internet. The device receives the encrypted data as input and sends the data to the cloud server as output.

[1392] Step 4:

[1393] The cloud server decrypts the received encrypted data. When the server receives the data, it uses a decryption library to decrypt it. It takes the encrypted data as input and gets the decrypted data as output.

[1394] Step 5:

[1395] The decoded data is input into a generative AI model for analysis. The server uses a generative AI model (e.g., GPT-4) to receive the decoded data as input, analyze facial color, facial expressions, and physical features from the image data, and extract the user's complaints and subjective symptoms from the voice data using natural language processing. The output is a diagnosis of the disease condition and the user's current health condition.

[1396] Step 6:

[1397] Based on the analysis results, the server generates advice, initial treatment instructions, and optimal diet and nutritional support recommendations. Utilizing a generative AI model, the server receives the condition assessment results as input and generates advice, diet, and nutritional support as output. For example, if foods rich in vitamin C are needed, the server creates a specific recommendation such as, "A diet containing foods rich in vitamin C would be good."

[1398] Step 7:

[1399] The generated advice and instructions are sent to the terminal. The server sends the generated advice and instructions to the terminal via the Internet. It receives advice and instructions as input and sends them to the terminal as output.

[1400] Step 8:

[1401] The device notifies the user of the received advice or instructions. The device displays the advice or instructions on the screen or notifies the user by voice. The device receives advice or instructions from the server as input and notifies the user visually or audibly as output. For example, the device may display advice such as "It would be good to eat meals that include ingredients rich in vitamin C," and prioritize the corresponding ingredients on the food delivery selection screen.

[1402] Step 9:

[1403] The user acts according to the advice and instructions received. The user selects the necessary diet and nutritional support based on the displayed advice examples and takes action accordingly. Specifically, the user performs an action such as ordering a meal using the recommended ingredients.

[1404] Through this series of steps, the user is provided with real-time advice and dietary suggestions based on their health status.

[1405] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1406] This invention combines a system that captures a user's health data in real time and provides advice and primary treatment instructions based on the analysis results with an emotion engine that recognizes the user's emotions. To implement this system, the following main components are required:

[1407] 1. Device and System Configuration

[1408] Device: AI glasses are a wearable device worn by the user. Equipped with a camera and multiple sensors (temperature sensor, heart rate sensor, etc.), they capture emotions along with health data. They also have the ability to collect image and audio data, encrypt it, and send it to a server.

[1409] Server: Receives the captured data and analyzes the medical condition using a generative artificial intelligence model and emotion engine. Based on the analysis results, it generates advice and primary treatment instructions and sends them to the device.

[1410] Generative artificial intelligence model: An advanced AI algorithm that analyzes health data, determines medical conditions, and generates advice.

[1411] Emotion engine: An AI algorithm that recognizes emotions from the user's facial expressions and voice data and integrates that data into the analysis results.

[1412] 2. System operation explanation

[1413] The system operates as follows.

[1414] Data collection: The user puts on the AI ​​glasses and turns them on. The device's camera and sensors capture real-time health data, such as the user's complexion, facial expressions, body temperature, and heart rate. At the same time, the voice input function records the user's current symptoms and sensations, and the camera records the user's facial expressions.

[1415] Data encryption and transmission: The device organizes the captured image data, voice data, and facial expression data in real time, encrypts them to protect privacy, and then transmits the data to the server.

[1416] Data analysis: The server decrypts the received encrypted data and converts it into a data format for analysis. Image data is analyzed using computer vision technology to evaluate facial color, facial expressions, and physical characteristics. Voice data is analyzed using natural language processing (NLP) algorithms to extract the user's complaints and subjective symptoms as text data.

[1417] Emotion Recognition: The server uses an emotion engine to analyze the user's emotions from facial expressions and voice, for example, to determine whether the user is stressed or relaxed.

[1418] Data integration and analysis result generation: The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the condition.

[1419] Advice generation: Based on the analysis results, advice and instructions for first-line treatment are generated for the user. Emotional data is also taken into consideration at this time. For example, if the user is feeling stressed, advice on how to relax is added.

[1420] Notification and implementation: The server sends the generated advice and instructions to the terminal, which notifies the user visually or audibly. The user follows the displayed advice and instructions and takes the necessary steps or actions.

[1421] Specific examples

[1422] Example 1: Early symptoms of a cold

[1423] 1. User: A user who feels unwell in the morning puts on the AI ​​glasses.

[1424] 2. Device: Captures the user's complexion, body temperature, and facial expressions, and records symptoms such as "sore throat" or "headache" via voice input.

[1425] 3. Server: Analyzes the captured data and determines whether the user is showing early symptoms of a cold. At the same time, it recognizes that the user's facial expression indicates fatigue.

[1426] 4. Server: Generate the following advice: "You have a sore throat and headache, so drink plenty of fluids and get plenty of rest. You also feel fatigued, so it's recommended that you get plenty of rest."

[1427] 5. Device: Advice is displayed visually on the screen and also provided as an audio notification.

[1428] Example 2: Acute abdominal pain and stress

[1429] 1. User: Feels severe abdominal pain in the middle of the night and puts on the AI ​​glasses.

[1430] 2. Device: The camera captures the user's facial expression and facial color, and the user is given a voice command such as "I suddenly have a stomachache." The device also determines whether the user is feeling stressed based on the user's facial expression.

[1431] 3. Server: Analyzes the data, detects acute severe abdominal pain and high stress levels, and determines that the situation is urgent.

[1432] 4. Server: Generate the following advice: "Severe abdominal pain has been detected, so please call an ambulance immediately. High stress levels have also been detected, so please adopt relaxation techniques once safety is assured."

[1433] 5. Device: Advice is displayed on the screen and also notified by voice.

[1434] In this way, the present invention, which combines an emotion engine, is a system that provides more accurate and personalized medical support while taking into account the user's emotional state. This is expected to improve users' self-management abilities, reduce unnecessary medical expenses, and ease the burden on medical facilities.

[1435] The processing flow will be explained below.

[1436] Step 1:

[1437] The user puts on the AI ​​glasses and turns them on. The AI ​​glasses are equipped with a camera and multiple sensors, which then become ready.

[1438] Step 2:

[1439] The device uses cameras and sensors to capture real-time health data such as the user's complexion, facial expressions, body temperature, and heart rate, while also recording current symptoms and sensations via voice input.

[1440] Step 3:

[1441] The device encrypts the image, voice, and facial expression data it captures to protect privacy and prevent unauthorized access to the data.

[1442] Step 4:

[1443] The device sends the encrypted data to the server, which transmits it using a secure communication protocol.

[1444] Step 5:

[1445] The server decrypts the received encrypted data and converts it into a data format for analysis.

[1446] Step 6:

[1447] The server analyzes the image data using computer vision technology, which evaluates the user's complexion, facial expressions, and physical features.

[1448] Step 7:

[1449] The server analyzes the voice data using natural language processing (NLP) algorithms, which extracts the user's complaints and subjective symptoms as text data.

[1450] Step 8:

[1451] The server uses an emotion engine to analyze the user's emotions from facial expressions and voice. Emotion recognition determines whether the user is stressed, relaxed, or in other emotional states.

[1452] Step 9:

[1453] The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the condition.

[1454] Step 10:

[1455] Based on the analysis results, the server generates appropriate advice and instructions for first-line treatment to provide to the user. Emotional data is also taken into account in this process. For example, if the user is feeling stressed, advice on how to relax will be added.

[1456] Step 11:

[1457] The server sends generated advice and instructions to the terminal.

[1458] Step 12:

[1459] The advice and instructions received by the terminal are visually displayed on the display and also notified to the user audibly.

[1460] Step 13:

[1461] The user follows the displayed advice and instructions and takes the necessary steps or actions, such as drinking more fluids or taking emergency care.

[1462] Step 14:

[1463] The server records the analysis results, generated advice, and the user's reactions in a database, which will be used for future analysis and health management.

[1464] Step 15:

[1465] If the server obtains the user's consent, it will send the analysis results and important health information to the affiliated medical institution.

[1466] Step 16:

[1467] When users provide feedback after using the system, the server receives that information and uses it to improve the accuracy of the generative artificial intelligence model.

[1468] This series of steps will create a system that, through the collaboration between the AI ​​glasses and a server, will analyze the user's health and emotional data in real time and immediately provide appropriate advice and instructions for primary treatment.

[1469] Example 2

[1470] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1471] Conventional health management systems lack analysis and advice that take into account the user's emotional state, making it difficult to accurately assess health status and provide appropriate treatment. They also lack real-time data capture and analysis, making it difficult to respond quickly to user conditions. Furthermore, they lack the ability to compare health data with past data or continuously train generative artificial intelligence algorithms, making it difficult to improve the system's accuracy.

[1472] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information terminal equipped with an image capture device and a biometric sensor for capturing health data and emotional data in real time, an information terminal that encrypts the captured image data and audio data and transmits them to a central processing unit, a central processing unit that analyzes the received data using a generative artificial intelligence algorithm to determine the user's health status, a central processing unit that generates appropriate advice and instructions for primary treatment for the user based on the analysis results, an information terminal that transmits the generated advice and instructions to the information terminal and notifies the user, and a data management device used to link the recorded data with a medical institution. This enables accurate health assessments that take the user's emotional state into consideration, provision of appropriate advice, and prompt real-time responses. Furthermore, the accuracy of the system can be improved by comparing with past health data and continuously training the generative artificial intelligence algorithm.

[1473] "Image capture device" refers to a device such as a camera for capturing a user's facial expression.

[1474] A "biometric sensor" refers to a sensor used to measure a user's health data, such as body temperature or heart rate.

[1475] "Information terminal" refers to a device that captures health and emotional data, encrypts it, and transmits it to a central processing unit.

[1476] "Central Processing Unit" refers to the electronic device that analyzes the received data, determines the health status, and generates appropriate advice and instructions for primary treatment.

[1477] "Generative artificial intelligence algorithm" refers to an advanced computational method that analyzes health and emotional data to determine medical conditions and generate advice.

[1478] "Data management device" refers to the system used to store recorded data and to communicate with medical institutions as needed.

[1479] "Real-time" refers to a processing format in which data is captured immediately and analyzed and notified.

[1480] "Encryption" refers to the technology of converting data to protect the privacy of that information, making it unreadable to third parties.

[1481] "Emotional data" refers to information about the emotional state obtained from the user's facial expressions and voice.

[1482] "Health status" refers to the user's physical and psychological state as assessed by body temperature, heart rate, facial expression, etc.

[1483] "Advice" refers to instructions or recommendations provided to the user based on the analysis results.

[1484] "First steps" refers to instructions that indicate the initial response that a user should take in an emergency.

[1485] This invention is a system that captures and analyzes a user's health and emotional state in real time and provides advice and primary treatment instructions based on the analysis results. This system consists of an information terminal worn by the user, a central processing unit that analyzes the data, a means for providing the generated advice, and a device that manages the recorded data and connects with medical institutions.

[1486] Hardware and software used

[1487] Information terminal: A wearable device worn by the user, equipped with a camera (image capture device) and multiple biometric sensors (temperature sensor, heart rate sensor, etc.). This terminal captures health and emotional data in real time, encrypts the data, and transmits it to a central processing unit.

[1488] Central Processing Unit: A server on the cloud or a local high-performance computing device that analyzes data using generative artificial intelligence algorithms and sentiment analysis engines.

[1489] Data management device: A cloud storage or on-premise database system that records and stores analysis results, advice, past health data, and medical history data.

[1490] Data processing and calculation

[1491] Once worn by the user, the device begins collecting data using its camera and sensors. The camera captures the user's facial expression and complexion, the temperature sensor measures body temperature, the heart rate sensor measures heart rate, and the voice input function records the user's description of symptoms. This collected data is encrypted in real time and sent to a server.

[1492] The server decrypts the received encrypted data and converts it into the appropriate data format. Image data is analyzed using computer vision technology to evaluate the user's facial color and facial expressions. Voice data is analyzed using natural language processing (NLP) algorithms to extract the user's complaints and subjective symptoms as text data. Furthermore, an emotion analysis engine analyzes the user's emotional state from facial expressions and voice.

[1493] The generative AI algorithm analyzes the integrated health and emotional data to assess the user's health status. Based on this assessment, advice and primary treatment instructions are generated. The accuracy of the analysis results is improved by comparing them with past health data and medical history to ensure the appropriateness of advice for the user.

[1494] The generated advice and instructions are then encrypted and sent to the terminal, where the user can receive the information visually or audibly.

[1495] Specific examples

[1496] Example 1: Early symptoms of a cold

[1497] 1. The user feels unwell in the morning and puts on the AI ​​glasses.

[1498] 2. The device captures the user's complexion, body temperature, and facial expressions, and records symptoms such as "sore throat" and "headache" through voice input.

[1499] 3. The server analyzes the captured data and determines whether the user is showing early symptoms of a cold. At the same time, the server recognizes the user's fatigue from their facial expression.

[1500] 4. The server generates the advice, "You have a sore throat and a headache, so drink plenty of fluids and rest. You also feel fatigued, so it's recommended that you get plenty of rest."

[1501] 5. The device will display advice on the screen and also provide audio notifications.

[1502] Example 2: Acute abdominal pain and stress

[1503] 1. The user experiences severe stomach pain in the middle of the night and puts on the AI ​​glasses.

[1504] 2. The device captures facial expression and facial color with a camera and records the user's voice input, such as "I suddenly have a stomachache." It also determines whether the user is feeling stressed based on their facial expression.

[1505] 3. The server analyzes the data, detects acute severe abdominal pain and high stress levels, and determines that the condition is urgent.

[1506] 4. The server generates the advice, "You are experiencing severe abdominal pain, so please call an ambulance immediately. Also, high stress levels have been detected, so please adopt relaxation techniques once safety is assured."

[1507] 5. The device will display advice on the screen and also provide audio notifications.

[1508] Prompt Sentence Examples

[1509] "My temperature is rising, my throat is sore, and I also have a slight headache this morning."

[1510] "I suddenly get a very strong stomach ache."

[1511] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1512] Step 1: Booting the device

[1513] The user puts on the AI ​​glasses and turns on the device. Once the device is successfully booted and ready, the camera and sensors are initialized and ready to collect data. The input is the user's actions, and the output is the device's ready state.

[1514] Specific example of operation: When the user presses the power button on the AI ​​Glasses, the device starts up with a startup sound and displays a notification that it is ready.

[1515] Step 2: Data collection

[1516] The device activates the camera and sensors to capture facial color, facial expressions, body temperature, and heart rate. The voice input function is enabled to record what the user says. The input is the user's health status and voice, and the output is these data.

[1517] Specific operation example: The camera captures the user's face, the temperature sensor measures body temperature, and the heart rate sensor measures heart rate. When the user says something like "I have a headache," the voice is recorded.

[1518] Step 3: Encrypt and send data

[1519] The device encrypts the captured image, audio, and biometric data using the AES-256 encryption algorithm. The encrypted data is sent to the server. The input is raw data, and the output is encrypted data.

[1520] Specific operation example: Collected data (e.g., facial color images, body temperature, heart rate, and voice data) is encrypted and sent to a server via high-speed wireless communication.

[1521] Step 4: Data analysis

[1522] The server decrypts the received encrypted data and converts it into the appropriate data format. Image data is analyzed using computer vision technology to evaluate the user's facial color and facial expression. Voice data is analyzed using natural language processing (NLP) technology to extract the user's complaints and subjective symptoms as text data. The input is encrypted data, and the output is the analysis results.

[1523] Specific example: Analyze image data using OpenCV and extract text from audio data using the Google Cloud Speech-to-Text API.

[1524] Step 5: Emotion Recognition

[1525] The server uses an emotion engine to analyze the user's emotions from facial expressions and voice, for example, to determine stress levels and relaxation states. The input is the analyzed health data and voice data, and the output is an assessment of the user's emotional state.

[1526] Specific example of operation: Using the Emotion AI API to analyze facial expression data and evaluate it as "high stress level."

[1527] Step 6: Integrate and score the data

[1528] The server integrates the acquired health data, emotional data, past health data, and medical history, and uses a generative artificial intelligence algorithm to assess health status. Each data point is assigned a score to determine overall health status. The input is the integrated data, and the output is an overall health score.

[1529] Specific example of how it works: Using TensorFlow to score health status using a generative AI model.

[1530] Step 7: Advice Generation

[1531] The server generates advice and instructions for first-line treatment for the user based on the analysis results. It also takes into account emotional data and adds appropriate recommendations. The input is the analysis results of health data and emotional data, and the output is advice and instructions for first-line treatment.

[1532] Example of specific operation: Advice such as "Drink plenty of fluids and get plenty of rest" is generated in text format.

[1533] Step 8: Notification and enforcement

[1534] The server encrypts the generated advice and instructions and sends them to the terminal. The terminal notifies the user of this information visually or audibly. The input is the encrypted advice data, and the output is the notification to the user.

[1535] Specific example of operation: The AI ​​glasses display will show "Drink plenty of fluids and get plenty of rest," and a voice notification will also be given.

[1536] This detailed processing step allows the user to receive an accurate assessment of their health status in real time and receive appropriate advice and primary treatment instructions.

[1537] (Application example 2)

[1538] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1539] Current security services lack a means to monitor the health and emotional state of security guards in real time, making it difficult to respond quickly when abnormalities in the guard's health or mental state occur. In addition, systems for accurately detecting suspicious individuals and abnormal behavior are still insufficient. This places a heavy burden on security guards and risks reducing overall safety. To address these issues, the present invention aims to provide a comprehensive system that captures health and emotional data in real time and provides appropriate advice and first-line treatment instructions.

[1540] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means equipped with a camera and sensor for capturing health data and emotional data in real time, means for encrypting the captured image data and audio data and transmitting them to the server, server means for analyzing the received data using a generative artificial intelligence model and an emotion engine to determine the medical condition and emotional state, means for generating appropriate advice and instructions for primary treatment for the user based on the analysis results, means for transmitting the generated advice and instructions to the terminal and notifying the user, means for using the recorded data to apply to security management, and means for detecting anomalies and taking emergency measures. This makes it possible to monitor the health and emotional state of security guards in real time and to take appropriate action immediately if an abnormality occurs.

[1541] "Terminal means" refers to a device equipped with a camera and sensors for capturing health and emotional data in real time.

[1542] "Encryption means" refers to the technology and method for encrypting captured image and audio data for secure transmission.

[1543] "Server means" is a computer system that analyzes the received data, determines the medical and emotional state, and generates necessary advice and primary treatment instructions.

[1544] A "generative artificial intelligence model" is a machine learning algorithm that analyzes health and emotional data to generate appropriate advice and first-line treatment instructions.

[1545] An "emotion engine" is an algorithm and technology for analyzing a user's facial expressions and voice data to determine their emotional state.

[1546] The "notification means" is a method for transmitting the generated advice and instructions to the terminal and notifying the user visually or audibly.

[1547] "Security management measures" are techniques and methods for utilizing recorded data for security management.

[1548] "Abnormality detection means" refers to techniques and methods for detecting abnormalities in the health or emotional state of security guards.

[1549] "Emergency response measures" are techniques and methods for taking immediate and appropriate action when an abnormality is detected.

[1550] This invention is a system that captures health and emotional data of security guards in real time and provides appropriate advice and instructions for primary treatment based on the analysis results. This system is composed of terminal means, encryption means, server means, generative artificial intelligence model, emotion engine, notification means, security management means, anomaly detection means, and emergency response means.

[1551] 1. Hardware and Software Used

[1552] Terminal means

[1553] A wearable device worn by a user, equipped with a camera and sensors (such as a temperature sensor and a heart rate sensor) to capture health and emotional data. Examples include smart glasses and head-mounted displays.

[1554] Encryption method

[1555] Used to encrypt the captured data. The encryption technology used is a cryptographic library such as OpenSSL.

[1556] Server Means

[1557] It is a computer system that analyzes the received data and determines the patient's medical condition and emotional state. It mainly uses server-side frameworks such as Flask or Django as its software.

[1558] Generative AI model and emotion engine

[1559] Health and emotion data are analyzed using deep learning frameworks such as TensorFlow and Keras, as well as Hugging Face's Transformers.

[1560] Notification means

[1561] This is a mechanism for transmitting advice and instructions for primary measures generated by the server to the terminal means and notifying the user visually and audibly.

[1562] Security Control Measures

[1563] It includes techniques and methods for applying recorded data to security management.

[1564] Anomaly detection and emergency response measures

[1565] This includes techniques and methods for detecting abnormalities in the health or emotional state of security guards and for immediately taking appropriate action.

[1566] 2. Data processing and calculation

[1567] Data collection

[1568] The user wears a device, and cameras and sensors capture data in real time. The collected data is then organized in real time and sent to a server using encryption technology.

[1569] Data analysis

[1570] The server decodes the received data and analyzes it using computer vision techniques and natural language processing algorithms, mobilizing generative artificial intelligence models and emotion engines to determine health and emotional states.

[1571] Advice Generation and Notification

[1572] Based on the analysis results, the server generates the necessary advice and instructions for the first action, which are sent to the terminal means and notified to the user visually and audibly.

[1573] 3. Specific Examples

[1574] Example 1: Suspicious person detection

[1575] If a user (security guard) detects a suspicious person while patrolling, the AI ​​glasses capture the scene and send it to the server, which analyzes it and provides advice such as, "A suspicious person has been spotted. Ensure safety and contact the police."

[1576] Example 2: Detecting health abnormalities

[1577] If a security guard experiences chest pain while on patrol, the AI ​​glasses will capture the image and send it to a server, which will then analyze it and provide advice such as, "Chest pain detected. Please rest immediately."

[1578] Prompt Sentence Examples

[1579] Here are some example prompts to input to the generative AI model:

[1580] Suspicious person detection

[1581] Security Guard A detects a suspicious person while on patrol. Please analyze the data captured by the AI ​​Glasses and provide appropriate countermeasures.

[1582] Security guard's health condition

[1583] Security Guard B experiences chest pains while on patrol. Please provide appropriate first aid advice from captured data.

[1584] As described above, by using the system of the present invention, it is possible to monitor the health and emotional state of security guards in real time, and to take appropriate action immediately if an abnormality occurs.

[1585] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1586] Step 1: Data collection

[1587] The user puts on the terminal (wearable device) and turns it on. The camera and sensors on the terminal capture the user's health and emotional data, such as complexion, facial expression, body temperature, and heart rate, in real time. At the same time, the voice input function records the user's current symptoms and sensations. The input is camera footage, audio, and health data from the sensors, and the output is the captured data.

[1588] Step 2: Data Encryption

[1589] The terminal means organizes the captured image data, voice data, and facial expression data and encrypts them for privacy protection. Specifically, the data is encrypted using the OpenSSL library. The input is the captured data, and the output is the encrypted data.

[1590] Step 3: Send data

[1591] Send encrypted data to a server using a secure communication protocol (e.g., HTTPS). The input is the encrypted data, and the output is a notification to the server that the data has been sent.

[1592] Step 4: Receive and decrypt data

[1593] The server decrypts the encrypted data it receives and converts it into a data format for analysis. Specifically, it uses the OpenSSL library to decrypt the data. The input is the encrypted data received by the server, and the output is the decrypted data.

[1594] Step 5: Data analysis

[1595] The server analyzes image data and audio data. Image data is analyzed for facial color and facial expressions using OpenCV, and audio data is converted to text data using the Transformers library. Health data is analyzed using TensorFlow and Keras. The input is the decoded data, and the output is the analysis result.

[1596] Step 6: Emotion Recognition

[1597] The server uses an emotion engine to analyze emotions from facial expressions and voice data, for example, to determine whether the user is stressed or relaxed. The input is the analyzed facial expression and voice data, and the output is the emotion recognition result.

[1598] Step 7: Data integration and analysis

[1599] The server integrates health data, emotional data, and past health data and medical history, and uses a generative artificial intelligence model to determine the patient's condition. The input is the analyzed health data and emotional data, and the output is the integrated analysis result.

[1600] Step 8: Advice Generation

[1601] Based on the analysis results, the server generates advice and instructions for the user, taking into account emotional data. For example, if the user feels stressed, it adds advice on how to relax. The input is the integrated analysis results, and the output is the generated advice and instructions.

[1602] Step 9: Advice Notification

[1603] The server sends the generated advice and instructions to the terminal means, which then notifies the user of the advice and instructions visually or audibly. Specifically, the terminal means uses a display or speaker. The input is the generated advice and instructions, and the output is the notification to the user.

[1604] Step 10: Security Management and Emergency Response

[1605] The server applies the recorded data to security management, detects suspicious individuals, and takes emergency action when a security guard has a health abnormality. The input is user feedback and security data after notification, and the output is improvements to security management and the results of emergency responses.

[1606] These are the specific processing steps for the system that realizes this application example. It is possible to monitor the user's health condition and emotional state in real time and take appropriate action immediately if an abnormality occurs.

[1607] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1608] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1609] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1610] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1611] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1612] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1613] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1614] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1615] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1616] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1617] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1618] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1619] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1620] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1621] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1622] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1623] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1624] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1625] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1626] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustratio...

Claims

1. a terminal means equipped with a camera and sensors for capturing health data in real time; means for encrypting the captured image data and audio data and transmitting the encrypted data to a server; a server means for analyzing the received data using a generative artificial intelligence model to determine the condition of the patient; means for generating appropriate advice and instructions for primary treatment to the user based on the analysis results; means for transmitting the generated advice and instructions to a terminal and notifying a user; and means for using the recorded data to communicate with a medical institution.

2. 2. The system according to claim 1, further comprising: means for referencing past health data and medical history and comparing the past health data with current health data; and means for recording the analysis results and advice contents in a database.

3. The system of claim 1 , further comprising means for continuously training the generative artificial intelligence model to improve its accuracy based on feedback from users and diagnostic results from medical institutions.

Images