System

Abstract

A system is provided.SOLUTION: A system including means for collecting conversation data of an elderly person, means for collecting walking data of the elderly person, means for encrypting the conversation data and the walking data and transmitting the encrypted conversation data and walking data to a cloud server, means for decrypting the received conversation data and walking data and analyzing the decrypted conversation data and walking data using an artificial intelligence model in the cloud server, means for digitizing an analysis result and making a determination on the basis of a reference for abnormality detection, and means for notifying a user and a family of the user of a determination result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The risk of dementia in the elderly increases with age, but dementia often does not show noticeable symptoms in the early stages, making early detection difficult. In particular, even if individuals or their families notice signs of dementia, they often are reluctant to acknowledge it. For this reason, there is a need for a means to objectively evaluate the signs of dementia and recognize them early. Furthermore, early detection can slow the progression of dementia by promptly implementing appropriate medical intervention. The present invention aims to solve these problems. [Means for solving the problem]

[0005] The present invention provides a system for collecting and analyzing conversation and walking data from elderly people. Specifically, the system includes a means for collecting conversation and walking data from elderly people using a smartphone and a means for encrypting and transmitting the data to a cloud server. The cloud server is equipped with a means for decrypting the received data and analyzing it using an artificial intelligence model. The system also includes a means for quantifying the analysis results, making a judgment based on anomaly detection criteria, and notifying the user and their family of the results. This allows for objective evaluation of dementia symptoms and early recognition, enabling appropriate diagnosis and prompt medical intervention.

[0006] The term "elderly" generally refers to people in the age group of 60 or older.

[0007] "Speech Data" refers to recordings of a user's speech and linguistic features obtained from that speech.

[0008] "Walking data" refers to data measured from a user's movements and movement patterns while walking.

[0009] A "cloud server" refers to a remote server used to store and analyze data over the Internet.

[0010] An "artificial intelligence model" refers to a software algorithm that learns from large amounts of data and performs specific tasks automatically.

[0011] "Analysis results" refers to the output results after analyzing collected data using an artificial intelligence model.

[0012] "Quantification" refers to converting the analysis results into quantitative numbers and making them into a format that allows for comparative evaluation.

[0013] "Anomaly detection criteria" refers to thresholds or indicators for determining whether something is normal or abnormal.

[0014] "Means of Notification" refers to electronic means for communicating important information to users and their families. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system that collects and analyzes conversation and walking data from elderly people to detect early signs of dementia. The system includes a smartphone, a cloud server, and users and their families.

[0037] System configuration

[0038] Smartphone (device)

[0039] 1. Data Collection Methods

[0040] The smartphone is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). This allows the smartphone to periodically collect the elderly person's conversation and walking data in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking pattern (step length, walking speed, balance fluctuations, etc.) and save the data.

[0041] 2. Data Encryption Methods

[0042] The collected speech and walking data is encrypted within the device and prepared for transmission to a cloud server. This encryption process protects the privacy and security of the data.

[0043] 3. Data Transmission Method

[0044] The encrypted data is then sent over the internet to a cloud server using the HTTP or HTTPS protocol.

[0045] Cloud Server (Server)

[0046] 1. Data Receiving Method

[0047] The cloud server receives the encrypted data sent from the device, decrypts it, and stores it in temporary storage to prepare for analysis.

[0048] 2. Data analysis methods

[0049] The cloud server inputs the decoded conversation and walking data into an artificial intelligence model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[0050] 3. Methods for quantifying analysis results

[0051] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[0052] 4. Means of notification

[0053] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[0054] User and his / her family (User)

[0055] 1. Check the results

[0056] Users and their family members can check the analysis results by opening a dedicated smartphone app, which will display a detailed message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0057] 2. Collaboration with medical institutions

[0058] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[0059] Specific examples

[0060] Example 1: Acquiring and analyzing audio data

[0061] If elderly person A has a smartphone, the voice recognition module automatically records A's conversations. For example, if A says, "What was that thing that happened yesterday?", the audio is recorded and sent to a cloud server. The server analyzes this data and determines whether there are any signs of dementia by detecting the frequent occurrence of "this, this, that."

[0062] Example 2: Gait data collection and analysis

[0063] Similarly, when elderly person A goes for a walk every day, the built-in sensor in his smartphone records his walking pattern. For example, data such as extremely short strides or unstable walking speed is collected. This data is also analyzed by the cloud server to evaluate the stability of his walking.

[0064] In this way, the system of the present invention supports early detection of dementia in the elderly and appropriate medical intervention, providing important information to users and their families.

[0065] The processing flow will be explained below.

[0066] Step 1: Data Collection (Device)

[0067] The device will run a voice recognition module in the background and record the user's voice conversations.

[0068] Specific operation: The device will record audio every 30 seconds and save the recording data to the internal storage.

[0069] The device uses built-in sensors (accelerometer, gyro sensor, etc.) to record the user's walking data in real time.

[0070] Specific operation: The device captures the user's movements (e.g., walking, shaking) and saves the data in the specified format on the internal storage.

[0071] Step 2: Data encryption (device)

[0072] The device encrypts the collected conversation and walking data.

[0073] Specific operation: Encrypt the data using an encryption algorithm (e.g., AES-256) and store it in a temporary buffer.

[0074] Step 3: Send data (terminal)

[0075] The device transmits the encrypted conversation data and walking data to a cloud server.

[0076] Specific operation: Uploads encrypted data to a specific endpoint on a cloud server using HTTP or HTTPS protocols.

[0077] Step 4: Data reception and decryption (server)

[0078] The server receives the encrypted data sent from the terminal.

[0079] Specific operation: The server receives the data through the firewall and stores it in temporary storage.

[0080] The server decrypts the received data.

[0081] What it does: It uses a data decoding algorithm to return the data to its original form and stores it in a database for analysis.

[0082] Step 5: Data Analysis (Server)

[0083] The server analyzes the conversation data using natural language processing (NLP) algorithms.

[0084] Specific operation: The server converts the voice data into text data and detects frequent patterns of "this, that, that," jumps in vocabulary, and repetitions.

[0085] The server analyzes the walking data using a data analysis algorithm.

[0086] Specific operation: The server evaluates walking stability, stride length variations, speed changes, etc., and detects abnormal patterns.

[0087] Step 6: Quantifying the analysis results (server)

[0088] The server quantifies the analysis results and evaluates them based on anomaly detection criteria.

[0089] Specific operation: The server statistically processes the analysis results and converts them into numbers based on a scoring system.

[0090] Step 7: Notification Generation and Sending (Server)

[0091] The server generates notifications to the user and their family based on the analysis results.

[0092] Specific behavior: The server generates a notification message and prepares it to be sent in the form of an email or push notification.

[0093] The server generates and sends notifications to the user and their family members.

[0094] Specific operation: The server sends a notification message using the SMTP protocol and notification service.

[0095] Step 8: Check the results (user)

[0096] Users and their family members open the app to check the analysis results.

[0097] What happens: The user taps on the notification on their smartphone to view detailed analysis results and recommendations.

[0098] Step 9: Collaboration with medical institutions (user)

[0099] Users and their families make medical appointments as needed.

[0100] Specific behavior: Click on a link within the app to access the medical institution's appointment page and complete the appointment.

[0101] Step 10: Continuous data collection (device)

[0102] Your smartphone will continue to collect data and analyze it periodically.

[0103] Specific operations: The steps described above, from data collection to transmission, are continuously carried out to conduct long-term monitoring.

[0104] Example 1

[0105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] Early detection of dementia in the elderly is extremely important in medical and nursing care settings. However, conventional methods can only collect limited data, making it difficult to detect abnormalities early. Furthermore, the security of collected data and protection of privacy are also important issues. Effective solutions to resolve these issues are needed.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0108] In this invention, the server includes a means for collecting conversation data from the elderly, a means for collecting walking data from the elderly, a means for encrypting the conversation data and walking data and transmitting them to a cloud server, a means for decrypting the received conversation data and walking data and analyzing them using a generative AI model in the cloud server, a means for quantifying the analysis results and making a judgment based on anomaly detection criteria, and a means for notifying the judgment results to the user and the user's family. This enables early detection of signs of dementia in the elderly and appropriate medical intervention. Furthermore, data encryption and processing on the cloud server enable data analysis in a secure, privacy-protected environment.

[0109] "Conversation data" is information that records the voices of elderly people and is saved in the form of text or audio files.

[0110] "Gait data" is information that records information about an elderly person's walking pattern, including data such as stride length, walking speed, and balance fluctuations.

[0111] "Encryption" is a technology that converts data to protect the content of the data being transmitted, making it difficult for third parties to access.

[0112] A "cloud server" is a remote computing resource that stores and processes data over the Internet.

[0113] A "generative AI model" is an artificial intelligence algorithm that learns for a specific task and analyzes data and makes predictions.

[0114] "Analysis" is the process of examining collected data, extracting information, and drawing useful conclusions.

[0115] "Quantification" is a means of quantitatively expressing analytical results to facilitate evaluation and comparison.

[0116] "Anomaly detection" is the process of identifying unusual patterns in data analysis and identifying potential problems.

[0117] "Notification" refers to the act of informing the user and their family of the analysis results and judgment results, and is done by means of email, push notification, etc.

[0118] This invention is a system that collects and analyzes conversation data and walking data from elderly people to detect early signs of dementia. This system includes a smartphone, a cloud server, and the user and their family. The following describes how to specifically implement this system.

[0119] Use of smartphones (devices)

[0120] Data collection methods

[0121] The device is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). Using this function, the device periodically collects user conversation and walking data in the background. For example, if a user says, "What was the weather like yesterday?", this voice will be recorded. Meanwhile, as walking data, the device records the user's stride length, walking speed, and balance fluctuations as they walk.

[0122] Data encryption methods

[0123] The collected data is encrypted on the device using the AES-256 algorithm. This encryption process protects the privacy and security of the data. The encrypted data is then prepared for network transmission.

[0124] Data transmission method

[0125] The encrypted data is sent over the Internet to a cloud server using the HTTPS protocol, ensuring secure data transmission.

[0126] Use of cloud servers

[0127] Data reception and decoding means

[0128] The server receives the encrypted data sent by the device and decrypts it using the AES-256 algorithm, storing the decrypted data in temporary storage and preparing it for analysis.

[0129] Data Analysis Methods

[0130] The server then feeds the decoded conversation data into natural language processing (NLP) algorithms to detect certain patterns (e.g., frequent use of "this, this, that," repetition, and jumps in content). Data analysis algorithms are also applied to walking data to evaluate gait stability, stride length variability, and speed fluctuations. For example, if a user frequently says, "What was that thing that happened yesterday?", the NLP algorithms will analyze this pattern.

[0131] Methods for quantifying analysis results

[0132] The analysis results are quantified and evaluated based on anomaly detection criteria. For example, abnormalities in speech patterns or fluctuations in gait stability are output as numerical values.

[0133] Notification means

[0134] The server generates notifications based on the analysis results and sends them to the user and their family. Notifications can be sent via email or push notification. For example, they could include a message like, "Your father's recent conversation patterns have been unusual. We recommend that you see a doctor."

[0135] Use by the User and his / her Family (User)

[0136] Checking the results

[0137] Users and their families can check the analysis results using a dedicated smartphone app. Detailed messages and graphs are displayed on the app screen. For example, a message might read, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0138] Collaboration with medical institutions

[0139] Users and their family members who receive the notification can immediately make an appointment with a medical institution by clicking a link within the app, such as "Click here to make an appointment with a local specialist."

[0140] Prompt Sentence Examples

[0141] "Please explain a system that collects and analyzes conversation and walking data from elderly people to detect early signs of dementia."

[0142] In this way, the system supports early detection of dementia in the elderly and appropriate medical intervention.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1:

[0145] Data collection

[0146] Subject: Device

[0147] Specific operation: The device activates the voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.) to collect the user's conversation data and walking data.

[0148] Input: User voice and walking movements

[0149] Data processing: Audio data is recorded in local storage, and walking data is obtained from sensors.

[0150] Output: Collected voice and walking data is stored in the device's local storage.

[0151] Step 2:

[0152] Data Encryption

[0153] Subject: Device

[0154] Specific operation: The device encrypts collected speech and walking data using the AES-256 algorithm.

[0155] Input: Audio data and walking data stored in local storage

[0156] Data processing: Data is converted using an encryption algorithm to make it unreadable to third parties.

[0157] Output: Encrypted audio and gait data is generated.

[0158] Step 3:

[0159] Data transmission

[0160] Subject: Device

[0161] Specific operation: The device sends encrypted data to the cloud server using the HTTPS protocol.

[0162] Input: Encrypted voice and gait data

[0163] Data processing: Sending data over the internet.

[0164] Output: The encrypted data reaches the cloud server.

[0165] Step 4:

[0166] Data Reception and Decryption

[0167] Subject: Server

[0168] What happens: The server receives the encrypted data and decrypts it using the AES-256 algorithm.

[0169] Input: Encrypted data sent from the device

[0170] Data processing: The data is converted using a decoding algorithm to obtain the original voice and walking data.

[0171] Output: The decoded voice data and walking data are stored in the server storage.

[0172] Step 5:

[0173] Data analysis

[0174] Subject: Server

[0175] Specific operation: The server inputs the decoded data into a generative AI model and natural language processing (NLP) algorithm to perform pattern analysis of the speech data and stability analysis of the walking data.

[0176] Input: Decoded audio data and walking data

[0177] Data processing: Speech data is analyzed using NLP algorithms to identify specific language patterns, while walking data is analyzed using data analysis algorithms to assess walking stability, stride length variability, and speed fluctuations.

[0178] Output: Analysis results of conversation data and walking data are generated.

[0179] Step 6:

[0180] Quantification and evaluation of analysis results

[0181] Subject: Server

[0182] Specific operation: The server quantifies the analysis results and evaluates them based on the anomaly detection criteria.

[0183] Input: Analysis results of conversation data and walking data

[0184] Data processing: Quantify the analysis results and compare abnormalities with baseline values.

[0185] Output: Evaluated numerical data is generated.

[0186] Step 7:

[0187] Generate and send notifications

[0188] Subject: Server

[0189] Specific behavior: The server generates notifications based on the evaluation results and sends them to the user and their family members.

[0190] Input: Numerical data of the evaluated analysis results

[0191] Data processing: Generate notification messages and inform users and their families in an appropriate tone.

[0192] Output: A notification message is sent via email or push notification.

[0193] Step 8:

[0194] Checking the results

[0195] Subject: User

[0196] Specific operation: Users and their family members check the notification message on a dedicated smartphone app.

[0197] Input: Notification message sent by the server

[0198] Data processing: Display a notification message in the smartphone app and check the analysis results.

[0199] Output: The notification message is received and confirmed by the user and his / her family members.

[0200] Step 9:

[0201] Collaboration with medical institutions

[0202] Subject: User

[0203] What it does: Users and their families use a link in the app to schedule a medical appointment.

[0204] Input: Notification message in smartphone app

[0205] Data processing: Based on the user's actions, a link is clicked to display the medical institution's appointment page.

[0206] Output: The medical appointment is completed.

[0207] (Application example 1)

[0208] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0209] Early detection of dementia in elderly people and appropriate medical intervention require the effective collection and analysis of conversation and walking data from daily life. However, there is currently no system that can collect this data efficiently and accurately and notify analysis results in a timely manner. Furthermore, devices used to collect this data must be easy to use and designed to integrate seamlessly into daily life. For example, portable devices such as smartphones have limitations, and devices that can more naturally adapt to daily life are needed.

[0210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0211] In this invention, the server includes a means for monitoring the conversation and walking data of the elderly person in real time using sensors built into the smart glasses, and a means for transmitting the data from the smart glasses to the cloud server via a smartphone, thereby enabling early detection of signs of dementia in the elderly person and notifying the user and their family of the information in a timely manner.

[0212] "Elderly" refers to people in an age group who are considered to be at higher risk of dementia and other conditions based on their age and health condition.

[0213] "Conversation data" refers to data that has been recorded and collected from audio information of statements and conversations that elderly people have in their daily lives.

[0214] "Walking data" refers to data that collects movement information such as elderly people's walking patterns, stride length, walking speed, and changes in balance.

[0215] "Encryption" refers to the process of converting collected data into a form that cannot be deciphered by third parties.

[0216] A "cloud server" refers to a remote server that provides data storage and processing functions over the Internet.

[0217] "Decryption" refers to the process of returning encrypted data to its original form.

[0218] An "artificial intelligence model" refers to an algorithm or computational model that discovers patterns and makes predictions and classifications based on large amounts of data.

[0219] "Analysis" refers to the process of extracting useful information or features from collected data.

[0220] "Quantification" refers to the process of converting analytical results into quantitative data that can be evaluated and compared.

[0221] "Anomaly detection" refers to a method for identifying abnormal patterns or signs that deviate from normal conditions.

[0222] "Notification" refers to the process of informing users and their families of the analysis results.

[0223] "Smart glasses" are a wearable eyeglass-type device connected to a computer, equipped with sensors and microphones, and capable of collecting and transmitting data.

[0224] "Data transmission" refers to the process of sending collected data from one device to another device or server.

[0225] This invention is a system for early detection of signs of dementia in the elderly. This system, in particular, uses smart glasses and a smartphone to efficiently perform a series of processes including data collection, encryption, transmission, analysis, and notification.

[0226] System configuration

[0227] Smart glasses (terminal)

[0228] 1. Data Collection Methods

[0229] The smart glasses are equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.), which allow them to periodically collect conversation and walking data of the elderly person in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking patterns (step length, walking speed, balance fluctuations, etc.) and save the data.

[0230] 2. Data Encryption Methods

[0231] The collected conversation and walking data is encrypted within the smart glasses device, and then prepared for transmission to a cloud server via a smartphone. This encryption process protects the privacy and security of the data.

[0232] 3. Data Transmission Method

[0233] The encrypted data is sent over Wi-Fi or Bluetooth to the smartphone, and from there over the internet to a cloud server, using the HTTP or HTTPS protocol.

[0234] Cloud Server (Server)

[0235] 1. Data Receiving Method

[0236] The cloud server receives the encrypted data sent from the smartphone, decrypts it, and stores it in temporary storage to prepare for analysis.

[0237] 2. Data analysis methods

[0238] The cloud server inputs the decoded conversation and walking data into the generative AI model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[0239] 3. Methods for quantifying analysis results

[0240] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[0241] 4. Means of notification

[0242] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[0243] User and his / her family (User)

[0244] 1. Check the results

[0245] Users and their family members can check the analysis results by opening a dedicated smartphone application, which will display a detailed message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0246] 2. Collaboration with medical institutions

[0247] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the application.

[0248] Specific examples

[0249] Example 1: Acquiring and analyzing audio data

[0250] When an elderly person wears smart glasses, the voice recognition module automatically records their conversations. For example, utterances such as "What was that thing yesterday?" are recorded and sent to a cloud server. The server analyzes this data and detects the frequent use of "this, this, that" to determine possible signs of dementia.

[0251] Example 2: Gait data collection and analysis

[0252] Similarly, when an elderly person goes for a walk every day, the built-in sensors in the smart glasses record their walking patterns. For example, data such as extremely short strides or unstable walking speed is collected. This data is also analyzed by the cloud server to evaluate the stability of the person's walking.

[0253] The system of this invention supports early detection of dementia in the elderly and appropriate medical intervention, and provides important information to users and their families.

[0254] Prompt Sentence Examples

[0255] "Please implement a system in which elderly people wear smart glasses and collect and analyze conversation data and walking data in real time during daily life. A natural language processing (NLP) algorithm will be used to detect specific language patterns in the voice data, and a data analysis algorithm will be used to evaluate walking stability in the walking data. The acquired data will be analyzed on a cloud server, and the analysis results will be notified to the elderly person and their family."

[0256] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0257] Step 1:

[0258] Data collection with smart glasses

[0259] The smart glasses, which serve as the device, use a built-in voice recognition module and sensors to collect conversation and walking data from the elderly. The voice recognition module records what the elderly say and saves the audio data. In addition, the accelerometer and gyro sensor measure the elderly's walking patterns (e.g., stride length, walking speed, and balance fluctuations) and save the data.

[0260] Input: Daily conversation and walking patterns of older adults

[0261] Output: Recorded audio data and measured walking data

[0262] Step 2:

[0263] Data Encryption

[0264] The smart glasses, which are the terminals, encrypt the collected conversation and walking data, which protects the privacy and security of the data. The encryption algorithm generally used is the Advanced Encryption Standard (AES).

[0265] Input: Recorded audio data and measured walking data

[0266] Output: Encrypted voice and walking data

[0267] Step 3:

[0268] Data transmission

[0269] The smart glasses send encrypted data to a smartphone via Wi-Fi or Bluetooth, and the smartphone then sends the data over the internet to a cloud server, securely transmitting the data using HTTP or HTTPS protocols.

[0270] Input: Encrypted voice and walking data

[0271] Output: Data sent to the cloud server

[0272] Step 4:

[0273] Data Reception and Decryption

[0274] The cloud server receives the encrypted data sent from the smartphone, decrypts it, and stores it in temporary storage. The decryption algorithm is generally AES.

[0275] Input: Encrypted voice and walking data

[0276] Output: Decoded audio and gait data

[0277] Step 5:

[0278] Data analysis

[0279] The cloud server inputs the decoded voice and walking data into the generative AI model for analysis. Natural language processing (NLP) algorithms are applied to the voice data to detect specific language patterns (e.g., frequent use of "this," "that," "this," "that"), repetition, and jumps in content). Data analysis algorithms are applied to the walking data to evaluate walking stability, stride length variation, and changes in speed.

[0280] Input: Decoded speech and gait data

[0281] Output: Analysis results (specific language patterns, walking stability, etc.)

[0282] Step 6:

[0283] Quantifying analysis results

[0284] The cloud server quantifies the results of the analysis, such as the frequency of a particular language pattern or the degree of instability in walking.

[0285] Input: Analysis results

[0286] Output: Quantified analysis results

[0287] Step 7:

[0288] Anomaly detection and assessment

[0289] The cloud server applies anomaly detection criteria based on the quantified analysis results to determine whether there are signs of dementia. The anomaly detection algorithm uses threshold judgment and machine learning models.

[0290] Input: Quantified analysis results

[0291] Output: Judgment result (normal / abnormal)

[0292] Step 8:

[0293] Generate and send notifications

[0294] The cloud server generates a notification based on the results of the assessment, and the notification is sent to the user and their family via a smartphone app or email, containing the analysis results and diagnostic recommendations. The notification is sent using the SMTP protocol or in-app push notifications.

[0295] Input: Judgment result

[0296] Output: Information messages (analysis results and diagnostic recommendations)

[0297] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0298] The present invention is a system that collects and analyzes conversation data and walking data of elderly people and also evaluates the user's emotional state using an emotion engine. This system includes a smartphone, a cloud server, and the user and their family. This section describes in detail specific embodiments, the program processing, and specific examples.

[0299] System configuration

[0300] Smartphone (device)

[0301] 1. Data Collection Methods

[0302] The smartphone is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). This allows the smartphone to periodically collect the elderly person's conversation and walking data in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking pattern (step length, walking speed, balance fluctuations, etc.) and save the data.

[0303] 2. Data Encryption Methods

[0304] The collected speech and walking data is encrypted within the device and prepared for transmission to a cloud server. This encryption process protects the privacy and security of the data.

[0305] 3. Data Transmission Method

[0306] The encrypted data is then sent over the internet to a cloud server using the HTTP or HTTPS protocol.

[0307] Cloud Server (Server)

[0308] 1. Data Receiving Method

[0309] The cloud server receives the encrypted data sent from the device, decrypts it, and stores it in temporary storage to prepare for analysis.

[0310] 2. Data analysis methods

[0311] The cloud server inputs the decoded conversation and walking data into an artificial intelligence model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[0312] 3. Emotion Engine

[0313] The cloud server also analyzes the user's emotional state using an emotion engine. The emotion engine evaluates the user's emotions (e.g., joy, sadness, anger, surprise, etc.) based on the conversation data and integrates the results into the analysis. It also evaluates the user's physical and emotional state from the walking data to generate an overall health index.

[0314] 4. Methods for quantifying analysis results

[0315] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[0316] 5. Means of notification

[0317] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[0318] User and his / her family (User)

[0319] 1. Check the results

[0320] Users and their family members can check the analysis results by opening a dedicated smartphone app, which will display detailed messages such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0321] 2. Collaboration with medical institutions

[0322] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[0323] Specific examples

[0324] Example 1: Acquiring and analyzing audio data

[0325] If elderly person A has a smartphone, the voice recognition module automatically records A's conversations. For example, if A says, "What was that thing yesterday?", the audio is recorded and sent to a cloud server. The server analyzes this data and detects the frequent occurrence of "this, that, that" to determine possible signs of dementia. The emotion engine also evaluates A's emotional state from the tone of voice and choice of words.

[0326] Example 2: Gait data collection and analysis

[0327] Similarly, when elderly person A goes for a walk every day, the built-in sensor in his smartphone records his walking pattern. For example, data such as extremely short strides or unsteady walking speed is collected. This data is also analyzed on the cloud server to evaluate the stability of his walking. The emotion engine then uses this data to comprehensively evaluate his physical and emotional state and generate an overall health index.

[0328] In this way, the system of the present invention supports early detection of dementia in the elderly and appropriate medical intervention, providing important information to users and their families. In addition, by combining it with an emotion engine, the system can also evaluate the user's emotional state, realizing comprehensive health management.

[0329] The processing flow will be explained below.

[0330] Step 1: Data Collection (Device)

[0331] The device will run a voice recognition module in the background and record the user's voice conversations.

[0332] Specific operation: The device will record audio every 30 seconds and save the recording data to the internal storage.

[0333] The device uses built-in sensors (accelerometer, gyro sensor, etc.) to record the user's walking data in real time.

[0334] Specific operation: The device captures the user's movements (e.g., walking, shaking) and saves the data in the specified format on the internal storage.

[0335] Step 2: Data encryption (device)

[0336] The device encrypts the collected conversation and walking data.

[0337] Specific operation: Encrypt the data using an encryption algorithm (e.g., AES-256) and store it in a temporary buffer.

[0338] Step 3: Send data (terminal)

[0339] The device transmits the encrypted conversation data and walking data to a cloud server.

[0340] Specific operation: Uploads encrypted data to a specific endpoint on a cloud server using HTTP or HTTPS protocols.

[0341] Step 4: Data reception and decryption (server)

[0342] The server receives the encrypted data sent from the terminal.

[0343] Specific operation: The server receives the data through the firewall and stores it in temporary storage.

[0344] The server decrypts the received data.

[0345] What it does: It uses a data decoding algorithm to return the data to its original form and stores it in a database for analysis.

[0346] Step 5: Analyzing conversation data (server)

[0347] The server analyzes the conversation data using natural language processing (NLP) algorithms.

[0348] Specific operation: The server converts the voice data into text data and detects frequent patterns of "this, that, that," jumps in vocabulary, and repetitions.

[0349] Step 6: Sentiment Analysis (Server)

[0350] The server uses an emotion engine to assess the emotional state from the conversation data.

[0351] How it works: The server analyzes the tone, pitch, speed, etc. of the voice and evaluates the user's emotions (happiness, sadness, anger, surprise, etc.).

[0352] Step 7: Analyzing gait data (server)

[0353] The server analyzes the walking data using a data analysis algorithm.

[0354] Specific operation: The server evaluates walking stability, stride length variations, speed changes, etc., and detects abnormal patterns.

[0355] Step 8: Overall Health Assessment (Server)

[0356] The results of the emotion engine are combined with the analysis of walking data to assess overall health.

[0357] Specific actions: Integrate the emotion analysis results and walking data analysis results to generate a comprehensive health index.

[0358] Step 9: Quantifying the analysis results (server)

[0359] The server quantifies the analysis results and evaluates them based on anomaly detection criteria.

[0360] Specific operation: The server statistically processes the analysis results and converts them into numbers based on a scoring system.

[0361] Step 10: Notification Generation and Sending (Server)

[0362] The server generates notifications to the user and their family based on the analysis results.

[0363] Specific behavior: The server generates a notification message and prepares it to be sent in the form of an email or push notification.

[0364] The server generates and sends notifications to the user and their family members.

[0365] Specific operation: The server sends a notification message using the SMTP protocol and notification service.

[0366] Step 11: Check the results (user)

[0367] Users and their family members open the app to check the analysis results.

[0368] What happens: The user taps on the notification on their smartphone to view detailed analysis results and recommendations.

[0369] Step 12: Collaboration with medical institutions (user)

[0370] Users and their families make medical appointments as needed.

[0371] Specific behavior: Click on a link within the app to access the medical institution's appointment page and complete the appointment.

[0372] Step 13: Continuous Data Collection (Device)

[0373] Your smartphone will continue to collect data and analyze it periodically.

[0374] Specific operations: The steps described above, from data collection to transmission, are continuously carried out to conduct long-term monitoring.

[0375] Example 2

[0376] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0377] In recent years, dementia and other health problems have been increasing among the elderly, making early detection and appropriate medical intervention important. However, it is difficult for elderly people themselves and their families to accurately grasp the early signs of dementia and their daily health status. Furthermore, conventional diagnostic methods require hospital visits and evaluations by specialists, which are time-consuming and labor-intensive. Therefore, there is a need for a system that can continuously monitor the health status of elderly people in their daily lives and detect abnormalities early.

[0378] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0379] In this invention, the server includes means for decoding the conversation data and walking data and analyzing them using a natural language processing algorithm and a data analysis algorithm, means for evaluating the user's emotional state using an emotion engine based on the analysis results, and means for quantifying the analysis results and emotion evaluation results and making a judgment based on anomaly detection criteria. This makes it possible to continuously monitor the health status of elderly people, detect early signs of dementia and other health abnormalities at an early stage, and provide the user and their family with prompt and appropriate information and recommendations for medical intervention.

[0380] "Elderly people's conversation data" is voice information of utterances and conversations that elderly people make in their daily lives.

[0381] "Elderly person walking data" refers to data such as walking patterns, speed, and balance when elderly people move around.

[0382] "Encryption" is the process of converting data content into a form that is unintelligible to third parties.

[0383] A "cloud server" is an external server system that stores, manages, and processes data via the Internet.

[0384] "Decryption" is the process of returning encrypted data to its original form.

[0385] A "natural language processing algorithm" is a computer program used to analyze and understand natural language in text or speech.

[0386] A "data analysis algorithm" is a computer program that analyzes collected data and detects specific patterns or anomalies.

[0387] An "emotion engine" is a computer program that evaluates and estimates a user's emotional state from voice and behavioral data.

[0388] "Anomaly detection" is the process of identifying data or patterns that deviate from normal conditions.

[0389] "User's family" refers to the close relatives of the elderly person using the system and people who provide care or nursing care.

[0390] "Notification means" refers to a means of transmitting information to the user and their family members about the analysis results and judgment results.

[0391] A "voice recognition device" is a device or software that converts speech into text data.

[0392] "Built-in sensors" are sensors such as accelerometers and gyro sensors built into the device.

[0393] The present invention is a system that collects and analyzes conversation data and walking data of elderly people and evaluates the user's emotional state using an emotion engine. This system mainly includes a smartphone, a cloud server, and the user and their family. As a specific embodiment, the detailed processing of the program is described below.

[0394] Smartphone (device)

[0395] The device is equipped with a voice recognition device and built-in sensors (acceleration sensor, gyro sensor, etc.), which collect conversation and walking data of the elderly.

[0396] 1. Data Collection Methods

[0397] The device uses a voice recognition device to record and save the elderly person's conversations in the background. For example, if Mr. A says, "I can't remember what happened yesterday," the device collects this voice data. At the same time, the device uses built-in sensors to record the elderly person's walking patterns (e.g., stride length, speed, and fluctuations in balance). Data is continuously collected as Mr. A goes for a walk.

[0398] 2. Data Encryption Methods

[0399] Collected speech and walking data is encrypted within the device using the AES-256 algorithm, a process that protects the privacy and security of the data.

[0400] 3. Data Transmission Method

[0401] The encrypted data is sent to the cloud server using the HTTPS protocol, and the device checks for a response from the server to confirm that the data was sent successfully.

[0402] Cloud Server (Server)

[0403] The server has high processing power and the ability to receive, decode, and analyze the transmitted data.

[0404] 1. Data Receiving Method

[0405] The server receives the encrypted data sent from the device and immediately decrypts it. The decrypted data is then separated into voice data and walking data.

[0406] 2. Data analysis methods

[0407] The decoded voice data is analyzed using natural language processing (NLP) algorithms to detect, for example, frequent use of "this," "that," "that" phrases, repetition, and jumps in content. Meanwhile, data analysis algorithms are used to evaluate walking stability, stride length variability, and changes in speed.

[0408] 3. Emotional assessment measures

[0409] The server uses an emotion engine to evaluate the user's emotional state based on the analysis results. For example, when Person A says, "I can't remember what happened yesterday," the emotion engine detects emotions such as sadness and anxiety. It also evaluates the user's physical and emotional state based on walking data.

[0410] 4. Methods for quantifying analysis results

[0411] The analysis results and emotion assessment results are quantified and judged based on anomaly detection criteria. Based on specific criteria, it is evaluated whether there are signs of dementia.

[0412] 5. Means of notification

[0413] The server generates a notification to the user and their family based on the analysis results and its judgment. The generated notification is sent via email or push notification to a dedicated app. For example, it may say, "Your father's recent conversation patterns show signs of dementia. We recommend that you see a specialist."

[0414] User and his / her family (User)

[0415] Users and their families can check the analysis results through a dedicated smartphone app and take appropriate measures if necessary.

[0416] 1. Check the results

[0417] Users and their families can check the analysis results using a dedicated smartphone app, which displays detailed messages to help detect abnormalities early.

[0418] 2. Collaboration with medical institutions

[0419] Users who receive the notification can use the link in the app to book an appointment at a corresponding medical institution, making it possible to book medical appointments smoothly.

[0420] Specific examples

[0421] Example 1: Acquiring and analyzing audio data

[0422] The device records an elderly person, Mr. A, saying, "I can't remember what happened yesterday," and sends the recording to a cloud server. The server analyzes this data and determines possible signs of dementia by detecting the frequent use of the words "this, that, that." The emotion engine evaluates Mr. A's emotional state based on his tone of voice and choice of words.

[0423] Example 2: Gait data collection and analysis

[0424] The device records A's walking patterns (e.g., stride length, speed, and instability) as he or she goes for a walk. This data is analyzed by a cloud server to evaluate the stability of the person's walking. An emotion engine also uses this data to evaluate the person's physical and emotional state and generate a comprehensive health index.

[0425] Prompt Sentence Examples

[0426] "Describe a system that collects speech and gait data from older adults and assesses their emotional state."

[0427] "Please detail the specific implementation of the system for early detection of dementia in the elderly and how it works."

[0428] "Please tell me the process for analyzing data collected using a smartphone on a cloud server to evaluate the user's health status."

[0429] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0430] Step 1: Data collection (device)

[0431] The device collects conversation data and walking data from the elderly. The voice recognition device records what the elderly person says and records their walking patterns using built-in sensors (acceleration sensor, gyro sensor). For example, if person A says, "I can't remember what happened yesterday," the voice recognition device records this speech. At the same time, the built-in sensor measures walking data such as stride length, speed, and balance fluctuations as person A goes for a walk. The input data is the elderly person's conversation voice and walking sensor data, and the output is the collected voice file and walking dataset.

[0432] Step 2: Data encryption (device)

[0433] The device encrypts the collected data using the AES-256 algorithm, which reduces the risk of unauthorized access to conversation data and walking data by third parties. For example, when Person A says, "I can't remember what happened yesterday," the device encrypts the data, and the walking data is encrypted as well. The input is the collected audio file and walking data set, and the output is the encrypted audio data and walking data.

[0434] Step 3: Send data (terminal)

[0435] The device sends the encrypted data to the cloud server using the HTTPS protocol. If the transmission is successful, the device waits for a response from the server. For example, when encrypted voice data and walking data are sent to the cloud server, a confirmation message is returned that the server has received them. The input is the encrypted voice data and walking data, and the output is the transmission status (success / failure).

[0436] Step 4: Data Reception and Decryption (Server)

[0437] The server receives the encrypted data sent from the terminal and decrypts it using the AES-256 algorithm. The decrypted data is divided into voice data and walking data. For example, the server obtains the voice data "I can't remember yesterday" and the corresponding walking data. The input is the encrypted voice data and walking data, and the output is the decrypted voice data and walking data.

[0438] Step 5: Data analysis (server)

[0439] The server analyzes the decoded voice data using a natural language processing algorithm (NLP). The walking data is evaluated using a data analysis algorithm. For example, the voice data is analyzed for the frequency of "this, that, that," and the walking data is evaluated for walking stability, stride length variation, and speed changes. The input is the decoded voice data and walking data, and the output is the analysis results.

[0440] Step 6: Emotion Evaluation (Server)

[0441] The server uses an emotion engine to evaluate the user's emotional state based on the analysis results. For example, if Person A says, "I can't remember what happened yesterday," the emotion engine detects sadness or anxiety. Physical and emotional states are also evaluated from walking data. The input is the analysis results, and the output is the emotion evaluation results.

[0442] Step 7: Quantifying the analysis results (server)

[0443] The server quantifies the analysis results and emotion evaluation results and makes a judgment based on anomaly detection criteria. For example, if the frequency of "this, that, that" in conversation data is higher than normal, it is quantified as a sign of dementia. The input is the emotion evaluation results and analysis results, and the output is a quantified judgment result.

[0444] Step 8: Notification Generation and Sending (Server)

[0445] The server generates a notification based on the assessment result and sends it to the user and their family. The notification is provided as an email or a push notification to a dedicated app. For example, a notification may be generated stating, "Your father's recent conversation patterns show signs of dementia." The input is the quantified assessment result, and the output is the generated notification message.

[0446] Step 9: Check the results (user)

[0447] The user checks the analysis results using a dedicated smartphone app. Detailed messages are displayed to support early detection of abnormalities. For example, when the user opens the app, a message such as "Your father's recent behavioral patterns show signs of dementia" is displayed. The input is the notification message, and the output is the displayed analysis results.

[0448] Step 10: Collaboration with medical institutions (user)

[0449] After receiving the notification, the user can make an appointment with a medical institution through a link in the app. For example, the user can click the link in the app to access the nearest medical institution's appointment system and complete the appointment. The input is the appointment link in the app, and the output is a notification that the appointment has been completed.

[0450] (Application example 2)

[0451] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0452] Remotely monitoring the health and safety of elderly people in real time is an important challenge for caregivers and families. However, current technology lacks a system that can effectively collect elderly people's conversation and movement information, analyze that data, and detect abnormalities. Furthermore, there is a lack of systems that can promptly notify family members or caregivers when an abnormality is detected. Therefore, an effective solution for properly managing the health and safety of elderly people is needed.

[0453] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting conversation information of the elderly, means for collecting movement information of the elderly, means for encrypting the conversation information and movement information and transmitting them to a cloud server, means for decrypting the received conversation information and movement information in the cloud server and analyzing them using an artificial intelligence model, means for quantifying the analysis results and making a determination based on anomaly detection criteria, means for notifying the user and the user's family of the determination results, and additional means for notifying the user's family in real time if an abnormality is detected based on the analysis results. This makes it possible to continuously monitor the user's condition and immediately notify family members or caregivers if an abnormality is detected.

[0454] "Elderly people's conversation information" refers to audio data of elderly people speaking, including the content, choice of words, tone of voice, etc.

[0455] "Elderly person movement information" is data that indicates the walking and movement patterns of elderly people, including stride length, speed, and changes in balance.

[0456] "Encryption" is a process of converting data using a specific algorithm to make it into a format that cannot be easily understood by third parties in order to protect the content of the data.

[0457] A "cloud server" is a remote server that stores, manages, and analyzes data via the Internet, and can be used without the user having to own physical server equipment.

[0458] An "artificial intelligence model" is an algorithm or system that has the ability to analyze large amounts of data and identify patterns and anomalies from them.

[0459] "Quantifying analytical results" is the process of expressing the results of data analysis in numerical form so that they can be objectively evaluated.

[0460] "Anomaly detection criteria" refers to quantitative or qualitative thresholds or conditions for determining anomalies as a result of data analysis.

[0461] A "notification" is the act of communicating information about a certain event, situation, or when certain conditions are met to a specific recipient.

[0462] "Real time" refers to processing or responding to an event at the moment or almost at the same time as the event occurs.

[0463] A "voice recognition module" is a device or software that receives voice input and converts it into a readable form and further digital data that can be analyzed.

[0464] "Built-in sensors" are sensors that are built into a device and are typically used to measure acceleration, gyroscope, location information, etc.

[0465] "Background recording" refers to the continuous, unobtrusive collection of necessary data while the user is using the device.

[0466] This invention is a system that accumulates and analyzes conversation and movement information of elderly people, and further evaluates the emotional state of the user using an emotion engine. The specific implementation method and procedures for this system are described in detail below.

[0467] System configuration:

[0468] Device (smartphone):

[0469] 1. Data collection methods:

[0470] Speech Recognition Module:

[0471] The device is equipped with a voice recognition module that automatically records conversations between elderly people. For example, if an elderly person says, "What was that thing that happened yesterday?", the audio will be recorded.

[0472] Built-in sensors:

[0473] Built-in sensors (accelerometer, gyro sensor, etc.) are used to record elderly people's walking data (step length, speed, balance fluctuations, etc.). For example, when the stride length becomes extremely short, this data is collected.

[0474] 2. Data Encryption Methods:

[0475] Collected conversation and movement information is encrypted within the device, and this encryption process protects the privacy and security of the data.

[0476] 3. Means of data transmission:

[0477] The encrypted data is sent to the cloud server using the HTTP or HTTPS protocol.

[0478] Server (Cloud Server):

[0479] 1. Data receiving means:

[0480] The cloud server receives the encrypted data sent from the terminal and decrypts the data.

[0481] 2. Data analysis methods:

[0482] The received data is analyzed using artificial intelligence models. Natural language processing (NLP) techniques are used on the conversational information to examine specific language patterns, such as frequent use of "this" and "that," or repetition.

[0483] Data analysis algorithms are used on the movement information to evaluate walking stability and changes in speed.

[0484] 3. Emotion Engine:

[0485] The emotion engine evaluates the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) from conversation data, and also evaluates the user's physical and emotional state from walking data to generate a health index.

[0486] 4. Methods for quantifying analysis results:

[0487] The analysis results are quantified and evaluated based on anomaly detection criteria, which determines whether the elderly person is showing signs of dementia.

[0488] 5. Means of notification:

[0489] The results are then sent to the user and their family via email or push notification, and include the analysis results and diagnostic recommendations.

[0490] User and his / her family (User):

[0491] 1. Check the results:

[0492] Users and their family members can open the app to check the analysis results, which may include a message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0493] 2. Collaboration with medical institutions:

[0494] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[0495] Examples and prompts:

[0496] Examples:

[0497] When elderly person A is spending time at home, conversation information (e.g., "I forgot what I did yesterday") and movement information (e.g., his stride suddenly shortened) are collected and analyzed on a cloud server. If an abnormality is detected, a push notification is sent to his family saying, "Abnormalities have been observed in A's behavior. Please check."

[0498] Example prompt sentence:

[0499] Write a Python program that collects and encrypts conversation and movement data of elderly people and sends it to a cloud server. The cloud server decrypts the data, analyzes it using an AI engine, and notifies family members if an abnormality is detected. Please include specific steps for data collection, encryption, and transmission to create a system.

[0500] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0501] Step 1:

[0502] The device collects voice and walking data:

[0503] The elderly person's conversation information is collected by a voice recognition module, and movement information is recorded using built-in sensors (accelerometer and gyro sensor). The input is the elderly person's conversation and walking patterns, and the output is a voice data file and a walking data file, respectively. Specifically, the device periodically collects voice and walking data in the background and stores it in temporary storage.

[0504] Step 2:

[0505] Encrypt data collected by the device:

[0506] Voice and walking data are encrypted. The input is the collected raw data, and the output is an encrypted data file. Specifically, the device generates an encryption key (or uses an existing key) and encrypts the data using the Fernet algorithm, etc. This protects privacy.

[0507] Step 3:

[0508] The device sends the encrypted data to the cloud server:

[0509] The encrypted data is sent to the cloud server using the HTTP or HTTPS protocol. The input is the encrypted data file, and the output is a transmission status code. Specifically, the device periodically sends an HTTP POST request to the cloud server endpoint to upload the data.

[0510] Step 4:

[0511] The server decrypts the received data:

[0512] The cloud server receives and decrypts the encrypted data sent from the device. The input is the encrypted data and the encryption key, and the output is the decrypted data. Specifically, the server decrypts the data that arrives and converts it into an analyzable format.

[0513] Step 5:

[0514] The server parses the data:

[0515] The server inputs the decoded conversation data and walking data into an artificial intelligence model for analysis. The input is the decoded data, and the output is the analysis results. Specifically, the server analyzes the conversation data using a natural language processing (NLP) algorithm and the walking data using a data analysis algorithm. For example, it evaluates specific language patterns and walking stability.

[0516] Step 6:

[0517] The server uses the emotion engine to assess the emotional state:

[0518] The server uses an emotion engine to evaluate the user's emotional state based on conversation data. It also evaluates emotions and physical condition from walking data. The input is analyzed conversation data and walking data, and the output is the emotion evaluation result. Specifically, the emotion engine analyzes the user's emotions based on their tone of voice and walking patterns, and generates a comprehensive health index.

[0519] Step 7:

[0520] The server quantifies the analysis results and evaluates them based on anomaly detection criteria:

[0521] The server quantifies the analysis results and evaluates them based on anomaly detection criteria. The inputs are emotion evaluation results and data analysis results, and the output is the confirmation results of anomaly detection. Specifically, the server displays the analysis data quantitatively and determines whether or not there are any anomalies according to the set criteria.

[0522] Step 8:

[0523] The server notifies the result:

[0524] If an anomaly is detected, the server generates and sends a notification to the user and their family. The input is the anomaly detection confirmation result, and the output is a notification message. Specifically, the server generates an email or push notification and sends it to the user and their family. The notification includes the analysis results and diagnostic recommendations.

[0525] Step 9:

[0526] Users and their families can review their results and contact their healthcare provider:

[0527] Users and their families can check the analysis results using a dedicated app. If cooperation with a medical institution is required, appointments can be made via a link within the app. The input is a notification message, and the output is a confirmation of the appointment. Specifically, the user and their family check the analysis results and, if necessary, make an appointment with a specialist.

[0528] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0529] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0530] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0531] [Second embodiment]

[0532] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0533] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0534] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0535] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0536] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0537] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0538] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0539] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0540] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0541] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0542] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0543] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0544] This invention is a system that collects and analyzes conversation and walking data from elderly people to detect early signs of dementia. The system includes a smartphone, a cloud server, and users and their families.

[0545] System configuration

[0546] Smartphone (device)

[0547] 1. Data Collection Methods

[0548] The smartphone is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). This allows the smartphone to periodically collect the elderly person's conversation and walking data in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking pattern (step length, walking speed, balance fluctuations, etc.) and save the data.

[0549] 2. Data Encryption Methods

[0550] The collected speech and walking data is encrypted within the device and prepared for transmission to a cloud server. This encryption process protects the privacy and security of the data.

[0551] 3. Data Transmission Method

[0552] The encrypted data is then sent over the internet to a cloud server using the HTTP or HTTPS protocol.

[0553] Cloud Server (Server)

[0554] 1. Data Receiving Method

[0555] The cloud server receives the encrypted data sent from the device, decrypts it, and stores it in temporary storage to prepare for analysis.

[0556] 2. Data analysis methods

[0557] The cloud server inputs the decoded conversation and walking data into an artificial intelligence model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[0558] 3. Methods for quantifying analysis results

[0559] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[0560] 4. Means of notification

[0561] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[0562] User and his / her family (User)

[0563] 1. Check the results

[0564] Users and their family members can check the analysis results by opening a dedicated smartphone app, which will display a detailed message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0565] 2. Collaboration with medical institutions

[0566] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[0567] Specific examples

[0568] Example 1: Acquiring and analyzing audio data

[0569] If elderly person A has a smartphone, the voice recognition module automatically records A's conversations. For example, if A says, "What was that thing that happened yesterday?", the audio is recorded and sent to a cloud server. The server analyzes this data and determines whether there are any signs of dementia by detecting the frequent occurrence of "this, this, that."

[0570] Example 2: Gait data collection and analysis

[0571] Similarly, when elderly person A goes for a walk every day, the built-in sensor in his smartphone records his walking pattern. For example, data such as extremely short strides or unstable walking speed is collected. This data is also analyzed by the cloud server to evaluate the stability of his walking.

[0572] In this way, the system of the present invention supports early detection of dementia in the elderly and appropriate medical intervention, providing important information to users and their families.

[0573] The processing flow will be explained below.

[0574] Step 1: Data Collection (Device)

[0575] The device will run a voice recognition module in the background and record the user's voice conversations.

[0576] Specific operation: The device will record audio every 30 seconds and save the recording data to the internal storage.

[0577] The device uses built-in sensors (accelerometer, gyro sensor, etc.) to record the user's walking data in real time.

[0578] Specific operation: The device captures the user's movements (e.g., walking, shaking) and saves the data in the specified format on the internal storage.

[0579] Step 2: Data encryption (device)

[0580] The device encrypts the collected conversation and walking data.

[0581] Specific operation: Encrypt the data using an encryption algorithm (e.g., AES-256) and store it in a temporary buffer.

[0582] Step 3: Send data (terminal)

[0583] The device transmits the encrypted conversation data and walking data to a cloud server.

[0584] Specific operation: Uploads encrypted data to a specific endpoint on a cloud server using HTTP or HTTPS protocols.

[0585] Step 4: Data reception and decryption (server)

[0586] The server receives the encrypted data sent from the terminal.

[0587] Specific operation: The server receives the data through the firewall and stores it in temporary storage.

[0588] The server decrypts the received data.

[0589] What it does: It uses a data decoding algorithm to return the data to its original form and stores it in a database for analysis.

[0590] Step 5: Data Analysis (Server)

[0591] The server analyzes the conversation data using natural language processing (NLP) algorithms.

[0592] Specific operation: The server converts the voice data into text data and detects frequent patterns of "this, that, that," jumps in vocabulary, and repetitions.

[0593] The server analyzes the walking data using a data analysis algorithm.

[0594] Specific operation: The server evaluates walking stability, stride length variations, speed changes, etc., and detects abnormal patterns.

[0595] Step 6: Quantifying the analysis results (server)

[0596] The server quantifies the analysis results and evaluates them based on anomaly detection criteria.

[0597] Specific operation: The server statistically processes the analysis results and converts them into numbers based on a scoring system.

[0598] Step 7: Notification Generation and Sending (Server)

[0599] The server generates notifications to the user and their family based on the analysis results.

[0600] Specific behavior: The server generates a notification message and prepares it to be sent in the form of an email or push notification.

[0601] The server generates and sends notifications to the user and their family members.

[0602] Specific operation: The server sends a notification message using the SMTP protocol and notification service.

[0603] Step 8: Check the results (user)

[0604] Users and their family members open the app to check the analysis results.

[0605] What happens: The user taps on the notification on their smartphone to view detailed analysis results and recommendations.

[0606] Step 9: Collaboration with medical institutions (user)

[0607] Users and their families make medical appointments as needed.

[0608] Specific behavior: Click on a link within the app to access the medical institution's appointment page and complete the appointment.

[0609] Step 10: Continuous data collection (device)

[0610] Your smartphone will continue to collect data and analyze it periodically.

[0611] Specific operations: The steps described above, from data collection to transmission, are continuously carried out to conduct long-term monitoring.

[0612] Example 1

[0613] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0614] Early detection of dementia in the elderly is extremely important in medical and nursing care settings. However, conventional methods can only collect limited data, making it difficult to detect abnormalities early. Furthermore, the security of collected data and protection of privacy are also important issues. Effective solutions to resolve these issues are needed.

[0615] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0616] In this invention, the server includes a means for collecting conversation data from the elderly, a means for collecting walking data from the elderly, a means for encrypting the conversation data and walking data and transmitting them to a cloud server, a means for decrypting the received conversation data and walking data and analyzing them using a generative AI model in the cloud server, a means for quantifying the analysis results and making a judgment based on anomaly detection criteria, and a means for notifying the judgment results to the user and the user's family. This enables early detection of signs of dementia in the elderly and appropriate medical intervention. Furthermore, data encryption and processing on the cloud server enable data analysis in a secure, privacy-protected environment.

[0617] "Conversation data" is information that records the voices of elderly people and is saved in the form of text or audio files.

[0618] "Gait data" is information that records information about an elderly person's walking pattern, including data such as stride length, walking speed, and balance fluctuations.

[0619] "Encryption" is a technology that converts data to protect the content of the data being transmitted, making it difficult for third parties to access.

[0620] A "cloud server" is a remote computing resource that stores and processes data over the Internet.

[0621] A "generative AI model" is an artificial intelligence algorithm that learns for a specific task and analyzes data and makes predictions.

[0622] "Analysis" is the process of examining collected data, extracting information, and drawing useful conclusions.

[0623] "Quantification" is a means of quantitatively expressing analytical results to facilitate evaluation and comparison.

[0624] "Anomaly detection" is the process of identifying unusual patterns in data analysis and identifying potential problems.

[0625] "Notification" refers to the act of informing the user and their family of the analysis results and judgment results, and is done by means of email, push notification, etc.

[0626] This invention is a system that collects and analyzes conversation data and walking data from elderly people to detect early signs of dementia. This system includes a smartphone, a cloud server, and the user and their family. The following describes how to specifically implement this system.

[0627] Use of smartphones (devices)

[0628] Data collection methods

[0629] The device is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). Using this function, the device periodically collects user conversation and walking data in the background. For example, if a user says, "What was the weather like yesterday?", this voice will be recorded. Meanwhile, as walking data, the device records the user's stride length, walking speed, and balance fluctuations as they walk.

[0630] Data encryption methods

[0631] The collected data is encrypted on the device using the AES-256 algorithm. This encryption process protects the privacy and security of the data. The encrypted data is then prepared for network transmission.

[0632] Data transmission method

[0633] The encrypted data is sent over the Internet to a cloud server using the HTTPS protocol, ensuring secure data transmission.

[0634] Use of cloud servers

[0635] Data reception and decoding means

[0636] The server receives the encrypted data sent by the device and decrypts it using the AES-256 algorithm, storing the decrypted data in temporary storage and preparing it for analysis.

[0637] Data Analysis Methods

[0638] The server then feeds the decoded conversation data into natural language processing (NLP) algorithms to detect certain patterns (e.g., frequent use of "this, this, that," repetition, and jumps in content). Data analysis algorithms are also applied to walking data to evaluate gait stability, stride length variability, and speed fluctuations. For example, if a user frequently says, "What was that thing that happened yesterday?", the NLP algorithms will analyze this pattern.

[0639] Methods for quantifying analysis results

[0640] The analysis results are quantified and evaluated based on anomaly detection criteria. For example, abnormalities in speech patterns or fluctuations in gait stability are output as numerical values.

[0641] Notification means

[0642] The server generates notifications based on the analysis results and sends them to the user and their family. Notifications can be sent via email or push notification. For example, they could include a message like, "Your father's recent conversation patterns have been unusual. We recommend that you see a doctor."

[0643] Use by the User and his / her Family (User)

[0644] Checking the results

[0645] Users and their families can check the analysis results using a dedicated smartphone app. Detailed messages and graphs are displayed on the app screen. For example, a message might read, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0646] Collaboration with medical institutions

[0647] Users and their family members who receive the notification can immediately make an appointment with a medical institution by clicking a link within the app, such as "Click here to make an appointment with a local specialist."

[0648] Prompt Sentence Examples

[0649] "Please explain a system that collects and analyzes conversation and walking data from elderly people to detect early signs of dementia."

[0650] In this way, the system supports early detection of dementia in the elderly and appropriate medical intervention.

[0651] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0652] Step 1:

[0653] Data collection

[0654] Subject: Device

[0655] Specific operation: The device activates the voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.) to collect the user's conversation data and walking data.

[0656] Input: User voice and walking movements

[0657] Data processing: Audio data is recorded in local storage, and walking data is obtained from sensors.

[0658] Output: Collected voice and walking data is stored in the device's local storage.

[0659] Step 2:

[0660] Data Encryption

[0661] Subject: Device

[0662] Specific operation: The device encrypts collected speech and walking data using the AES-256 algorithm.

[0663] Input: Audio data and walking data stored in local storage

[0664] Data processing: Data is converted using an encryption algorithm to make it unreadable to third parties.

[0665] Output: Encrypted audio and gait data is generated.

[0666] Step 3:

[0667] Data transmission

[0668] Subject: Device

[0669] Specific operation: The device sends encrypted data to the cloud server using the HTTPS protocol.

[0670] Input: Encrypted voice and gait data

[0671] Data processing: Sending data over the internet.

[0672] Output: The encrypted data reaches the cloud server.

[0673] Step 4:

[0674] Data Reception and Decryption

[0675] Subject: Server

[0676] What happens: The server receives the encrypted data and decrypts it using the AES-256 algorithm.

[0677] Input: Encrypted data sent from the device

[0678] Data processing: The data is converted using a decoding algorithm to obtain the original voice and walking data.

[0679] Output: The decoded voice data and walking data are stored in the server storage.

[0680] Step 5:

[0681] Data analysis

[0682] Subject: Server

[0683] Specific operation: The server inputs the decoded data into a generative AI model and natural language processing (NLP) algorithm to perform pattern analysis of the speech data and stability analysis of the walking data.

[0684] Input: Decoded audio data and walking data

[0685] Data processing: Speech data is analyzed using NLP algorithms to identify specific language patterns, while walking data is analyzed using data analysis algorithms to assess walking stability, stride length variability, and speed fluctuations.

[0686] Output: Analysis results of conversation data and walking data are generated.

[0687] Step 6:

[0688] Quantification and evaluation of analysis results

[0689] Subject: Server

[0690] Specific operation: The server quantifies the analysis results and evaluates them based on the anomaly detection criteria.

[0691] Input: Analysis results of conversation data and walking data

[0692] Data processing: Quantify the analysis results and compare abnormalities with baseline values.

[0693] Output: Evaluated numerical data is generated.

[0694] Step 7:

[0695] Generate and send notifications

[0696] Subject: Server

[0697] Specific behavior: The server generates notifications based on the evaluation results and sends them to the user and their family members.

[0698] Input: Numerical data of the evaluated analysis results

[0699] Data processing: Generate notification messages and inform users and their families in an appropriate tone.

[0700] Output: A notification message is sent via email or push notification.

[0701] Step 8:

[0702] Checking the results

[0703] Subject: User

[0704] Specific operation: Users and their family members check the notification message on a dedicated smartphone app.

[0705] Input: Notification message sent by the server

[0706] Data processing: Display a notification message in the smartphone app and check the analysis results.

[0707] Output: The notification message is received and confirmed by the user and his / her family members.

[0708] Step 9:

[0709] Collaboration with medical institutions

[0710] Subject: User

[0711] What it does: Users and their families use a link in the app to schedule a medical appointment.

[0712] Input: Notification message in smartphone app

[0713] Data processing: Based on the user's actions, a link is clicked to display the medical institution's appointment page.

[0714] Output: The medical appointment is completed.

[0715] (Application example 1)

[0716] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0717] Early detection of dementia in elderly people and appropriate medical intervention require the effective collection and analysis of conversation and walking data from daily life. However, there is currently no system that can collect this data efficiently and accurately and notify analysis results in a timely manner. Furthermore, devices used to collect this data must be easy to use and designed to integrate seamlessly into daily life. For example, portable devices such as smartphones have limitations, and devices that can more naturally adapt to daily life are needed.

[0718] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0719] In this invention, the server includes a means for monitoring the conversation and walking data of the elderly person in real time using sensors built into the smart glasses, and a means for transmitting the data from the smart glasses to the cloud server via a smartphone, thereby enabling early detection of signs of dementia in the elderly person and notifying the user and their family of the information in a timely manner.

[0720] "Elderly" refers to people in an age group who are considered to be at higher risk of dementia and other conditions based on their age and health condition.

[0721] "Conversation data" refers to data that has been recorded and collected from audio information of statements and conversations that elderly people have in their daily lives.

[0722] "Walking data" refers to data that collects movement information such as elderly people's walking patterns, stride length, walking speed, and changes in balance.

[0723] "Encryption" refers to the process of converting collected data into a form that cannot be deciphered by third parties.

[0724] A "cloud server" refers to a remote server that provides data storage and processing functions over the Internet.

[0725] "Decryption" refers to the process of returning encrypted data to its original form.

[0726] An "artificial intelligence model" refers to an algorithm or computational model that discovers patterns and makes predictions and classifications based on large amounts of data.

[0727] "Analysis" refers to the process of extracting useful information or features from collected data.

[0728] "Quantification" refers to the process of converting analytical results into quantitative data that can be evaluated and compared.

[0729] "Anomaly detection" refers to a method for identifying abnormal patterns or signs that deviate from normal conditions.

[0730] "Notification" refers to the process of informing users and their families of the analysis results.

[0731] "Smart glasses" are a wearable eyeglass-type device connected to a computer, equipped with sensors and microphones, and capable of collecting and transmitting data.

[0732] "Data transmission" refers to the process of sending collected data from one device to another device or server.

[0733] This invention is a system for early detection of signs of dementia in the elderly. This system, in particular, uses smart glasses and a smartphone to efficiently perform a series of processes including data collection, encryption, transmission, analysis, and notification.

[0734] System configuration

[0735] Smart glasses (terminal)

[0736] 1. Data Collection Methods

[0737] The smart glasses are equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.), which allow them to periodically collect conversation and walking data of the elderly person in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking patterns (step length, walking speed, balance fluctuations, etc.) and save the data.

[0738] 2. Data Encryption Methods

[0739] The collected conversation and walking data is encrypted within the smart glasses device, and then prepared for transmission to a cloud server via a smartphone. This encryption process protects the privacy and security of the data.

[0740] 3. Data Transmission Method

[0741] The encrypted data is sent over Wi-Fi or Bluetooth to the smartphone, and from there over the internet to a cloud server, using the HTTP or HTTPS protocol.

[0742] Cloud Server (Server)

[0743] 1. Data Receiving Method

[0744] The cloud server receives the encrypted data sent from the smartphone, decrypts it, and stores it in temporary storage to prepare for analysis.

[0745] 2. Data analysis methods

[0746] The cloud server inputs the decoded conversation and walking data into the generative AI model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[0747] 3. Methods for quantifying analysis results

[0748] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[0749] 4. Means of notification

[0750] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[0751] User and his / her family (User)

[0752] 1. Check the results

[0753] Users and their family members can check the analysis results by opening a dedicated smartphone application, which will display a detailed message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0754] 2. Collaboration with medical institutions

[0755] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the application.

[0756] Specific examples

[0757] Example 1: Acquiring and analyzing audio data

[0758] When an elderly person wears smart glasses, the voice recognition module automatically records their conversations. For example, utterances such as "What was that thing yesterday?" are recorded and sent to a cloud server. The server analyzes this data and detects the frequent use of "this, this, that" to determine possible signs of dementia.

[0759] Example 2: Gait data collection and analysis

[0760] Similarly, when an elderly person goes for a walk every day, the built-in sensors in the smart glasses record their walking patterns. For example, data such as extremely short strides or unstable walking speed is collected. This data is also analyzed by the cloud server to evaluate the stability of the person's walking.

[0761] The system of this invention supports early detection of dementia in the elderly and appropriate medical intervention, and provides important information to users and their families.

[0762] Prompt Sentence Examples

[0763] "Please implement a system in which elderly people wear smart glasses and collect and analyze conversation data and walking data in real time during daily life. A natural language processing (NLP) algorithm will be used to detect specific language patterns in the voice data, and a data analysis algorithm will be used to evaluate walking stability in the walking data. The acquired data will be analyzed on a cloud server, and the analysis results will be notified to the elderly person and their family."

[0764] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0765] Step 1:

[0766] Data collection with smart glasses

[0767] The smart glasses, which serve as the device, use a built-in voice recognition module and sensors to collect conversation and walking data from the elderly. The voice recognition module records what the elderly say and saves the audio data. In addition, the accelerometer and gyro sensor measure the elderly's walking patterns (e.g., stride length, walking speed, and balance fluctuations) and save the data.

[0768] Input: Daily conversation and walking patterns of older adults

[0769] Output: Recorded audio data and measured walking data

[0770] Step 2:

[0771] Data Encryption

[0772] The smart glasses, which are the terminals, encrypt the collected conversation and walking data, which protects the privacy and security of the data. The encryption algorithm generally used is the Advanced Encryption Standard (AES).

[0773] Input: Recorded audio data and measured walking data

[0774] Output: Encrypted voice and walking data

[0775] Step 3:

[0776] Data transmission

[0777] The smart glasses send encrypted data to a smartphone via Wi-Fi or Bluetooth, and the smartphone then sends the data over the internet to a cloud server, securely transmitting the data using HTTP or HTTPS protocols.

[0778] Input: Encrypted voice and walking data

[0779] Output: Data sent to the cloud server

[0780] Step 4:

[0781] Data Reception and Decryption

[0782] The cloud server receives the encrypted data sent from the smartphone, decrypts it, and stores it in temporary storage. The decryption algorithm is generally AES.

[0783] Input: Encrypted voice and walking data

[0784] Output: Decoded audio and gait data

[0785] Step 5:

[0786] Data analysis

[0787] The cloud server inputs the decoded voice and walking data into the generative AI model for analysis. Natural language processing (NLP) algorithms are applied to the voice data to detect specific language patterns (e.g., frequent use of "this," "that," "this," "that"), repetition, and jumps in content). Data analysis algorithms are applied to the walking data to evaluate walking stability, stride length variation, and changes in speed.

[0788] Input: Decoded speech and gait data

[0789] Output: Analysis results (specific language patterns, walking stability, etc.)

[0790] Step 6:

[0791] Quantifying analysis results

[0792] The cloud server quantifies the results of the analysis, such as the frequency of a particular language pattern or the degree of instability in walking.

[0793] Input: Analysis results

[0794] Output: Quantified analysis results

[0795] Step 7:

[0796] Anomaly detection and assessment

[0797] The cloud server applies anomaly detection criteria based on the quantified analysis results to determine whether there are signs of dementia. The anomaly detection algorithm uses threshold judgment and machine learning models.

[0798] Input: Quantified analysis results

[0799] Output: Judgment result (normal / abnormal)

[0800] Step 8:

[0801] Generate and send notifications

[0802] The cloud server generates a notification based on the results of the assessment, and the notification is sent to the user and their family via a smartphone app or email, containing the analysis results and diagnostic recommendations. The notification is sent using the SMTP protocol or in-app push notifications.

[0803] Input: Judgment result

[0804] Output: Information messages (analysis results and diagnostic recommendations)

[0805] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0806] The present invention is a system that collects and analyzes conversation data and walking data of elderly people and also evaluates the user's emotional state using an emotion engine. This system includes a smartphone, a cloud server, and the user and their family. This section describes in detail specific embodiments, the program processing, and specific examples.

[0807] System configuration

[0808] Smartphone (device)

[0809] 1. Data Collection Methods

[0810] The smartphone is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). This allows the smartphone to periodically collect the elderly person's conversation and walking data in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking pattern (step length, walking speed, balance fluctuations, etc.) and save the data.

[0811] 2. Data Encryption Methods

[0812] The collected speech and walking data is encrypted within the device and prepared for transmission to a cloud server. This encryption process protects the privacy and security of the data.

[0813] 3. Data Transmission Method

[0814] The encrypted data is then sent over the internet to a cloud server using the HTTP or HTTPS protocol.

[0815] Cloud Server (Server)

[0816] 1. Data Receiving Method

[0817] The cloud server receives the encrypted data sent from the device, decrypts it, and stores it in temporary storage to prepare for analysis.

[0818] 2. Data analysis methods

[0819] The cloud server inputs the decoded conversation and walking data into an artificial intelligence model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[0820] 3. Emotion Engine

[0821] The cloud server also analyzes the user's emotional state using an emotion engine. The emotion engine evaluates the user's emotions (e.g., joy, sadness, anger, surprise, etc.) based on the conversation data and integrates the results into the analysis. It also evaluates the user's physical and emotional state from the walking data to generate an overall health index.

[0822] 4. Methods for quantifying analysis results

[0823] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[0824] 5. Means of notification

[0825] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[0826] User and his / her family (User)

[0827] 1. Check the results

[0828] Users and their family members can check the analysis results by opening a dedicated smartphone app, which will display detailed messages such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[0829] 2. Collaboration with medical institutions

[0830] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[0831] Specific examples

[0832] Example 1: Acquiring and analyzing audio data

[0833] If elderly person A has a smartphone, the voice recognition module automatically records A's conversations. For example, if A says, "What was that thing yesterday?", the audio is recorded and sent to a cloud server. The server analyzes this data and detects the frequent occurrence of "this, that, that" to determine possible signs of dementia. The emotion engine also evaluates A's emotional state from the tone of voice and choice of words.

[0834] Example 2: Gait data collection and analysis

[0835] Similarly, when elderly person A goes for a walk every day, the built-in sensor in his smartphone records his walking pattern. For example, data such as extremely short strides or unsteady walking speed is collected. This data is also analyzed on the cloud server to evaluate the stability of his walking. The emotion engine then uses this data to comprehensively evaluate his physical and emotional state and generate an overall health index.

[0836] In this way, the system of the present invention supports early detection of dementia in the elderly and appropriate medical intervention, providing important information to users and their families. In addition, by combining it with an emotion engine, the system can also evaluate the user's emotional state, realizing comprehensive health management.

[0837] The processing flow will be explained below.

[0838] Step 1: Data Collection (Device)

[0839] The device will run a voice recognition module in the background and record the user's voice conversations.

[0840] Specific operation: The device will record audio every 30 seconds and save the recording data to the internal storage.

[0841] The device uses built-in sensors (accelerometer, gyro sensor, etc.) to record the user's walking data in real time.

[0842] Specific operation: The device captures the user's movements (e.g., walking, shaking) and saves the data in the specified format on the internal storage.

[0843] Step 2: Data encryption (device)

[0844] The device encrypts the collected conversation and walking data.

[0845] Specific operation: Encrypt the data using an encryption algorithm (e.g., AES-256) and store it in a temporary buffer.

[0846] Step 3: Send data (terminal)

[0847] The device transmits the encrypted conversation data and walking data to a cloud server.

[0848] Specific operation: Uploads encrypted data to a specific endpoint on a cloud server using HTTP or HTTPS protocols.

[0849] Step 4: Data reception and decryption (server)

[0850] The server receives the encrypted data sent from the terminal.

[0851] Specific operation: The server receives the data through the firewall and stores it in temporary storage.

[0852] The server decrypts the received data.

[0853] What it does: It uses a data decoding algorithm to return the data to its original form and stores it in a database for analysis.

[0854] Step 5: Analyzing conversation data (server)

[0855] The server analyzes the conversation data using natural language processing (NLP) algorithms.

[0856] Specific operation: The server converts the voice data into text data and detects frequent patterns of "this, that, that," jumps in vocabulary, and repetitions.

[0857] Step 6: Sentiment Analysis (Server)

[0858] The server uses an emotion engine to assess the emotional state from the conversation data.

[0859] How it works: The server analyzes the tone, pitch, speed, etc. of the voice and evaluates the user's emotions (happiness, sadness, anger, surprise, etc.).

[0860] Step 7: Analyzing gait data (server)

[0861] The server analyzes the walking data using a data analysis algorithm.

[0862] Specific operation: The server evaluates walking stability, stride length variations, speed changes, etc., and detects abnormal patterns.

[0863] Step 8: Overall Health Assessment (Server)

[0864] The results of the emotion engine are combined with the analysis of walking data to assess overall health.

[0865] Specific actions: Integrate the emotion analysis results and walking data analysis results to generate a comprehensive health index.

[0866] Step 9: Quantifying the analysis results (server)

[0867] The server quantifies the analysis results and evaluates them based on anomaly detection criteria.

[0868] Specific operation: The server statistically processes the analysis results and converts them into numbers based on a scoring system.

[0869] Step 10: Notification Generation and Sending (Server)

[0870] The server generates notifications to the user and their family based on the analysis results.

[0871] Specific behavior: The server generates a notification message and prepares it to be sent in the form of an email or push notification.

[0872] The server generates and sends notifications to the user and their family members.

[0873] Specific operation: The server sends a notification message using the SMTP protocol and notification service.

[0874] Step 11: Check the results (user)

[0875] Users and their family members open the app to check the analysis results.

[0876] What happens: The user taps on the notification on their smartphone to view detailed analysis results and recommendations.

[0877] Step 12: Collaboration with medical institutions (user)

[0878] Users and their families make medical appointments as needed.

[0879] Specific behavior: Click on a link within the app to access the medical institution's appointment page and complete the appointment.

[0880] Step 13: Continuous Data Collection (Device)

[0881] Your smartphone will continue to collect data and analyze it periodically.

[0882] Specific operations: The steps described above, from data collection to transmission, are continuously carried out to conduct long-term monitoring.

[0883] Example 2

[0884] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0885] In recent years, dementia and other health problems have been increasing among the elderly, making early detection and appropriate medical intervention important. However, it is difficult for elderly people themselves and their families to accurately grasp the early signs of dementia and their daily health status. Furthermore, conventional diagnostic methods require hospital visits and evaluations by specialists, which are time-consuming and labor-intensive. Therefore, there is a need for a system that can continuously monitor the health status of elderly people in their daily lives and detect abnormalities early.

[0886] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0887] In this invention, the server includes means for decoding the conversation data and walking data and analyzing them using a natural language processing algorithm and a data analysis algorithm, means for evaluating the user's emotional state using an emotion engine based on the analysis results, and means for quantifying the analysis results and emotion evaluation results and making a judgment based on anomaly detection criteria. This makes it possible to continuously monitor the health status of elderly people, detect early signs of dementia and other health abnormalities at an early stage, and provide the user and their family with prompt and appropriate information and recommendations for medical intervention.

[0888] "Elderly people's conversation data" is voice information of utterances and conversations that elderly people make in their daily lives.

[0889] "Elderly person walking data" refers to data such as walking patterns, speed, and balance when elderly people move around.

[0890] "Encryption" is the process of converting data content into a form that is unintelligible to third parties.

[0891] A "cloud server" is an external server system that stores, manages, and processes data via the Internet.

[0892] "Decryption" is the process of returning encrypted data to its original form.

[0893] A "natural language processing algorithm" is a computer program used to analyze and understand natural language in text or speech.

[0894] A "data analysis algorithm" is a computer program that analyzes collected data and detects specific patterns or anomalies.

[0895] An "emotion engine" is a computer program that evaluates and estimates a user's emotional state from voice and behavioral data.

[0896] "Anomaly detection" is the process of identifying data or patterns that deviate from normal conditions.

[0897] "User's family" refers to the close relatives of the elderly person using the system and people who provide care or nursing care.

[0898] "Notification means" refers to a means of transmitting information to the user and their family members about the analysis results and judgment results.

[0899] A "voice recognition device" is a device or software that converts speech into text data.

[0900] "Built-in sensors" are sensors such as accelerometers and gyro sensors built into the device.

[0901] The present invention is a system that collects and analyzes conversation data and walking data of elderly people and evaluates the user's emotional state using an emotion engine. This system mainly includes a smartphone, a cloud server, and the user and their family. As a specific embodiment, the detailed processing of the program is described below.

[0902] Smartphone (device)

[0903] The device is equipped with a voice recognition device and built-in sensors (acceleration sensor, gyro sensor, etc.), which collect conversation and walking data of the elderly.

[0904] 1. Data Collection Methods

[0905] The device uses a voice recognition device to record and save the elderly person's conversations in the background. For example, if Mr. A says, "I can't remember what happened yesterday," the device collects this voice data. At the same time, the device uses built-in sensors to record the elderly person's walking patterns (e.g., stride length, speed, and fluctuations in balance). Data is continuously collected as Mr. A goes for a walk.

[0906] 2. Data Encryption Methods

[0907] Collected speech and walking data is encrypted within the device using the AES-256 algorithm, a process that protects the privacy and security of the data.

[0908] 3. Data Transmission Method

[0909] The encrypted data is sent to the cloud server using the HTTPS protocol, and the device checks for a response from the server to confirm that the data was sent successfully.

[0910] Cloud Server (Server)

[0911] The server has high processing power and the ability to receive, decode, and analyze the transmitted data.

[0912] 1. Data Receiving Method

[0913] The server receives the encrypted data sent from the device and immediately decrypts it. The decrypted data is then separated into voice data and walking data.

[0914] 2. Data analysis methods

[0915] The decoded voice data is analyzed using natural language processing (NLP) algorithms to detect, for example, frequent use of "this," "that," "that" phrases, repetition, and jumps in content. Meanwhile, data analysis algorithms are used to evaluate walking stability, stride length variability, and changes in speed.

[0916] 3. Emotional assessment measures

[0917] The server uses an emotion engine to evaluate the user's emotional state based on the analysis results. For example, when Person A says, "I can't remember what happened yesterday," the emotion engine detects emotions such as sadness and anxiety. It also evaluates the user's physical and emotional state based on walking data.

[0918] 4. Methods for quantifying analysis results

[0919] The analysis results and emotion assessment results are quantified and judged based on anomaly detection criteria. Based on specific criteria, it is evaluated whether there are signs of dementia.

[0920] 5. Means of notification

[0921] The server generates a notification to the user and their family based on the analysis results and its judgment. The generated notification is sent via email or push notification to a dedicated app. For example, it may say, "Your father's recent conversation patterns show signs of dementia. We recommend that you see a specialist."

[0922] User and his / her family (User)

[0923] Users and their families can check the analysis results through a dedicated smartphone app and take appropriate measures if necessary.

[0924] 1. Check the results

[0925] Users and their families can check the analysis results using a dedicated smartphone app, which displays detailed messages to help detect abnormalities early.

[0926] 2. Collaboration with medical institutions

[0927] Users who receive the notification can use the link in the app to book an appointment at a corresponding medical institution, making it possible to book medical appointments smoothly.

[0928] Specific examples

[0929] Example 1: Acquiring and analyzing audio data

[0930] The device records an elderly person, Mr. A, saying, "I can't remember what happened yesterday," and sends the recording to a cloud server. The server analyzes this data and determines possible signs of dementia by detecting the frequent use of the words "this, that, that." The emotion engine evaluates Mr. A's emotional state based on his tone of voice and choice of words.

[0931] Example 2: Gait data collection and analysis

[0932] The device records A's walking patterns (e.g., stride length, speed, and instability) as he or she goes for a walk. This data is analyzed by a cloud server to evaluate the stability of the person's walking. An emotion engine also uses this data to evaluate the person's physical and emotional state and generate a comprehensive health index.

[0933] Prompt Sentence Examples

[0934] "Describe a system that collects speech and gait data from older adults and assesses their emotional state."

[0935] "Please detail the specific implementation of the system for early detection of dementia in the elderly and how it works."

[0936] "Please tell me the process for analyzing data collected using a smartphone on a cloud server to evaluate the user's health status."

[0937] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0938] Step 1: Data collection (device)

[0939] The device collects conversation data and walking data from the elderly. The voice recognition device records what the elderly person says and records their walking patterns using built-in sensors (acceleration sensor, gyro sensor). For example, if person A says, "I can't remember what happened yesterday," the voice recognition device records this speech. At the same time, the built-in sensor measures walking data such as stride length, speed, and balance fluctuations as person A goes for a walk. The input data is the elderly person's conversation voice and walking sensor data, and the output is the collected voice file and walking dataset.

[0940] Step 2: Data encryption (device)

[0941] The device encrypts the collected data using the AES-256 algorithm, which reduces the risk of unauthorized access to conversation data and walking data by third parties. For example, when Person A says, "I can't remember what happened yesterday," the device encrypts the data, and the walking data is encrypted as well. The input is the collected audio file and walking data set, and the output is the encrypted audio data and walking data.

[0942] Step 3: Send data (terminal)

[0943] The device sends the encrypted data to the cloud server using the HTTPS protocol. If the transmission is successful, the device waits for a response from the server. For example, when encrypted voice data and walking data are sent to the cloud server, a confirmation message is returned that the server has received them. The input is the encrypted voice data and walking data, and the output is the transmission status (success / failure).

[0944] Step 4: Data Reception and Decryption (Server)

[0945] The server receives the encrypted data sent from the terminal and decrypts it using the AES-256 algorithm. The decrypted data is divided into voice data and walking data. For example, the server obtains the voice data "I can't remember yesterday" and the corresponding walking data. The input is the encrypted voice data and walking data, and the output is the decrypted voice data and walking data.

[0946] Step 5: Data analysis (server)

[0947] The server analyzes the decoded voice data using a natural language processing algorithm (NLP). The walking data is evaluated using a data analysis algorithm. For example, the voice data is analyzed for the frequency of "this, that, that," and the walking data is evaluated for walking stability, stride length variation, and speed changes. The input is the decoded voice data and walking data, and the output is the analysis results.

[0948] Step 6: Emotion Evaluation (Server)

[0949] The server uses an emotion engine to evaluate the user's emotional state based on the analysis results. For example, if Person A says, "I can't remember what happened yesterday," the emotion engine detects sadness or anxiety. Physical and emotional states are also evaluated from walking data. The input is the analysis results, and the output is the emotion evaluation results.

[0950] Step 7: Quantifying the analysis results (server)

[0951] The server quantifies the analysis results and emotion evaluation results and makes a judgment based on anomaly detection criteria. For example, if the frequency of "this, that, that" in conversation data is higher than normal, it is quantified as a sign of dementia. The input is the emotion evaluation results and analysis results, and the output is a quantified judgment result.

[0952] Step 8: Notification Generation and Sending (Server)

[0953] The server generates a notification based on the assessment result and sends it to the user and their family. The notification is provided as an email or a push notification to a dedicated app. For example, a notification may be generated stating, "Your father's recent conversation patterns show signs of dementia." The input is the quantified assessment result, and the output is the generated notification message.

[0954] Step 9: Check the results (user)

[0955] The user checks the analysis results using a dedicated smartphone app. Detailed messages are displayed to support early detection of abnormalities. For example, when the user opens the app, a message such as "Your father's recent behavioral patterns show signs of dementia" is displayed. The input is the notification message, and the output is the displayed analysis results.

[0956] Step 10: Collaboration with medical institutions (user)

[0957] After receiving the notification, the user can make an appointment with a medical institution through a link in the app. For example, the user can click the link in the app to access the nearest medical institution's appointment system and complete the appointment. The input is the appointment link in the app, and the output is a notification that the appointment has been completed.

[0958] (Application example 2)

[0959] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0960] Remotely monitoring the health and safety of elderly people in real time is an important challenge for caregivers and families. However, current technology lacks a system that can effectively collect elderly people's conversation and movement information, analyze that data, and detect abnormalities. Furthermore, there is a lack of systems that can promptly notify family members or caregivers when an abnormality is detected. Therefore, an effective solution for properly managing the health and safety of elderly people is needed.

[0961] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting conversation information of the elderly, means for collecting movement information of the elderly, means for encrypting the conversation information and movement information and transmitting them to a cloud server, means for decrypting the received conversation information and movement information in the cloud server and analyzing them using an artificial intelligence model, means for quantifying the analysis results and making a determination based on anomaly detection criteria, means for notifying the user and the user's family of the determination results, and additional means for notifying the user's family in real time if an abnormality is detected based on the analysis results. This makes it possible to continuously monitor the user's condition and immediately notify family members or caregivers if an abnormality is detected.

[0962] "Elderly people's conversation information" refers to audio data of elderly people speaking, including the content, choice of words, tone of voice, etc.

[0963] "Elderly person movement information" is data that indicates the walking and movement patterns of elderly people, including stride length, speed, and changes in balance.

[0964] "Encryption" is a process of converting data using a specific algorithm to make it into a format that cannot be easily understood by third parties in order to protect the content of the data.

[0965] A "cloud server" is a remote server that stores, manages, and analyzes data via the Internet, and can be used without the user having to own physical server equipment.

[0966] An "artificial intelligence model" is an algorithm or system that has the ability to analyze large amounts of data and identify patterns and anomalies from them.

[0967] "Quantifying analytical results" is the process of expressing the results of data analysis in numerical form so that they can be objectively evaluated.

[0968] "Anomaly detection criteria" refers to quantitative or qualitative thresholds or conditions for determining anomalies as a result of data analysis.

[0969] A "notification" is the act of communicating information about a certain event, situation, or when certain conditions are met to a specific recipient.

[0970] "Real time" refers to processing or responding to an event at the moment or almost at the same time as the event occurs.

[0971] A "voice recognition module" is a device or software that receives voice input and converts it into a readable form and further digital data that can be analyzed.

[0972] "Built-in sensors" are sensors that are built into a device and are typically used to measure acceleration, gyroscope, location information, etc.

[0973] "Background recording" refers to the continuous, unobtrusive collection of necessary data while the user is using the device.

[0974] This invention is a system that accumulates and analyzes conversation and movement information of elderly people, and further evaluates the emotional state of the user using an emotion engine. The specific implementation method and procedures for this system are described in detail below.

[0975] System configuration:

[0976] Device (smartphone):

[0977] 1. Data collection methods:

[0978] Speech Recognition Module:

[0979] The device is equipped with a voice recognition module that automatically records conversations between elderly people. For example, if an elderly person says, "What was that thing that happened yesterday?", the audio will be recorded.

[0980] Built-in sensors:

[0981] Built-in sensors (accelerometer, gyro sensor, etc.) are used to record elderly people's walking data (step length, speed, balance fluctuations, etc.). For example, when the stride length becomes extremely short, this data is collected.

[0982] 2. Data Encryption Methods:

[0983] Collected conversation and movement information is encrypted within the device, and this encryption process protects the privacy and security of the data.

[0984] 3. Means of data transmission:

[0985] The encrypted data is sent to the cloud server using the HTTP or HTTPS protocol.

[0986] Server (Cloud Server):

[0987] 1. Data receiving means:

[0988] The cloud server receives the encrypted data sent from the terminal and decrypts the data.

[0989] 2. Data analysis methods:

[0990] The received data is analyzed using artificial intelligence models. Natural language processing (NLP) techniques are used on the conversational information to examine specific language patterns, such as frequent use of "this" and "that," or repetition.

[0991] Data analysis algorithms are used on the movement information to evaluate walking stability and changes in speed.

[0992] 3. Emotion Engine:

[0993] The emotion engine evaluates the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) from conversation data, and also evaluates the user's physical and emotional state from walking data to generate a health index.

[0994] 4. Methods for quantifying analysis results:

[0995] The analysis results are quantified and evaluated based on anomaly detection criteria, which determines whether the elderly person is showing signs of dementia.

[0996] 5. Means of notification:

[0997] The results are then sent to the user and their family via email or push notification, and include the analysis results and diagnostic recommendations.

[0998] User and his / her family (User):

[0999] 1. Check the results:

[1000] Users and their family members can open the app to check the analysis results, which may include a message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1001] 2. Collaboration with medical institutions:

[1002] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[1003] Examples and prompts:

[1004] Examples:

[1005] When elderly person A is spending time at home, conversation information (e.g., "I forgot what I did yesterday") and movement information (e.g., his stride suddenly shortened) are collected and analyzed on a cloud server. If an abnormality is detected, a push notification is sent to his family saying, "Abnormalities have been observed in A's behavior. Please check."

[1006] Example prompt sentence:

[1007] Write a Python program that collects and encrypts conversation and movement data of elderly people and sends it to a cloud server. The cloud server decrypts the data, analyzes it using an AI engine, and notifies family members if an abnormality is detected. Please include specific steps for data collection, encryption, and transmission to create a system.

[1008] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1009] Step 1:

[1010] The device collects voice and walking data:

[1011] The elderly person's conversation information is collected by a voice recognition module, and movement information is recorded using built-in sensors (accelerometer and gyro sensor). The input is the elderly person's conversation and walking patterns, and the output is a voice data file and a walking data file, respectively. Specifically, the device periodically collects voice and walking data in the background and stores it in temporary storage.

[1012] Step 2:

[1013] Encrypt data collected by the device:

[1014] Voice and walking data are encrypted. The input is the collected raw data, and the output is an encrypted data file. Specifically, the device generates an encryption key (or uses an existing key) and encrypts the data using the Fernet algorithm, etc. This protects privacy.

[1015] Step 3:

[1016] The device sends the encrypted data to the cloud server:

[1017] The encrypted data is sent to the cloud server using the HTTP or HTTPS protocol. The input is the encrypted data file, and the output is a transmission status code. Specifically, the device periodically sends an HTTP POST request to the cloud server endpoint to upload the data.

[1018] Step 4:

[1019] The server decrypts the received data:

[1020] The cloud server receives and decrypts the encrypted data sent from the device. The input is the encrypted data and the encryption key, and the output is the decrypted data. Specifically, the server decrypts the data that arrives and converts it into an analyzable format.

[1021] Step 5:

[1022] The server parses the data:

[1023] The server inputs the decoded conversation data and walking data into an artificial intelligence model for analysis. The input is the decoded data, and the output is the analysis results. Specifically, the server analyzes the conversation data using a natural language processing (NLP) algorithm and the walking data using a data analysis algorithm. For example, it evaluates specific language patterns and walking stability.

[1024] Step 6:

[1025] The server uses the emotion engine to assess the emotional state:

[1026] The server uses an emotion engine to evaluate the user's emotional state based on conversation data. It also evaluates emotions and physical condition from walking data. The input is analyzed conversation data and walking data, and the output is the emotion evaluation result. Specifically, the emotion engine analyzes the user's emotions based on their tone of voice and walking patterns, and generates a comprehensive health index.

[1027] Step 7:

[1028] The server quantifies the analysis results and evaluates them based on anomaly detection criteria:

[1029] The server quantifies the analysis results and evaluates them based on anomaly detection criteria. The inputs are emotion evaluation results and data analysis results, and the output is the confirmation results of anomaly detection. Specifically, the server displays the analysis data quantitatively and determines whether or not there are any anomalies according to the set criteria.

[1030] Step 8:

[1031] The server notifies the result:

[1032] If an anomaly is detected, the server generates and sends a notification to the user and their family. The input is the anomaly detection confirmation result, and the output is a notification message. Specifically, the server generates an email or push notification and sends it to the user and their family. The notification includes the analysis results and diagnostic recommendations.

[1033] Step 9:

[1034] Users and their families can review their results and contact their healthcare provider:

[1035] Users and their families can check the analysis results using a dedicated app. If cooperation with a medical institution is required, appointments can be made via a link within the app. The input is a notification message, and the output is a confirmation of the appointment. Specifically, the user and their family check the analysis results and, if necessary, make an appointment with a specialist.

[1036] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1037] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1038] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1039] [Third embodiment]

[1040] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1041] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1042] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1043] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1044] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1045] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1046] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1047] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1048] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1049] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1050] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1051] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1052] This invention is a system that collects and analyzes conversation and walking data from elderly people to detect early signs of dementia. The system includes a smartphone, a cloud server, and users and their families.

[1053] System configuration

[1054] Smartphone (device)

[1055] 1. Data Collection Methods

[1056] The smartphone is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). This allows the smartphone to periodically collect the elderly person's conversation and walking data in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking pattern (step length, walking speed, balance fluctuations, etc.) and save the data.

[1057] 2. Data Encryption Methods

[1058] The collected speech and walking data is encrypted within the device and prepared for transmission to a cloud server. This encryption process protects the privacy and security of the data.

[1059] 3. Data Transmission Method

[1060] The encrypted data is then sent over the internet to a cloud server using the HTTP or HTTPS protocol.

[1061] Cloud Server (Server)

[1062] 1. Data Receiving Method

[1063] The cloud server receives the encrypted data sent from the device, decrypts it, and stores it in temporary storage to prepare for analysis.

[1064] 2. Data analysis methods

[1065] The cloud server inputs the decoded conversation and walking data into an artificial intelligence model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[1066] 3. Methods for quantifying analysis results

[1067] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[1068] 4. Means of notification

[1069] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[1070] User and his / her family (User)

[1071] 1. Check the results

[1072] Users and their family members can check the analysis results by opening a dedicated smartphone app, which will display a detailed message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1073] 2. Collaboration with medical institutions

[1074] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[1075] Specific examples

[1076] Example 1: Acquiring and analyzing audio data

[1077] If elderly person A has a smartphone, the voice recognition module automatically records A's conversations. For example, if A says, "What was that thing that happened yesterday?", the audio is recorded and sent to a cloud server. The server analyzes this data and determines whether there are any signs of dementia by detecting the frequent occurrence of "this, this, that."

[1078] Example 2: Gait data collection and analysis

[1079] Similarly, when elderly person A goes for a walk every day, the built-in sensor in his smartphone records his walking pattern. For example, data such as extremely short strides or unstable walking speed is collected. This data is also analyzed by the cloud server to evaluate the stability of his walking.

[1080] In this way, the system of the present invention supports early detection of dementia in the elderly and appropriate medical intervention, providing important information to users and their families.

[1081] The processing flow will be explained below.

[1082] Step 1: Data Collection (Device)

[1083] The device will run a voice recognition module in the background and record the user's voice conversations.

[1084] Specific operation: The device will record audio every 30 seconds and save the recording data to the internal storage.

[1085] The device uses built-in sensors (accelerometer, gyro sensor, etc.) to record the user's walking data in real time.

[1086] Specific operation: The device captures the user's movements (e.g., walking, shaking) and saves the data in the specified format on the internal storage.

[1087] Step 2: Data encryption (device)

[1088] The device encrypts the collected conversation and walking data.

[1089] Specific operation: Encrypt the data using an encryption algorithm (e.g., AES-256) and store it in a temporary buffer.

[1090] Step 3: Send data (terminal)

[1091] The device transmits the encrypted conversation data and walking data to a cloud server.

[1092] Specific operation: Uploads encrypted data to a specific endpoint on a cloud server using HTTP or HTTPS protocols.

[1093] Step 4: Data reception and decryption (server)

[1094] The server receives the encrypted data sent from the terminal.

[1095] Specific operation: The server receives the data through the firewall and stores it in temporary storage.

[1096] The server decrypts the received data.

[1097] What it does: It uses a data decoding algorithm to return the data to its original form and stores it in a database for analysis.

[1098] Step 5: Data Analysis (Server)

[1099] The server analyzes the conversation data using natural language processing (NLP) algorithms.

[1100] Specific operation: The server converts the voice data into text data and detects frequent patterns of "this, that, that," jumps in vocabulary, and repetitions.

[1101] The server analyzes the walking data using a data analysis algorithm.

[1102] Specific operation: The server evaluates walking stability, stride length variations, speed changes, etc., and detects abnormal patterns.

[1103] Step 6: Quantifying the analysis results (server)

[1104] The server quantifies the analysis results and evaluates them based on anomaly detection criteria.

[1105] Specific operation: The server statistically processes the analysis results and converts them into numbers based on a scoring system.

[1106] Step 7: Notification Generation and Sending (Server)

[1107] The server generates notifications to the user and their family based on the analysis results.

[1108] Specific behavior: The server generates a notification message and prepares it to be sent in the form of an email or push notification.

[1109] The server generates and sends notifications to the user and their family members.

[1110] Specific operation: The server sends a notification message using the SMTP protocol and notification service.

[1111] Step 8: Check the results (user)

[1112] Users and their family members open the app to check the analysis results.

[1113] What happens: The user taps on the notification on their smartphone to view detailed analysis results and recommendations.

[1114] Step 9: Collaboration with medical institutions (user)

[1115] Users and their families make medical appointments as needed.

[1116] Specific behavior: Click on a link within the app to access the medical institution's appointment page and complete the appointment.

[1117] Step 10: Continuous data collection (device)

[1118] Your smartphone will continue to collect data and analyze it periodically.

[1119] Specific operations: The steps described above, from data collection to transmission, are continuously carried out to conduct long-term monitoring.

[1120] Example 1

[1121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1122] Early detection of dementia in the elderly is extremely important in medical and nursing care settings. However, conventional methods can only collect limited data, making it difficult to detect abnormalities early. Furthermore, the security of collected data and protection of privacy are also important issues. Effective solutions to resolve these issues are needed.

[1123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1124] In this invention, the server includes a means for collecting conversation data from the elderly, a means for collecting walking data from the elderly, a means for encrypting the conversation data and walking data and transmitting them to a cloud server, a means for decrypting the received conversation data and walking data and analyzing them using a generative AI model in the cloud server, a means for quantifying the analysis results and making a judgment based on anomaly detection criteria, and a means for notifying the judgment results to the user and the user's family. This enables early detection of signs of dementia in the elderly and appropriate medical intervention. Furthermore, data encryption and processing on the cloud server enable data analysis in a secure, privacy-protected environment.

[1125] "Conversation data" is information that records the voices of elderly people and is saved in the form of text or audio files.

[1126] "Gait data" is information that records information about an elderly person's walking pattern, including data such as stride length, walking speed, and balance fluctuations.

[1127] "Encryption" is a technology that converts data to protect the content of the data being transmitted, making it difficult for third parties to access.

[1128] A "cloud server" is a remote computing resource that stores and processes data over the Internet.

[1129] A "generative AI model" is an artificial intelligence algorithm that learns for a specific task and analyzes data and makes predictions.

[1130] "Analysis" is the process of examining collected data, extracting information, and drawing useful conclusions.

[1131] "Quantification" is a means of quantitatively expressing analytical results to facilitate evaluation and comparison.

[1132] "Anomaly detection" is the process of identifying unusual patterns in data analysis and identifying potential problems.

[1133] "Notification" refers to the act of informing the user and their family of the analysis results and judgment results, and is done by means of email, push notification, etc.

[1134] This invention is a system that collects and analyzes conversation data and walking data from elderly people to detect early signs of dementia. This system includes a smartphone, a cloud server, and the user and their family. The following describes how to specifically implement this system.

[1135] Use of smartphones (devices)

[1136] Data collection methods

[1137] The device is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). Using this function, the device periodically collects user conversation and walking data in the background. For example, if a user says, "What was the weather like yesterday?", this voice will be recorded. Meanwhile, as walking data, the device records the user's stride length, walking speed, and balance fluctuations as they walk.

[1138] Data encryption methods

[1139] The collected data is encrypted on the device using the AES-256 algorithm. This encryption process protects the privacy and security of the data. The encrypted data is then prepared for network transmission.

[1140] Data transmission method

[1141] The encrypted data is sent over the Internet to a cloud server using the HTTPS protocol, ensuring secure data transmission.

[1142] Use of cloud servers

[1143] Data reception and decoding means

[1144] The server receives the encrypted data sent by the device and decrypts it using the AES-256 algorithm, storing the decrypted data in temporary storage and preparing it for analysis.

[1145] Data Analysis Methods

[1146] The server then feeds the decoded conversation data into natural language processing (NLP) algorithms to detect certain patterns (e.g., frequent use of "this, this, that," repetition, and jumps in content). Data analysis algorithms are also applied to walking data to evaluate gait stability, stride length variability, and speed fluctuations. For example, if a user frequently says, "What was that thing that happened yesterday?", the NLP algorithms will analyze this pattern.

[1147] Methods for quantifying analysis results

[1148] The analysis results are quantified and evaluated based on anomaly detection criteria. For example, abnormalities in speech patterns or fluctuations in gait stability are output as numerical values.

[1149] Notification means

[1150] The server generates notifications based on the analysis results and sends them to the user and their family. Notifications can be sent via email or push notification. For example, they could include a message like, "Your father's recent conversation patterns have been unusual. We recommend that you see a doctor."

[1151] Use by the User and his / her Family (User)

[1152] Checking the results

[1153] Users and their families can check the analysis results using a dedicated smartphone app. Detailed messages and graphs are displayed on the app screen. For example, a message might read, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1154] Collaboration with medical institutions

[1155] Users and their family members who receive the notification can immediately make an appointment with a medical institution by clicking a link within the app, such as "Click here to make an appointment with a local specialist."

[1156] Prompt Sentence Examples

[1157] "Please explain a system that collects and analyzes conversation and walking data from elderly people to detect early signs of dementia."

[1158] In this way, the system supports early detection of dementia in the elderly and appropriate medical intervention.

[1159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1160] Step 1:

[1161] Data collection

[1162] Subject: Device

[1163] Specific operation: The device activates the voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.) to collect the user's conversation data and walking data.

[1164] Input: User voice and walking movements

[1165] Data processing: Audio data is recorded in local storage, and walking data is obtained from sensors.

[1166] Output: Collected voice and walking data is stored in the device's local storage.

[1167] Step 2:

[1168] Data Encryption

[1169] Subject: Device

[1170] Specific operation: The device encrypts collected speech and walking data using the AES-256 algorithm.

[1171] Input: Audio data and walking data stored in local storage

[1172] Data processing: Data is converted using an encryption algorithm to make it unreadable to third parties.

[1173] Output: Encrypted audio and gait data is generated.

[1174] Step 3:

[1175] Data transmission

[1176] Subject: Device

[1177] Specific operation: The device sends encrypted data to the cloud server using the HTTPS protocol.

[1178] Input: Encrypted voice and gait data

[1179] Data processing: Sending data over the internet.

[1180] Output: The encrypted data reaches the cloud server.

[1181] Step 4:

[1182] Data Reception and Decryption

[1183] Subject: Server

[1184] What happens: The server receives the encrypted data and decrypts it using the AES-256 algorithm.

[1185] Input: Encrypted data sent from the device

[1186] Data processing: The data is converted using a decoding algorithm to obtain the original voice and walking data.

[1187] Output: The decoded voice data and walking data are stored in the server storage.

[1188] Step 5:

[1189] Data analysis

[1190] Subject: Server

[1191] Specific operation: The server inputs the decoded data into a generative AI model and natural language processing (NLP) algorithm to perform pattern analysis of the speech data and stability analysis of the walking data.

[1192] Input: Decoded audio data and walking data

[1193] Data processing: Speech data is analyzed using NLP algorithms to identify specific language patterns, while walking data is analyzed using data analysis algorithms to assess walking stability, stride length variability, and speed fluctuations.

[1194] Output: Analysis results of conversation data and walking data are generated.

[1195] Step 6:

[1196] Quantification and evaluation of analysis results

[1197] Subject: Server

[1198] Specific operation: The server quantifies the analysis results and evaluates them based on the anomaly detection criteria.

[1199] Input: Analysis results of conversation data and walking data

[1200] Data processing: Quantify the analysis results and compare abnormalities with baseline values.

[1201] Output: Evaluated numerical data is generated.

[1202] Step 7:

[1203] Generate and send notifications

[1204] Subject: Server

[1205] Specific behavior: The server generates notifications based on the evaluation results and sends them to the user and their family members.

[1206] Input: Numerical data of the evaluated analysis results

[1207] Data processing: Generate notification messages and inform users and their families in an appropriate tone.

[1208] Output: A notification message is sent via email or push notification.

[1209] Step 8:

[1210] Checking the results

[1211] Subject: User

[1212] Specific operation: Users and their family members check the notification message on a dedicated smartphone app.

[1213] Input: Notification message sent by the server

[1214] Data processing: Display a notification message in the smartphone app and check the analysis results.

[1215] Output: The notification message is received and confirmed by the user and his / her family members.

[1216] Step 9:

[1217] Collaboration with medical institutions

[1218] Subject: User

[1219] What it does: Users and their families use a link in the app to schedule a medical appointment.

[1220] Input: Notification message in smartphone app

[1221] Data processing: Based on the user's actions, a link is clicked to display the medical institution's appointment page.

[1222] Output: The medical appointment is completed.

[1223] (Application example 1)

[1224] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1225] Early detection of dementia in elderly people and appropriate medical intervention require the effective collection and analysis of conversation and walking data from daily life. However, there is currently no system that can collect this data efficiently and accurately and notify analysis results in a timely manner. Furthermore, devices used to collect this data must be easy to use and designed to integrate seamlessly into daily life. For example, portable devices such as smartphones have limitations, and devices that can more naturally adapt to daily life are needed.

[1226] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1227] In this invention, the server includes a means for monitoring the conversation and walking data of the elderly person in real time using sensors built into the smart glasses, and a means for transmitting the data from the smart glasses to the cloud server via a smartphone, thereby enabling early detection of signs of dementia in the elderly person and notifying the user and their family of the information in a timely manner.

[1228] "Elderly" refers to people in an age group who are considered to be at higher risk of dementia and other conditions based on their age and health condition.

[1229] "Conversation data" refers to data that has been recorded and collected from audio information of statements and conversations that elderly people have in their daily lives.

[1230] "Walking data" refers to data that collects movement information such as elderly people's walking patterns, stride length, walking speed, and changes in balance.

[1231] "Encryption" refers to the process of converting collected data into a form that cannot be deciphered by third parties.

[1232] A "cloud server" refers to a remote server that provides data storage and processing functions over the Internet.

[1233] "Decryption" refers to the process of returning encrypted data to its original form.

[1234] An "artificial intelligence model" refers to an algorithm or computational model that discovers patterns and makes predictions and classifications based on large amounts of data.

[1235] "Analysis" refers to the process of extracting useful information or features from collected data.

[1236] "Quantification" refers to the process of converting analytical results into quantitative data that can be evaluated and compared.

[1237] "Anomaly detection" refers to a method for identifying abnormal patterns or signs that deviate from normal conditions.

[1238] "Notification" refers to the process of informing users and their families of the analysis results.

[1239] "Smart glasses" are a wearable eyeglass-type device connected to a computer, equipped with sensors and microphones, and capable of collecting and transmitting data.

[1240] "Data transmission" refers to the process of sending collected data from one device to another device or server.

[1241] This invention is a system for early detection of signs of dementia in the elderly. This system, in particular, uses smart glasses and a smartphone to efficiently perform a series of processes including data collection, encryption, transmission, analysis, and notification.

[1242] System configuration

[1243] Smart glasses (terminal)

[1244] 1. Data Collection Methods

[1245] The smart glasses are equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.), which allow them to periodically collect conversation and walking data of the elderly person in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking patterns (step length, walking speed, balance fluctuations, etc.) and save the data.

[1246] 2. Data Encryption Methods

[1247] The collected conversation and walking data is encrypted within the smart glasses device, and then prepared for transmission to a cloud server via a smartphone. This encryption process protects the privacy and security of the data.

[1248] 3. Data Transmission Method

[1249] The encrypted data is sent over Wi-Fi or Bluetooth to the smartphone, and from there over the internet to a cloud server, using the HTTP or HTTPS protocol.

[1250] Cloud Server (Server)

[1251] 1. Data Receiving Method

[1252] The cloud server receives the encrypted data sent from the smartphone, decrypts it, and stores it in temporary storage to prepare for analysis.

[1253] 2. Data analysis methods

[1254] The cloud server inputs the decoded conversation and walking data into the generative AI model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[1255] 3. Methods for quantifying analysis results

[1256] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[1257] 4. Means of notification

[1258] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[1259] User and his / her family (User)

[1260] 1. Check the results

[1261] Users and their family members can check the analysis results by opening a dedicated smartphone application, which will display a detailed message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1262] 2. Collaboration with medical institutions

[1263] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the application.

[1264] Specific examples

[1265] Example 1: Acquiring and analyzing audio data

[1266] When an elderly person wears smart glasses, the voice recognition module automatically records their conversations. For example, utterances such as "What was that thing yesterday?" are recorded and sent to a cloud server. The server analyzes this data and detects the frequent use of "this, this, that" to determine possible signs of dementia.

[1267] Example 2: Gait data collection and analysis

[1268] Similarly, when an elderly person goes for a walk every day, the built-in sensors in the smart glasses record their walking patterns. For example, data such as extremely short strides or unstable walking speed is collected. This data is also analyzed by the cloud server to evaluate the stability of the person's walking.

[1269] The system of this invention supports early detection of dementia in the elderly and appropriate medical intervention, and provides important information to users and their families.

[1270] Prompt Sentence Examples

[1271] "Please implement a system in which elderly people wear smart glasses and collect and analyze conversation data and walking data in real time during daily life. A natural language processing (NLP) algorithm will be used to detect specific language patterns in the voice data, and a data analysis algorithm will be used to evaluate walking stability in the walking data. The acquired data will be analyzed on a cloud server, and the analysis results will be notified to the elderly person and their family."

[1272] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1273] Step 1:

[1274] Data collection with smart glasses

[1275] The smart glasses, which serve as the device, use a built-in voice recognition module and sensors to collect conversation and walking data from the elderly. The voice recognition module records what the elderly say and saves the audio data. In addition, the accelerometer and gyro sensor measure the elderly's walking patterns (e.g., stride length, walking speed, and balance fluctuations) and save the data.

[1276] Input: Daily conversation and walking patterns of older adults

[1277] Output: Recorded audio data and measured walking data

[1278] Step 2:

[1279] Data Encryption

[1280] The smart glasses, which are the terminals, encrypt the collected conversation and walking data, which protects the privacy and security of the data. The encryption algorithm generally used is the Advanced Encryption Standard (AES).

[1281] Input: Recorded audio data and measured walking data

[1282] Output: Encrypted voice and walking data

[1283] Step 3:

[1284] Data transmission

[1285] The smart glasses send encrypted data to a smartphone via Wi-Fi or Bluetooth, and the smartphone then sends the data over the internet to a cloud server, securely transmitting the data using HTTP or HTTPS protocols.

[1286] Input: Encrypted voice and walking data

[1287] Output: Data sent to the cloud server

[1288] Step 4:

[1289] Data Reception and Decryption

[1290] The cloud server receives the encrypted data sent from the smartphone, decrypts it, and stores it in temporary storage. The decryption algorithm is generally AES.

[1291] Input: Encrypted voice and walking data

[1292] Output: Decoded audio and gait data

[1293] Step 5:

[1294] Data analysis

[1295] The cloud server inputs the decoded voice and walking data into the generative AI model for analysis. Natural language processing (NLP) algorithms are applied to the voice data to detect specific language patterns (e.g., frequent use of "this," "that," "this," "that"), repetition, and jumps in content). Data analysis algorithms are applied to the walking data to evaluate walking stability, stride length variation, and changes in speed.

[1296] Input: Decoded speech and gait data

[1297] Output: Analysis results (specific language patterns, walking stability, etc.)

[1298] Step 6:

[1299] Quantifying analysis results

[1300] The cloud server quantifies the results of the analysis, such as the frequency of a particular language pattern or the degree of instability in walking.

[1301] Input: Analysis results

[1302] Output: Quantified analysis results

[1303] Step 7:

[1304] Anomaly detection and assessment

[1305] The cloud server applies anomaly detection criteria based on the quantified analysis results to determine whether there are signs of dementia. The anomaly detection algorithm uses threshold judgment and machine learning models.

[1306] Input: Quantified analysis results

[1307] Output: Judgment result (normal / abnormal)

[1308] Step 8:

[1309] Generate and send notifications

[1310] The cloud server generates a notification based on the results of the assessment, and the notification is sent to the user and their family via a smartphone app or email, containing the analysis results and diagnostic recommendations. The notification is sent using the SMTP protocol or in-app push notifications.

[1311] Input: Judgment result

[1312] Output: Information messages (analysis results and diagnostic recommendations)

[1313] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1314] The present invention is a system that collects and analyzes conversation data and walking data of elderly people and also evaluates the user's emotional state using an emotion engine. This system includes a smartphone, a cloud server, and the user and their family. This section describes in detail specific embodiments, the program processing, and specific examples.

[1315] System configuration

[1316] Smartphone (device)

[1317] 1. Data Collection Methods

[1318] The smartphone is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). This allows the smartphone to periodically collect the elderly person's conversation and walking data in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking pattern (step length, walking speed, balance fluctuations, etc.) and save the data.

[1319] 2. Data Encryption Methods

[1320] The collected speech and walking data is encrypted within the device and prepared for transmission to a cloud server. This encryption process protects the privacy and security of the data.

[1321] 3. Data Transmission Method

[1322] The encrypted data is then sent over the internet to a cloud server using the HTTP or HTTPS protocol.

[1323] Cloud Server (Server)

[1324] 1. Data Receiving Method

[1325] The cloud server receives the encrypted data sent from the device, decrypts it, and stores it in temporary storage to prepare for analysis.

[1326] 2. Data analysis methods

[1327] The cloud server inputs the decoded conversation and walking data into an artificial intelligence model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[1328] 3. Emotion Engine

[1329] The cloud server also analyzes the user's emotional state using an emotion engine. The emotion engine evaluates the user's emotions (e.g., joy, sadness, anger, surprise, etc.) based on the conversation data and integrates the results into the analysis. It also evaluates the user's physical and emotional state from the walking data to generate an overall health index.

[1330] 4. Methods for quantifying analysis results

[1331] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[1332] 5. Means of notification

[1333] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[1334] User and his / her family (User)

[1335] 1. Check the results

[1336] Users and their family members can check the analysis results by opening a dedicated smartphone app, which will display detailed messages such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1337] 2. Collaboration with medical institutions

[1338] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[1339] Specific examples

[1340] Example 1: Acquiring and analyzing audio data

[1341] If elderly person A has a smartphone, the voice recognition module automatically records A's conversations. For example, if A says, "What was that thing yesterday?", the audio is recorded and sent to a cloud server. The server analyzes this data and detects the frequent occurrence of "this, that, that" to determine possible signs of dementia. The emotion engine also evaluates A's emotional state from the tone of voice and choice of words.

[1342] Example 2: Gait data collection and analysis

[1343] Similarly, when elderly person A goes for a walk every day, the built-in sensor in his smartphone records his walking pattern. For example, data such as extremely short strides or unsteady walking speed is collected. This data is also analyzed on the cloud server to evaluate the stability of his walking. The emotion engine then uses this data to comprehensively evaluate his physical and emotional state and generate an overall health index.

[1344] In this way, the system of the present invention supports early detection of dementia in the elderly and appropriate medical intervention, providing important information to users and their families. In addition, by combining it with an emotion engine, the system can also evaluate the user's emotional state, realizing comprehensive health management.

[1345] The processing flow will be explained below.

[1346] Step 1: Data Collection (Device)

[1347] The device will run a voice recognition module in the background and record the user's voice conversations.

[1348] Specific operation: The device will record audio every 30 seconds and save the recording data to the internal storage.

[1349] The device uses built-in sensors (accelerometer, gyro sensor, etc.) to record the user's walking data in real time.

[1350] Specific operation: The device captures the user's movements (e.g., walking, shaking) and saves the data in the specified format on the internal storage.

[1351] Step 2: Data encryption (device)

[1352] The device encrypts the collected conversation and walking data.

[1353] Specific operation: Encrypt the data using an encryption algorithm (e.g., AES-256) and store it in a temporary buffer.

[1354] Step 3: Send data (terminal)

[1355] The device transmits the encrypted conversation data and walking data to a cloud server.

[1356] Specific operation: Uploads encrypted data to a specific endpoint on a cloud server using HTTP or HTTPS protocols.

[1357] Step 4: Data reception and decryption (server)

[1358] The server receives the encrypted data sent from the terminal.

[1359] Specific operation: The server receives the data through the firewall and stores it in temporary storage.

[1360] The server decrypts the received data.

[1361] What it does: It uses a data decoding algorithm to return the data to its original form and stores it in a database for analysis.

[1362] Step 5: Analyzing conversation data (server)

[1363] The server analyzes the conversation data using natural language processing (NLP) algorithms.

[1364] Specific operation: The server converts the voice data into text data and detects frequent patterns of "this, that, that," jumps in vocabulary, and repetitions.

[1365] Step 6: Sentiment Analysis (Server)

[1366] The server uses an emotion engine to assess the emotional state from the conversation data.

[1367] How it works: The server analyzes the tone, pitch, speed, etc. of the voice and evaluates the user's emotions (happiness, sadness, anger, surprise, etc.).

[1368] Step 7: Analyzing gait data (server)

[1369] The server analyzes the walking data using a data analysis algorithm.

[1370] Specific operation: The server evaluates walking stability, stride length variations, speed changes, etc., and detects abnormal patterns.

[1371] Step 8: Overall Health Assessment (Server)

[1372] The results of the emotion engine are combined with the analysis of walking data to assess overall health.

[1373] Specific actions: Integrate the emotion analysis results and walking data analysis results to generate a comprehensive health index.

[1374] Step 9: Quantifying the analysis results (server)

[1375] The server quantifies the analysis results and evaluates them based on anomaly detection criteria.

[1376] Specific operation: The server statistically processes the analysis results and converts them into numbers based on a scoring system.

[1377] Step 10: Notification Generation and Sending (Server)

[1378] The server generates notifications to the user and their family based on the analysis results.

[1379] Specific behavior: The server generates a notification message and prepares it to be sent in the form of an email or push notification.

[1380] The server generates and sends notifications to the user and their family members.

[1381] Specific operation: The server sends a notification message using the SMTP protocol and notification service.

[1382] Step 11: Check the results (user)

[1383] Users and their family members open the app to check the analysis results.

[1384] What happens: The user taps on the notification on their smartphone to view detailed analysis results and recommendations.

[1385] Step 12: Collaboration with medical institutions (user)

[1386] Users and their families make medical appointments as needed.

[1387] Specific behavior: Click on a link within the app to access the medical institution's appointment page and complete the appointment.

[1388] Step 13: Continuous Data Collection (Device)

[1389] Your smartphone will continue to collect data and analyze it periodically.

[1390] Specific operations: The steps described above, from data collection to transmission, are continuously carried out to conduct long-term monitoring.

[1391] Example 2

[1392] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1393] In recent years, dementia and other health problems have been increasing among the elderly, making early detection and appropriate medical intervention important. However, it is difficult for elderly people themselves and their families to accurately grasp the early signs of dementia and their daily health status. Furthermore, conventional diagnostic methods require hospital visits and evaluations by specialists, which are time-consuming and labor-intensive. Therefore, there is a need for a system that can continuously monitor the health status of elderly people in their daily lives and detect abnormalities early.

[1394] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1395] In this invention, the server includes means for decoding the conversation data and walking data and analyzing them using a natural language processing algorithm and a data analysis algorithm, means for evaluating the user's emotional state using an emotion engine based on the analysis results, and means for quantifying the analysis results and emotion evaluation results and making a judgment based on anomaly detection criteria. This makes it possible to continuously monitor the health status of elderly people, detect early signs of dementia and other health abnormalities at an early stage, and provide the user and their family with prompt and appropriate information and recommendations for medical intervention.

[1396] "Elderly people's conversation data" is voice information of utterances and conversations that elderly people make in their daily lives.

[1397] "Elderly person walking data" refers to data such as walking patterns, speed, and balance when elderly people move around.

[1398] "Encryption" is the process of converting data content into a form that is unintelligible to third parties.

[1399] A "cloud server" is an external server system that stores, manages, and processes data via the Internet.

[1400] "Decryption" is the process of returning encrypted data to its original form.

[1401] A "natural language processing algorithm" is a computer program used to analyze and understand natural language in text or speech.

[1402] A "data analysis algorithm" is a computer program that analyzes collected data and detects specific patterns or anomalies.

[1403] An "emotion engine" is a computer program that evaluates and estimates a user's emotional state from voice and behavioral data.

[1404] "Anomaly detection" is the process of identifying data or patterns that deviate from normal conditions.

[1405] "User's family" refers to the close relatives of the elderly person using the system and people who provide care or nursing care.

[1406] "Notification means" refers to a means of transmitting information to the user and their family members about the analysis results and judgment results.

[1407] A "voice recognition device" is a device or software that converts speech into text data.

[1408] "Built-in sensors" are sensors such as accelerometers and gyro sensors built into the device.

[1409] The present invention is a system that collects and analyzes conversation data and walking data of elderly people and evaluates the user's emotional state using an emotion engine. This system mainly includes a smartphone, a cloud server, and the user and their family. As a specific embodiment, the detailed processing of the program is described below.

[1410] Smartphone (device)

[1411] The device is equipped with a voice recognition device and built-in sensors (acceleration sensor, gyro sensor, etc.), which collect conversation and walking data of the elderly.

[1412] 1. Data Collection Methods

[1413] The device uses a voice recognition device to record and save the elderly person's conversations in the background. For example, if Mr. A says, "I can't remember what happened yesterday," the device collects this voice data. At the same time, the device uses built-in sensors to record the elderly person's walking patterns (e.g., stride length, speed, and fluctuations in balance). Data is continuously collected as Mr. A goes for a walk.

[1414] 2. Data Encryption Methods

[1415] Collected speech and walking data is encrypted within the device using the AES-256 algorithm, a process that protects the privacy and security of the data.

[1416] 3. Data Transmission Method

[1417] The encrypted data is sent to the cloud server using the HTTPS protocol, and the device checks for a response from the server to confirm that the data was sent successfully.

[1418] Cloud Server (Server)

[1419] The server has high processing power and the ability to receive, decode, and analyze the transmitted data.

[1420] 1. Data Receiving Method

[1421] The server receives the encrypted data sent from the device and immediately decrypts it. The decrypted data is then separated into voice data and walking data.

[1422] 2. Data analysis methods

[1423] The decoded voice data is analyzed using natural language processing (NLP) algorithms to detect, for example, frequent use of "this," "that," "that" phrases, repetition, and jumps in content. Meanwhile, data analysis algorithms are used to evaluate walking stability, stride length variability, and changes in speed.

[1424] 3. Emotional assessment measures

[1425] The server uses an emotion engine to evaluate the user's emotional state based on the analysis results. For example, when Person A says, "I can't remember what happened yesterday," the emotion engine detects emotions such as sadness and anxiety. It also evaluates the user's physical and emotional state based on walking data.

[1426] 4. Methods for quantifying analysis results

[1427] The analysis results and emotion assessment results are quantified and judged based on anomaly detection criteria. Based on specific criteria, it is evaluated whether there are signs of dementia.

[1428] 5. Means of notification

[1429] The server generates a notification to the user and their family based on the analysis results and its judgment. The generated notification is sent via email or push notification to a dedicated app. For example, it may say, "Your father's recent conversation patterns show signs of dementia. We recommend that you see a specialist."

[1430] User and his / her family (User)

[1431] Users and their families can check the analysis results through a dedicated smartphone app and take appropriate measures if necessary.

[1432] 1. Check the results

[1433] Users and their families can check the analysis results using a dedicated smartphone app, which displays detailed messages to help detect abnormalities early.

[1434] 2. Collaboration with medical institutions

[1435] Users who receive the notification can use the link in the app to book an appointment at a corresponding medical institution, making it possible to book medical appointments smoothly.

[1436] Specific examples

[1437] Example 1: Acquiring and analyzing audio data

[1438] The device records an elderly person, Mr. A, saying, "I can't remember what happened yesterday," and sends the recording to a cloud server. The server analyzes this data and determines possible signs of dementia by detecting the frequent use of the words "this, that, that." The emotion engine evaluates Mr. A's emotional state based on his tone of voice and choice of words.

[1439] Example 2: Gait data collection and analysis

[1440] The device records A's walking patterns (e.g., stride length, speed, and instability) as he or she goes for a walk. This data is analyzed by a cloud server to evaluate the stability of the person's walking. An emotion engine also uses this data to evaluate the person's physical and emotional state and generate a comprehensive health index.

[1441] Prompt Sentence Examples

[1442] "Describe a system that collects speech and gait data from older adults and assesses their emotional state."

[1443] "Please detail the specific implementation of the system for early detection of dementia in the elderly and how it works."

[1444] "Please tell me the process for analyzing data collected using a smartphone on a cloud server to evaluate the user's health status."

[1445] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1446] Step 1: Data collection (device)

[1447] The device collects conversation data and walking data from the elderly. The voice recognition device records what the elderly person says and records their walking patterns using built-in sensors (acceleration sensor, gyro sensor). For example, if person A says, "I can't remember what happened yesterday," the voice recognition device records this speech. At the same time, the built-in sensor measures walking data such as stride length, speed, and balance fluctuations as person A goes for a walk. The input data is the elderly person's conversation voice and walking sensor data, and the output is the collected voice file and walking dataset.

[1448] Step 2: Data encryption (device)

[1449] The device encrypts the collected data using the AES-256 algorithm, which reduces the risk of unauthorized access to conversation data and walking data by third parties. For example, when Person A says, "I can't remember what happened yesterday," the device encrypts the data, and the walking data is encrypted as well. The input is the collected audio file and walking data set, and the output is the encrypted audio data and walking data.

[1450] Step 3: Send data (terminal)

[1451] The device sends the encrypted data to the cloud server using the HTTPS protocol. If the transmission is successful, the device waits for a response from the server. For example, when encrypted voice data and walking data are sent to the cloud server, a confirmation message is returned that the server has received them. The input is the encrypted voice data and walking data, and the output is the transmission status (success / failure).

[1452] Step 4: Data Reception and Decryption (Server)

[1453] The server receives the encrypted data sent from the terminal and decrypts it using the AES-256 algorithm. The decrypted data is divided into voice data and walking data. For example, the server obtains the voice data "I can't remember yesterday" and the corresponding walking data. The input is the encrypted voice data and walking data, and the output is the decrypted voice data and walking data.

[1454] Step 5: Data analysis (server)

[1455] The server analyzes the decoded voice data using a natural language processing algorithm (NLP). The walking data is evaluated using a data analysis algorithm. For example, the voice data is analyzed for the frequency of "this, that, that," and the walking data is evaluated for walking stability, stride length variation, and speed changes. The input is the decoded voice data and walking data, and the output is the analysis results.

[1456] Step 6: Emotion Evaluation (Server)

[1457] The server uses an emotion engine to evaluate the user's emotional state based on the analysis results. For example, if Person A says, "I can't remember what happened yesterday," the emotion engine detects sadness or anxiety. Physical and emotional states are also evaluated from walking data. The input is the analysis results, and the output is the emotion evaluation results.

[1458] Step 7: Quantifying the analysis results (server)

[1459] The server quantifies the analysis results and emotion evaluation results and makes a judgment based on anomaly detection criteria. For example, if the frequency of "this, that, that" in conversation data is higher than normal, it is quantified as a sign of dementia. The input is the emotion evaluation results and analysis results, and the output is a quantified judgment result.

[1460] Step 8: Notification Generation and Sending (Server)

[1461] The server generates a notification based on the assessment result and sends it to the user and their family. The notification is provided as an email or a push notification to a dedicated app. For example, a notification may be generated stating, "Your father's recent conversation patterns show signs of dementia." The input is the quantified assessment result, and the output is the generated notification message.

[1462] Step 9: Check the results (user)

[1463] The user checks the analysis results using a dedicated smartphone app. Detailed messages are displayed to support early detection of abnormalities. For example, when the user opens the app, a message such as "Your father's recent behavioral patterns show signs of dementia" is displayed. The input is the notification message, and the output is the displayed analysis results.

[1464] Step 10: Collaboration with medical institutions (user)

[1465] After receiving the notification, the user can make an appointment with a medical institution through a link in the app. For example, the user can click the link in the app to access the nearest medical institution's appointment system and complete the appointment. The input is the appointment link in the app, and the output is a notification that the appointment has been completed.

[1466] (Application example 2)

[1467] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1468] Remotely monitoring the health and safety of elderly people in real time is an important challenge for caregivers and families. However, current technology lacks a system that can effectively collect elderly people's conversation and movement information, analyze that data, and detect abnormalities. Furthermore, there is a lack of systems that can promptly notify family members or caregivers when an abnormality is detected. Therefore, an effective solution for properly managing the health and safety of elderly people is needed.

[1469] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting conversation information of the elderly, means for collecting movement information of the elderly, means for encrypting the conversation information and movement information and transmitting them to a cloud server, means for decrypting the received conversation information and movement information in the cloud server and analyzing them using an artificial intelligence model, means for quantifying the analysis results and making a determination based on anomaly detection criteria, means for notifying the user and the user's family of the determination results, and additional means for notifying the user's family in real time if an abnormality is detected based on the analysis results. This makes it possible to continuously monitor the user's condition and immediately notify family members or caregivers if an abnormality is detected.

[1470] "Elderly people's conversation information" refers to audio data of elderly people speaking, including the content, choice of words, tone of voice, etc.

[1471] "Elderly person movement information" is data that indicates the walking and movement patterns of elderly people, including stride length, speed, and changes in balance.

[1472] "Encryption" is a process of converting data using a specific algorithm to make it into a format that cannot be easily understood by third parties in order to protect the content of the data.

[1473] A "cloud server" is a remote server that stores, manages, and analyzes data via the Internet, and can be used without the user having to own physical server equipment.

[1474] An "artificial intelligence model" is an algorithm or system that has the ability to analyze large amounts of data and identify patterns and anomalies from them.

[1475] "Quantifying analytical results" is the process of expressing the results of data analysis in numerical form so that they can be objectively evaluated.

[1476] "Anomaly detection criteria" refers to quantitative or qualitative thresholds or conditions for determining anomalies as a result of data analysis.

[1477] A "notification" is the act of communicating information about a certain event, situation, or when certain conditions are met to a specific recipient.

[1478] "Real time" refers to processing or responding to an event at the moment or almost at the same time as the event occurs.

[1479] A "voice recognition module" is a device or software that receives voice input and converts it into a readable form and further digital data that can be analyzed.

[1480] "Built-in sensors" are sensors that are built into a device and are typically used to measure acceleration, gyroscope, location information, etc.

[1481] "Background recording" refers to the continuous, unobtrusive collection of necessary data while the user is using the device.

[1482] This invention is a system that accumulates and analyzes conversation and movement information of elderly people, and further evaluates the emotional state of the user using an emotion engine. The specific implementation method and procedures for this system are described in detail below.

[1483] System configuration:

[1484] Device (smartphone):

[1485] 1. Data collection methods:

[1486] Speech Recognition Module:

[1487] The device is equipped with a voice recognition module that automatically records conversations between elderly people. For example, if an elderly person says, "What was that thing that happened yesterday?", the audio will be recorded.

[1488] Built-in sensors:

[1489] Built-in sensors (accelerometer, gyro sensor, etc.) are used to record elderly people's walking data (step length, speed, balance fluctuations, etc.). For example, when the stride length becomes extremely short, this data is collected.

[1490] 2. Data Encryption Methods:

[1491] Collected conversation and movement information is encrypted within the device, and this encryption process protects the privacy and security of the data.

[1492] 3. Means of data transmission:

[1493] The encrypted data is sent to the cloud server using the HTTP or HTTPS protocol.

[1494] Server (Cloud Server):

[1495] 1. Data receiving means:

[1496] The cloud server receives the encrypted data sent from the terminal and decrypts the data.

[1497] 2. Data analysis methods:

[1498] The received data is analyzed using artificial intelligence models. Natural language processing (NLP) techniques are used on the conversational information to examine specific language patterns, such as frequent use of "this" and "that," or repetition.

[1499] Data analysis algorithms are used on the movement information to evaluate walking stability and changes in speed.

[1500] 3. Emotion Engine:

[1501] The emotion engine evaluates the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) from conversation data, and also evaluates the user's physical and emotional state from walking data to generate a health index.

[1502] 4. Methods for quantifying analysis results:

[1503] The analysis results are quantified and evaluated based on anomaly detection criteria, which determines whether the elderly person is showing signs of dementia.

[1504] 5. Means of notification:

[1505] The results are then sent to the user and their family via email or push notification, and include the analysis results and diagnostic recommendations.

[1506] User and his / her family (User):

[1507] 1. Check the results:

[1508] Users and their family members can open the app to check the analysis results, which may include a message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1509] 2. Collaboration with medical institutions:

[1510] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[1511] Examples and prompts:

[1512] Examples:

[1513] When elderly person A is spending time at home, conversation information (e.g., "I forgot what I did yesterday") and movement information (e.g., his stride suddenly shortened) are collected and analyzed on a cloud server. If an abnormality is detected, a push notification is sent to his family saying, "Abnormalities have been observed in A's behavior. Please check."

[1514] Example prompt sentence:

[1515] Write a Python program that collects and encrypts conversation and movement data of elderly people and sends it to a cloud server. The cloud server decrypts the data, analyzes it using an AI engine, and notifies family members if an abnormality is detected. Please include specific steps for data collection, encryption, and transmission to create a system.

[1516] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1517] Step 1:

[1518] The device collects voice and walking data:

[1519] The elderly person's conversation information is collected by a voice recognition module, and movement information is recorded using built-in sensors (accelerometer and gyro sensor). The input is the elderly person's conversation and walking patterns, and the output is a voice data file and a walking data file, respectively. Specifically, the device periodically collects voice and walking data in the background and stores it in temporary storage.

[1520] Step 2:

[1521] Encrypt data collected by the device:

[1522] Voice and walking data are encrypted. The input is the collected raw data, and the output is an encrypted data file. Specifically, the device generates an encryption key (or uses an existing key) and encrypts the data using the Fernet algorithm, etc. This protects privacy.

[1523] Step 3:

[1524] The device sends the encrypted data to the cloud server:

[1525] The encrypted data is sent to the cloud server using the HTTP or HTTPS protocol. The input is the encrypted data file, and the output is a transmission status code. Specifically, the device periodically sends an HTTP POST request to the cloud server endpoint to upload the data.

[1526] Step 4:

[1527] The server decrypts the received data:

[1528] The cloud server receives and decrypts the encrypted data sent from the device. The input is the encrypted data and the encryption key, and the output is the decrypted data. Specifically, the server decrypts the data that arrives and converts it into an analyzable format.

[1529] Step 5:

[1530] The server parses the data:

[1531] The server inputs the decoded conversation data and walking data into an artificial intelligence model for analysis. The input is the decoded data, and the output is the analysis results. Specifically, the server analyzes the conversation data using a natural language processing (NLP) algorithm and the walking data using a data analysis algorithm. For example, it evaluates specific language patterns and walking stability.

[1532] Step 6:

[1533] The server uses the emotion engine to assess the emotional state:

[1534] The server uses an emotion engine to evaluate the user's emotional state based on conversation data. It also evaluates emotions and physical condition from walking data. The input is analyzed conversation data and walking data, and the output is the emotion evaluation result. Specifically, the emotion engine analyzes the user's emotions based on their tone of voice and walking patterns, and generates a comprehensive health index.

[1535] Step 7:

[1536] The server quantifies the analysis results and evaluates them based on anomaly detection criteria:

[1537] The server quantifies the analysis results and evaluates them based on anomaly detection criteria. The inputs are emotion evaluation results and data analysis results, and the output is the confirmation results of anomaly detection. Specifically, the server displays the analysis data quantitatively and determines whether or not there are any anomalies according to the set criteria.

[1538] Step 8:

[1539] The server notifies the result:

[1540] If an anomaly is detected, the server generates and sends a notification to the user and their family. The input is the anomaly detection confirmation result, and the output is a notification message. Specifically, the server generates an email or push notification and sends it to the user and their family. The notification includes the analysis results and diagnostic recommendations.

[1541] Step 9:

[1542] Users and their families can review their results and contact their healthcare provider:

[1543] Users and their families can check the analysis results using a dedicated app. If cooperation with a medical institution is required, appointments can be made via a link within the app. The input is a notification message, and the output is a confirmation of the appointment. Specifically, the user and their family check the analysis results and, if necessary, make an appointment with a specialist.

[1544] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1545] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1546] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1547] [Fourth embodiment]

[1548] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1549] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1550] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1551] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1552] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1553] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1554] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1555] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1556] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1557] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1558] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1559] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1560] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1561] This invention is a system that collects and analyzes conversation and walking data from elderly people to detect early signs of dementia. The system includes a smartphone, a cloud server, and users and their families.

[1562] System configuration

[1563] Smartphone (device)

[1564] 1. Data Collection Methods

[1565] The smartphone is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). This allows the smartphone to periodically collect the elderly person's conversation and walking data in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking pattern (step length, walking speed, balance fluctuations, etc.) and save the data.

[1566] 2. Data Encryption Methods

[1567] The collected speech and walking data is encrypted within the device and prepared for transmission to a cloud server. This encryption process protects the privacy and security of the data.

[1568] 3. Data Transmission Method

[1569] The encrypted data is then sent over the internet to a cloud server using the HTTP or HTTPS protocol.

[1570] Cloud Server (Server)

[1571] 1. Data Receiving Method

[1572] The cloud server receives the encrypted data sent from the device, decrypts it, and stores it in temporary storage to prepare for analysis.

[1573] 2. Data analysis methods

[1574] The cloud server inputs the decoded conversation and walking data into an artificial intelligence model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[1575] 3. Methods for quantifying analysis results

[1576] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[1577] 4. Means of notification

[1578] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[1579] User and his / her family (User)

[1580] 1. Check the results

[1581] Users and their family members can check the analysis results by opening a dedicated smartphone app, which will display a detailed message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1582] 2. Collaboration with medical institutions

[1583] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[1584] Specific examples

[1585] Example 1: Acquiring and analyzing audio data

[1586] If elderly person A has a smartphone, the voice recognition module automatically records A's conversations. For example, if A says, "What was that thing that happened yesterday?", the audio is recorded and sent to a cloud server. The server analyzes this data and determines whether there are any signs of dementia by detecting the frequent occurrence of "this, this, that."

[1587] Example 2: Gait data collection and analysis

[1588] Similarly, when elderly person A goes for a walk every day, the built-in sensor in his smartphone records his walking pattern. For example, data such as extremely short strides or unstable walking speed is collected. This data is also analyzed by the cloud server to evaluate the stability of his walking.

[1589] In this way, the system of the present invention supports early detection of dementia in the elderly and appropriate medical intervention, providing important information to users and their families.

[1590] The processing flow will be explained below.

[1591] Step 1: Data Collection (Device)

[1592] The device will run a voice recognition module in the background and record the user's voice conversations.

[1593] Specific operation: The device will record audio every 30 seconds and save the recording data to the internal storage.

[1594] The device uses built-in sensors (accelerometer, gyro sensor, etc.) to record the user's walking data in real time.

[1595] Specific operation: The device captures the user's movements (e.g., walking, shaking) and saves the data in the specified format on the internal storage.

[1596] Step 2: Data encryption (device)

[1597] The device encrypts the collected conversation and walking data.

[1598] Specific operation: Encrypt the data using an encryption algorithm (e.g., AES-256) and store it in a temporary buffer.

[1599] Step 3: Send data (terminal)

[1600] The device transmits the encrypted conversation data and walking data to a cloud server.

[1601] Specific operation: Uploads encrypted data to a specific endpoint on a cloud server using HTTP or HTTPS protocols.

[1602] Step 4: Data reception and decryption (server)

[1603] The server receives the encrypted data sent from the terminal.

[1604] Specific operation: The server receives the data through the firewall and stores it in temporary storage.

[1605] The server decrypts the received data.

[1606] What it does: It uses a data decoding algorithm to return the data to its original form and stores it in a database for analysis.

[1607] Step 5: Data Analysis (Server)

[1608] The server analyzes the conversation data using natural language processing (NLP) algorithms.

[1609] Specific operation: The server converts the voice data into text data and detects frequent patterns of "this, that, that," jumps in vocabulary, and repetitions.

[1610] The server analyzes the walking data using a data analysis algorithm.

[1611] Specific operation: The server evaluates walking stability, stride length variations, speed changes, etc., and detects abnormal patterns.

[1612] Step 6: Quantifying the analysis results (server)

[1613] The server quantifies the analysis results and evaluates them based on anomaly detection criteria.

[1614] Specific operation: The server statistically processes the analysis results and converts them into numbers based on a scoring system.

[1615] Step 7: Notification Generation and Sending (Server)

[1616] The server generates notifications to the user and their family based on the analysis results.

[1617] Specific behavior: The server generates a notification message and prepares it to be sent in the form of an email or push notification.

[1618] The server generates and sends notifications to the user and their family members.

[1619] Specific operation: The server sends a notification message using the SMTP protocol and notification service.

[1620] Step 8: Check the results (user)

[1621] Users and their family members open the app to check the analysis results.

[1622] What happens: The user taps on the notification on their smartphone to view detailed analysis results and recommendations.

[1623] Step 9: Collaboration with medical institutions (user)

[1624] Users and their families make medical appointments as needed.

[1625] Specific behavior: Click on a link within the app to access the medical institution's appointment page and complete the appointment.

[1626] Step 10: Continuous data collection (device)

[1627] Your smartphone will continue to collect data and analyze it periodically.

[1628] Specific operations: The steps described above, from data collection to transmission, are continuously carried out to conduct long-term monitoring.

[1629] Example 1

[1630] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1631] Early detection of dementia in the elderly is extremely important in medical and nursing care settings. However, conventional methods can only collect limited data, making it difficult to detect abnormalities early. Furthermore, the security of collected data and protection of privacy are also important issues. Effective solutions to resolve these issues are needed.

[1632] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1633] In this invention, the server includes a means for collecting conversation data from the elderly, a means for collecting walking data from the elderly, a means for encrypting the conversation data and walking data and transmitting them to a cloud server, a means for decrypting the received conversation data and walking data and analyzing them using a generative AI model in the cloud server, a means for quantifying the analysis results and making a judgment based on anomaly detection criteria, and a means for notifying the judgment results to the user and the user's family. This enables early detection of signs of dementia in the elderly and appropriate medical intervention. Furthermore, data encryption and processing on the cloud server enable data analysis in a secure, privacy-protected environment.

[1634] "Conversation data" is information that records the voices of elderly people and is saved in the form of text or audio files.

[1635] "Gait data" is information that records information about an elderly person's walking pattern, including data such as stride length, walking speed, and balance fluctuations.

[1636] "Encryption" is a technology that converts data to protect the content of the data being transmitted, making it difficult for third parties to access.

[1637] A "cloud server" is a remote computing resource that stores and processes data over the Internet.

[1638] A "generative AI model" is an artificial intelligence algorithm that learns for a specific task and analyzes data and makes predictions.

[1639] "Analysis" is the process of examining collected data, extracting information, and drawing useful conclusions.

[1640] "Quantification" is a means of quantitatively expressing analytical results to facilitate evaluation and comparison.

[1641] "Anomaly detection" is the process of identifying unusual patterns in data analysis and identifying potential problems.

[1642] "Notification" refers to the act of informing the user and their family of the analysis results and judgment results, and is done by means of email, push notification, etc.

[1643] This invention is a system that collects and analyzes conversation data and walking data from elderly people to detect early signs of dementia. This system includes a smartphone, a cloud server, and the user and their family. The following describes how to specifically implement this system.

[1644] Use of smartphones (devices)

[1645] Data collection methods

[1646] The device is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). Using this function, the device periodically collects user conversation and walking data in the background. For example, if a user says, "What was the weather like yesterday?", this voice will be recorded. Meanwhile, as walking data, the device records the user's stride length, walking speed, and balance fluctuations as they walk.

[1647] Data encryption methods

[1648] The collected data is encrypted on the device using the AES-256 algorithm. This encryption process protects the privacy and security of the data. The encrypted data is then prepared for network transmission.

[1649] Data transmission method

[1650] The encrypted data is sent over the Internet to a cloud server using the HTTPS protocol, ensuring secure data transmission.

[1651] Use of cloud servers

[1652] Data reception and decoding means

[1653] The server receives the encrypted data sent by the device and decrypts it using the AES-256 algorithm, storing the decrypted data in temporary storage and preparing it for analysis.

[1654] Data Analysis Methods

[1655] The server then feeds the decoded conversation data into natural language processing (NLP) algorithms to detect certain patterns (e.g., frequent use of "this, this, that," repetition, and jumps in content). Data analysis algorithms are also applied to walking data to evaluate gait stability, stride length variability, and speed fluctuations. For example, if a user frequently says, "What was that thing that happened yesterday?", the NLP algorithms will analyze this pattern.

[1656] Methods for quantifying analysis results

[1657] The analysis results are quantified and evaluated based on anomaly detection criteria. For example, abnormalities in speech patterns or fluctuations in gait stability are output as numerical values.

[1658] Notification means

[1659] The server generates notifications based on the analysis results and sends them to the user and their family. Notifications can be sent via email or push notification. For example, they could include a message like, "Your father's recent conversation patterns have been unusual. We recommend that you see a doctor."

[1660] Use by the User and his / her Family (User)

[1661] Checking the results

[1662] Users and their families can check the analysis results using a dedicated smartphone app. Detailed messages and graphs are displayed on the app screen. For example, a message might read, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1663] Collaboration with medical institutions

[1664] Users and their family members who receive the notification can immediately make an appointment with a medical institution by clicking a link within the app, such as "Click here to make an appointment with a local specialist."

[1665] Prompt Sentence Examples

[1666] "Please explain a system that collects and analyzes conversation and walking data from elderly people to detect early signs of dementia."

[1667] In this way, the system supports early detection of dementia in the elderly and appropriate medical intervention.

[1668] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1669] Step 1:

[1670] Data collection

[1671] Subject: Device

[1672] Specific operation: The device activates the voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.) to collect the user's conversation data and walking data.

[1673] Input: User voice and walking movements

[1674] Data processing: Audio data is recorded in local storage, and walking data is obtained from sensors.

[1675] Output: Collected voice and walking data is stored in the device's local storage.

[1676] Step 2:

[1677] Data Encryption

[1678] Subject: Device

[1679] Specific operation: The device encrypts collected speech and walking data using the AES-256 algorithm.

[1680] Input: Audio data and walking data stored in local storage

[1681] Data processing: Data is converted using an encryption algorithm to make it unreadable to third parties.

[1682] Output: Encrypted audio and gait data is generated.

[1683] Step 3:

[1684] Data transmission

[1685] Subject: Device

[1686] Specific operation: The device sends encrypted data to the cloud server using the HTTPS protocol.

[1687] Input: Encrypted voice and gait data

[1688] Data processing: Sending data over the internet.

[1689] Output: The encrypted data reaches the cloud server.

[1690] Step 4:

[1691] Data Reception and Decryption

[1692] Subject: Server

[1693] What happens: The server receives the encrypted data and decrypts it using the AES-256 algorithm.

[1694] Input: Encrypted data sent from the device

[1695] Data processing: The data is converted using a decoding algorithm to obtain the original voice and walking data.

[1696] Output: The decoded voice data and walking data are stored in the server storage.

[1697] Step 5:

[1698] Data analysis

[1699] Subject: Server

[1700] Specific operation: The server inputs the decoded data into a generative AI model and natural language processing (NLP) algorithm to perform pattern analysis of the speech data and stability analysis of the walking data.

[1701] Input: Decoded audio data and walking data

[1702] Data processing: Speech data is analyzed using NLP algorithms to identify specific language patterns, while walking data is analyzed using data analysis algorithms to assess walking stability, stride length variability, and speed fluctuations.

[1703] Output: Analysis results of conversation data and walking data are generated.

[1704] Step 6:

[1705] Quantification and evaluation of analysis results

[1706] Subject: Server

[1707] Specific operation: The server quantifies the analysis results and evaluates them based on the anomaly detection criteria.

[1708] Input: Analysis results of conversation data and walking data

[1709] Data processing: Quantify the analysis results and compare abnormalities with baseline values.

[1710] Output: Evaluated numerical data is generated.

[1711] Step 7:

[1712] Generate and send notifications

[1713] Subject: Server

[1714] Specific behavior: The server generates notifications based on the evaluation results and sends them to the user and their family members.

[1715] Input: Numerical data of the evaluated analysis results

[1716] Data processing: Generate notification messages and inform users and their families in an appropriate tone.

[1717] Output: A notification message is sent via email or push notification.

[1718] Step 8:

[1719] Checking the results

[1720] Subject: User

[1721] Specific operation: Users and their family members check the notification message on a dedicated smartphone app.

[1722] Input: Notification message sent by the server

[1723] Data processing: Display a notification message in the smartphone app and check the analysis results.

[1724] Output: The notification message is received and confirmed by the user and his / her family members.

[1725] Step 9:

[1726] Collaboration with medical institutions

[1727] Subject: User

[1728] What it does: Users and their families use a link in the app to schedule a medical appointment.

[1729] Input: Notification message in smartphone app

[1730] Data processing: Based on the user's actions, a link is clicked to display the medical institution's appointment page.

[1731] Output: The medical appointment is completed.

[1732] (Application example 1)

[1733] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1734] Early detection of dementia in elderly people and appropriate medical intervention require the effective collection and analysis of conversation and walking data from daily life. However, there is currently no system that can collect this data efficiently and accurately and notify analysis results in a timely manner. Furthermore, devices used to collect this data must be easy to use and designed to integrate seamlessly into daily life. For example, portable devices such as smartphones have limitations, and devices that can more naturally adapt to daily life are needed.

[1735] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1736] In this invention, the server includes a means for monitoring the conversation and walking data of the elderly person in real time using sensors built into the smart glasses, and a means for transmitting the data from the smart glasses to the cloud server via a smartphone, thereby enabling early detection of signs of dementia in the elderly person and notifying the user and their family of the information in a timely manner.

[1737] "Elderly" refers to people in an age group who are considered to be at higher risk of dementia and other conditions based on their age and health condition.

[1738] "Conversation data" refers to data that has been recorded and collected from audio information of statements and conversations that elderly people have in their daily lives.

[1739] "Walking data" refers to data that collects movement information such as elderly people's walking patterns, stride length, walking speed, and changes in balance.

[1740] "Encryption" refers to the process of converting collected data into a form that cannot be deciphered by third parties.

[1741] A "cloud server" refers to a remote server that provides data storage and processing functions over the Internet.

[1742] "Decryption" refers to the process of returning encrypted data to its original form.

[1743] An "artificial intelligence model" refers to an algorithm or computational model that discovers patterns and makes predictions and classifications based on large amounts of data.

[1744] "Analysis" refers to the process of extracting useful information or features from collected data.

[1745] "Quantification" refers to the process of converting analytical results into quantitative data that can be evaluated and compared.

[1746] "Anomaly detection" refers to a method for identifying abnormal patterns or signs that deviate from normal conditions.

[1747] "Notification" refers to the process of informing users and their families of the analysis results.

[1748] "Smart glasses" are a wearable eyeglass-type device connected to a computer, equipped with sensors and microphones, and capable of collecting and transmitting data.

[1749] "Data transmission" refers to the process of sending collected data from one device to another device or server.

[1750] This invention is a system for early detection of signs of dementia in the elderly. This system, in particular, uses smart glasses and a smartphone to efficiently perform a series of processes including data collection, encryption, transmission, analysis, and notification.

[1751] System configuration

[1752] Smart glasses (terminal)

[1753] 1. Data Collection Methods

[1754] The smart glasses are equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.), which allow them to periodically collect conversation and walking data of the elderly person in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking patterns (step length, walking speed, balance fluctuations, etc.) and save the data.

[1755] 2. Data Encryption Methods

[1756] The collected conversation and walking data is encrypted within the smart glasses device, and then prepared for transmission to a cloud server via a smartphone. This encryption process protects the privacy and security of the data.

[1757] 3. Data Transmission Method

[1758] The encrypted data is sent over Wi-Fi or Bluetooth to the smartphone, and from there over the internet to a cloud server, using the HTTP or HTTPS protocol.

[1759] Cloud Server (Server)

[1760] 1. Data Receiving Method

[1761] The cloud server receives the encrypted data sent from the smartphone, decrypts it, and stores it in temporary storage to prepare for analysis.

[1762] 2. Data analysis methods

[1763] The cloud server inputs the decoded conversation and walking data into the generative AI model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[1764] 3. Methods for quantifying analysis results

[1765] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[1766] 4. Means of notification

[1767] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[1768] User and his / her family (User)

[1769] 1. Check the results

[1770] Users and their family members can check the analysis results by opening a dedicated smartphone application, which will display a detailed message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1771] 2. Collaboration with medical institutions

[1772] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the application.

[1773] Specific examples

[1774] Example 1: Acquiring and analyzing audio data

[1775] When an elderly person wears smart glasses, the voice recognition module automatically records their conversations. For example, utterances such as "What was that thing yesterday?" are recorded and sent to a cloud server. The server analyzes this data and detects the frequent use of "this, this, that" to determine possible signs of dementia.

[1776] Example 2: Gait data collection and analysis

[1777] Similarly, when an elderly person goes for a walk every day, the built-in sensors in the smart glasses record their walking patterns. For example, data such as extremely short strides or unstable walking speed is collected. This data is also analyzed by the cloud server to evaluate the stability of the person's walking.

[1778] The system of this invention supports early detection of dementia in the elderly and appropriate medical intervention, and provides important information to users and their families.

[1779] Prompt Sentence Examples

[1780] "Please implement a system in which elderly people wear smart glasses and collect and analyze conversation data and walking data in real time during daily life. A natural language processing (NLP) algorithm will be used to detect specific language patterns in the voice data, and a data analysis algorithm will be used to evaluate walking stability in the walking data. The acquired data will be analyzed on a cloud server, and the analysis results will be notified to the elderly person and their family."

[1781] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1782] Step 1:

[1783] Data collection with smart glasses

[1784] The smart glasses, which serve as the device, use a built-in voice recognition module and sensors to collect conversation and walking data from the elderly. The voice recognition module records what the elderly say and saves the audio data. In addition, the accelerometer and gyro sensor measure the elderly's walking patterns (e.g., stride length, walking speed, and balance fluctuations) and save the data.

[1785] Input: Daily conversation and walking patterns of older adults

[1786] Output: Recorded audio data and measured walking data

[1787] Step 2:

[1788] Data Encryption

[1789] The smart glasses, which are the terminals, encrypt the collected conversation and walking data, which protects the privacy and security of the data. The encryption algorithm generally used is the Advanced Encryption Standard (AES).

[1790] Input: Recorded audio data and measured walking data

[1791] Output: Encrypted voice and walking data

[1792] Step 3:

[1793] Data transmission

[1794] The smart glasses send encrypted data to a smartphone via Wi-Fi or Bluetooth, and the smartphone then sends the data over the internet to a cloud server, securely transmitting the data using HTTP or HTTPS protocols.

[1795] Input: Encrypted voice and walking data

[1796] Output: Data sent to the cloud server

[1797] Step 4:

[1798] Data Reception and Decryption

[1799] The cloud server receives the encrypted data sent from the smartphone, decrypts it, and stores it in temporary storage. The decryption algorithm is generally AES.

[1800] Input: Encrypted voice and walking data

[1801] Output: Decoded audio and gait data

[1802] Step 5:

[1803] Data analysis

[1804] The cloud server inputs the decoded voice and walking data into the generative AI model for analysis. Natural language processing (NLP) algorithms are applied to the voice data to detect specific language patterns (e.g., frequent use of "this," "that," "this," "that"), repetition, and jumps in content). Data analysis algorithms are applied to the walking data to evaluate walking stability, stride length variation, and changes in speed.

[1805] Input: Decoded speech and gait data

[1806] Output: Analysis results (specific language patterns, walking stability, etc.)

[1807] Step 6:

[1808] Quantifying analysis results

[1809] The cloud server quantifies the results of the analysis, such as the frequency of a particular language pattern or the degree of instability in walking.

[1810] Input: Analysis results

[1811] Output: Quantified analysis results

[1812] Step 7:

[1813] Anomaly detection and assessment

[1814] The cloud server applies anomaly detection criteria based on the quantified analysis results to determine whether there are signs of dementia. The anomaly detection algorithm uses threshold judgment and machine learning models.

[1815] Input: Quantified analysis results

[1816] Output: Judgment result (normal / abnormal)

[1817] Step 8:

[1818] Generate and send notifications

[1819] The cloud server generates a notification based on the results of the assessment, and the notification is sent to the user and their family via a smartphone app or email, containing the analysis results and diagnostic recommendations. The notification is sent using the SMTP protocol or in-app push notifications.

[1820] Input: Judgment result

[1821] Output: Information messages (analysis results and diagnostic recommendations)

[1822] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1823] The present invention is a system that collects and analyzes conversation data and walking data of elderly people and also evaluates the user's emotional state using an emotion engine. This system includes a smartphone, a cloud server, and the user and their family. This section describes in detail specific embodiments, the program processing, and specific examples.

[1824] System configuration

[1825] Smartphone (device)

[1826] 1. Data Collection Methods

[1827] The smartphone is equipped with a voice recognition module and built-in sensors (accelerometer, gyro sensor, etc.). This allows the smartphone to periodically collect the elderly person's conversation and walking data in the background. The voice recognition module records what the user says and saves the audio data. The built-in sensors measure the user's walking pattern (step length, walking speed, balance fluctuations, etc.) and save the data.

[1828] 2. Data Encryption Methods

[1829] The collected speech and walking data is encrypted within the device and prepared for transmission to a cloud server. This encryption process protects the privacy and security of the data.

[1830] 3. Data Transmission Method

[1831] The encrypted data is then sent over the internet to a cloud server using the HTTP or HTTPS protocol.

[1832] Cloud Server (Server)

[1833] 1. Data Receiving Method

[1834] The cloud server receives the encrypted data sent from the device, decrypts it, and stores it in temporary storage to prepare for analysis.

[1835] 2. Data analysis methods

[1836] The cloud server inputs the decoded conversation and walking data into an artificial intelligence model for analysis. For the conversation data, natural language processing (NLP) algorithms are used to detect specific language patterns (e.g., frequent use of "this," "that," "repetition," and jumps in content). For the walking data, data analysis algorithms are used to evaluate walking stability, stride length variation, and changes in speed.

[1837] 3. Emotion Engine

[1838] The cloud server also analyzes the user's emotional state using an emotion engine. The emotion engine evaluates the user's emotions (e.g., joy, sadness, anger, surprise, etc.) based on the conversation data and integrates the results into the analysis. It also evaluates the user's physical and emotional state from the walking data to generate an overall health index.

[1839] 4. Methods for quantifying analysis results

[1840] The results of the analysis are quantified and evaluated based on anomaly detection criteria to determine whether there are signs of dementia.

[1841] 5. Means of notification

[1842] The cloud server generates notifications to users and their families based on the analysis results and their judgments. The notifications are sent via email or push notification and include the analysis results and diagnostic recommendations.

[1843] User and his / her family (User)

[1844] 1. Check the results

[1845] Users and their family members can check the analysis results by opening a dedicated smartphone app, which will display detailed messages such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[1846] 2. Collaboration with medical institutions

[1847] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[1848] Specific examples

[1849] Example 1: Acquiring and analyzing audio data

[1850] If elderly person A has a smartphone, the voice recognition module automatically records A's conversations. For example, if A says, "What was that thing yesterday?", the audio is recorded and sent to a cloud server. The server analyzes this data and detects the frequent occurrence of "this, that, that" to determine possible signs of dementia. The emotion engine also evaluates A's emotional state from the tone of voice and choice of words.

[1851] Example 2: Gait data collection and analysis

[1852] Similarly, when elderly person A goes for a walk every day, the built-in sensor in his smartphone records his walking pattern. For example, data such as extremely short strides or unsteady walking speed is collected. This data is also analyzed on the cloud server to evaluate the stability of his walking. The emotion engine then uses this data to comprehensively evaluate his physical and emotional state and generate an overall health index.

[1853] In this way, the system of the present invention supports early detection of dementia in the elderly and appropriate medical intervention, providing important information to users and their families. In addition, by combining it with an emotion engine, the system can also evaluate the user's emotional state, realizing comprehensive health management.

[1854] The processing flow will be explained below.

[1855] Step 1: Data Collection (Device)

[1856] The device will run a voice recognition module in the background and record the user's voice conversations.

[1857] Specific operation: The device will record audio every 30 seconds and save the recording data to the internal storage.

[1858] The device uses built-in sensors (accelerometer, gyro sensor, etc.) to record the user's walking data in real time.

[1859] Specific operation: The device captures the user's movements (e.g., walking, shaking) and saves the data in the specified format on the internal storage.

[1860] Step 2: Data encryption (device)

[1861] The device encrypts the collected conversation and walking data.

[1862] Specific operation: Encrypt the data using an encryption algorithm (e.g., AES-256) and store it in a temporary buffer.

[1863] Step 3: Send data (terminal)

[1864] The device transmits the encrypted conversation data and walking data to a cloud server.

[1865] Specific operation: Uploads encrypted data to a specific endpoint on a cloud server using HTTP or HTTPS protocols.

[1866] Step 4: Data reception and decryption (server)

[1867] The server receives the encrypted data sent from the terminal.

[1868] Specific operation: The server receives the data through the firewall and stores it in temporary storage.

[1869] The server decrypts the received data.

[1870] What it does: It uses a data decoding algorithm to return the data to its original form and stores it in a database for analysis.

[1871] Step 5: Analyzing conversation data (server)

[1872] The server analyzes the conversation data using natural language processing (NLP) algorithms.

[1873] Specific operation: The server converts the voice data into text data and detects frequent patterns of "this, that, that," jumps in vocabulary, and repetitions.

[1874] Step 6: Sentiment Analysis (Server)

[1875] The server uses an emotion engine to assess the emotional state from the conversation data.

[1876] How it works: The server analyzes the tone, pitch, speed, etc. of the voice and evaluates the user's emotions (happiness, sadness, anger, surprise, etc.).

[1877] Step 7: Analyzing gait data (server)

[1878] The server analyzes the walking data using a data analysis algorithm.

[1879] Specific operation: The server evaluates walking stability, stride length variations, speed changes, etc., and detects abnormal patterns.

[1880] Step 8: Overall Health Assessment (Server)

[1881] The results of the emotion engine are combined with the analysis of walking data to assess overall health.

[1882] Specific actions: Integrate the emotion analysis results and walking data analysis results to generate a comprehensive health index.

[1883] Step 9: Quantifying the analysis results (server)

[1884] The server quantifies the analysis results and evaluates them based on anomaly detection criteria.

[1885] Specific operation: The server statistically processes the analysis results and converts them into numbers based on a scoring system.

[1886] Step 10: Notification Generation and Sending (Server)

[1887] The server generates notifications to the user and their family based on the analysis results.

[1888] Specific behavior: The server generates a notification message and prepares it to be sent in the form of an email or push notification.

[1889] The server generates and sends notifications to the user and their family members.

[1890] Specific operation: The server sends a notification message using the SMTP protocol and notification service.

[1891] Step 11: Check the results (user)

[1892] Users and their family members open the app to check the analysis results.

[1893] What happens: The user taps on the notification on their smartphone to view detailed analysis results and recommendations.

[1894] Step 12: Collaboration with medical institutions (user)

[1895] Users and their families make medical appointments as needed.

[1896] Specific behavior: Click on a link within the app to access the medical institution's appointment page and complete the appointment.

[1897] Step 13: Continuous Data Collection (Device)

[1898] Your smartphone will continue to collect data and analyze it periodically.

[1899] Specific operations: The steps described above, from data collection to transmission, are continuously carried out to conduct long-term monitoring.

[1900] Example 2

[1901] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1902] In recent years, dementia and other health problems have been increasing among the elderly, making early detection and appropriate medical intervention important. However, it is difficult for elderly people themselves and their families to accurately grasp the early signs of dementia and their daily health status. Furthermore, conventional diagnostic methods require hospital visits and evaluations by specialists, which are time-consuming and labor-intensive. Therefore, there is a need for a system that can continuously monitor the health status of elderly people in their daily lives and detect abnormalities early.

[1903] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1904] In this invention, the server includes means for decoding the conversation data and walking data and analyzing them using a natural language processing algorithm and a data analysis algorithm, means for evaluating the user's emotional state using an emotion engine based on the analysis results, and means for quantifying the analysis results and emotion evaluation results and making a judgment based on anomaly detection criteria. This makes it possible to continuously monitor the health status of elderly people, detect early signs of dementia and other health abnormalities at an early stage, and provide the user and their family with prompt and appropriate information and recommendations for medical intervention.

[1905] "Elderly people's conversation data" is voice information of utterances and conversations that elderly people make in their daily lives.

[1906] "Elderly person walking data" refers to data such as walking patterns, speed, and balance when elderly people move around.

[1907] "Encryption" is the process of converting data content into a form that is unintelligible to third parties.

[1908] A "cloud server" is an external server system that stores, manages, and processes data via the Internet.

[1909] "Decryption" is the process of returning encrypted data to its original form.

[1910] A "natural language processing algorithm" is a computer program used to analyze and understand natural language in text or speech.

[1911] A "data analysis algorithm" is a computer program that analyzes collected data and detects specific patterns or anomalies.

[1912] An "emotion engine" is a computer program that evaluates and estimates a user's emotional state from voice and behavioral data.

[1913] "Anomaly detection" is the process of identifying data or patterns that deviate from normal conditions.

[1914] "User's family" refers to the close relatives of the elderly person using the system and people who provide care or nursing care.

[1915] "Notification means" refers to a means of transmitting information to the user and their family members about the analysis results and judgment results.

[1916] A "voice recognition device" is a device or software that converts speech into text data.

[1917] "Built-in sensors" are sensors such as accelerometers and gyro sensors built into the device.

[1918] The present invention is a system that collects and analyzes conversation data and walking data of elderly people and evaluates the user's emotional state using an emotion engine. This system mainly includes a smartphone, a cloud server, and the user and their family. As a specific embodiment, the detailed processing of the program is described below.

[1919] Smartphone (device)

[1920] The device is equipped with a voice recognition device and built-in sensors (acceleration sensor, gyro sensor, etc.), which collect conversation and walking data of the elderly.

[1921] 1. Data Collection Methods

[1922] The device uses a voice recognition device to record and save the elderly person's conversations in the background. For example, if Mr. A says, "I can't remember what happened yesterday," the device collects this voice data. At the same time, the device uses built-in sensors to record the elderly person's walking patterns (e.g., stride length, speed, and fluctuations in balance). Data is continuously collected as Mr. A goes for a walk.

[1923] 2. Data Encryption Methods

[1924] Collected speech and walking data is encrypted within the device using the AES-256 algorithm, a process that protects the privacy and security of the data.

[1925] 3. Data Transmission Method

[1926] The encrypted data is sent to the cloud server using the HTTPS protocol, and the device checks for a response from the server to confirm that the data was sent successfully.

[1927] Cloud Server (Server)

[1928] The server has high processing power and the ability to receive, decode, and analyze the transmitted data.

[1929] 1. Data Receiving Method

[1930] The server receives the encrypted data sent from the device and immediately decrypts it. The decrypted data is then separated into voice data and walking data.

[1931] 2. Data analysis methods

[1932] The decoded voice data is analyzed using natural language processing (NLP) algorithms to detect, for example, frequent use of "this," "that," "that" phrases, repetition, and jumps in content. Meanwhile, data analysis algorithms are used to evaluate walking stability, stride length variability, and changes in speed.

[1933] 3. Emotional assessment measures

[1934] The server uses an emotion engine to evaluate the user's emotional state based on the analysis results. For example, when Person A says, "I can't remember what happened yesterday," the emotion engine detects emotions such as sadness and anxiety. It also evaluates the user's physical and emotional state based on walking data.

[1935] 4. Methods for quantifying analysis results

[1936] The analysis results and emotion assessment results are quantified and judged based on anomaly detection criteria. Based on specific criteria, it is evaluated whether there are signs of dementia.

[1937] 5. Means of notification

[1938] The server generates a notification to the user and their family based on the analysis results and its judgment. The generated notification is sent via email or push notification to a dedicated app. For example, it may say, "Your father's recent conversation patterns show signs of dementia. We recommend that you see a specialist."

[1939] User and his / her family (User)

[1940] Users and their families can check the analysis results through a dedicated smartphone app and take appropriate measures if necessary.

[1941] 1. Check the results

[1942] Users and their families can check the analysis results using a dedicated smartphone app, which displays detailed messages to help detect abnormalities early.

[1943] 2. Collaboration with medical institutions

[1944] Users who receive the notification can use the link in the app to book an appointment at a corresponding medical institution, making it possible to book medical appointments smoothly.

[1945] Specific examples

[1946] Example 1: Acquiring and analyzing audio data

[1947] The device records an elderly person, Mr. A, saying, "I can't remember what happened yesterday," and sends the recording to a cloud server. The server analyzes this data and determines possible signs of dementia by detecting the frequent use of the words "this, that, that." The emotion engine evaluates Mr. A's emotional state based on his tone of voice and choice of words.

[1948] Example 2: Gait data collection and analysis

[1949] The device records A's walking patterns (e.g., stride length, speed, and instability) as he or she goes for a walk. This data is analyzed by a cloud server to evaluate the stability of the person's walking. An emotion engine also uses this data to evaluate the person's physical and emotional state and generate a comprehensive health index.

[1950] Prompt Sentence Examples

[1951] "Describe a system that collects speech and gait data from older adults and assesses their emotional state."

[1952] "Please detail the specific implementation of the system for early detection of dementia in the elderly and how it works."

[1953] "Please tell me the process for analyzing data collected using a smartphone on a cloud server to evaluate the user's health status."

[1954] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1955] Step 1: Data collection (device)

[1956] The device collects conversation data and walking data from the elderly. The voice recognition device records what the elderly person says and records their walking patterns using built-in sensors (acceleration sensor, gyro sensor). For example, if person A says, "I can't remember what happened yesterday," the voice recognition device records this speech. At the same time, the built-in sensor measures walking data such as stride length, speed, and balance fluctuations as person A goes for a walk. The input data is the elderly person's conversation voice and walking sensor data, and the output is the collected voice file and walking dataset.

[1957] Step 2: Data encryption (device)

[1958] The device encrypts the collected data using the AES-256 algorithm, which reduces the risk of unauthorized access to conversation data and walking data by third parties. For example, when Person A says, "I can't remember what happened yesterday," the device encrypts the data, and the walking data is encrypted as well. The input is the collected audio file and walking data set, and the output is the encrypted audio data and walking data.

[1959] Step 3: Send data (terminal)

[1960] The device sends the encrypted data to the cloud server using the HTTPS protocol. If the transmission is successful, the device waits for a response from the server. For example, when encrypted voice data and walking data are sent to the cloud server, a confirmation message is returned that the server has received them. The input is the encrypted voice data and walking data, and the output is the transmission status (success / failure).

[1961] Step 4: Data Reception and Decryption (Server)

[1962] The server receives the encrypted data sent from the terminal and decrypts it using the AES-256 algorithm. The decrypted data is divided into voice data and walking data. For example, the server obtains the voice data "I can't remember yesterday" and the corresponding walking data. The input is the encrypted voice data and walking data, and the output is the decrypted voice data and walking data.

[1963] Step 5: Data analysis (server)

[1964] The server analyzes the decoded voice data using a natural language processing algorithm (NLP). The walking data is evaluated using a data analysis algorithm. For example, the voice data is analyzed for the frequency of "this, that, that," and the walking data is evaluated for walking stability, stride length variation, and speed changes. The input is the decoded voice data and walking data, and the output is the analysis results.

[1965] Step 6: Emotion Evaluation (Server)

[1966] The server uses an emotion engine to evaluate the user's emotional state based on the analysis results. For example, if Person A says, "I can't remember what happened yesterday," the emotion engine detects sadness or anxiety. Physical and emotional states are also evaluated from walking data. The input is the analysis results, and the output is the emotion evaluation results.

[1967] Step 7: Quantifying the analysis results (server)

[1968] The server quantifies the analysis results and emotion evaluation results and makes a judgment based on anomaly detection criteria. For example, if the frequency of "this, that, that" in conversation data is higher than normal, it is quantified as a sign of dementia. The input is the emotion evaluation results and analysis results, and the output is a quantified judgment result.

[1969] Step 8: Notification Generation and Sending (Server)

[1970] The server generates a notification based on the assessment result and sends it to the user and their family. The notification is provided as an email or a push notification to a dedicated app. For example, a notification may be generated stating, "Your father's recent conversation patterns show signs of dementia." The input is the quantified assessment result, and the output is the generated notification message.

[1971] Step 9: Check the results (user)

[1972] The user checks the analysis results using a dedicated smartphone app. Detailed messages are displayed to support early detection of abnormalities. For example, when the user opens the app, a message such as "Your father's recent behavioral patterns show signs of dementia" is displayed. The input is the notification message, and the output is the displayed analysis results.

[1973] Step 10: Collaboration with medical institutions (user)

[1974] After receiving the notification, the user can make an appointment with a medical institution through a link in the app. For example, the user can click the link in the app to access the nearest medical institution's appointment system and complete the appointment. The input is the appointment link in the app, and the output is a notification that the appointment has been completed.

[1975] (Application example 2)

[1976] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1977] Remotely monitoring the health and safety of elderly people in real time is an important challenge for caregivers and families. However, current technology lacks a system that can effectively collect elderly people's conversation and movement information, analyze that data, and detect abnormalities. Furthermore, there is a lack of systems that can promptly notify family members or caregivers when an abnormality is detected. Therefore, an effective solution for properly managing the health and safety of elderly people is needed.

[1978] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting conversation information of the elderly, means for collecting movement information of the elderly, means for encrypting the conversation information and movement information and transmitting them to a cloud server, means for decrypting the received conversation information and movement information in the cloud server and analyzing them using an artificial intelligence model, means for quantifying the analysis results and making a determination based on anomaly detection criteria, means for notifying the user and the user's family of the determination results, and additional means for notifying the user's family in real time if an abnormality is detected based on the analysis results. This makes it possible to continuously monitor the user's condition and immediately notify family members or caregivers if an abnormality is detected.

[1979] "Elderly people's conversation information" refers to audio data of elderly people speaking, including the content, choice of words, tone of voice, etc.

[1980] "Elderly person movement information" is data that indicates the walking and movement patterns of elderly people, including stride length, speed, and changes in balance.

[1981] "Encryption" is a process of converting data using a specific algorithm to make it into a format that cannot be easily understood by third parties in order to protect the content of the data.

[1982] A "cloud server" is a remote server that stores, manages, and analyzes data via the Internet, and can be used without the user having to own physical server equipment.

[1983] An "artificial intelligence model" is an algorithm or system that has the ability to analyze large amounts of data and identify patterns and anomalies from them.

[1984] "Quantifying analytical results" is the process of expressing the results of data analysis in numerical form so that they can be objectively evaluated.

[1985] "Anomaly detection criteria" refers to quantitative or qualitative thresholds or conditions for determining anomalies as a result of data analysis.

[1986] A "notification" is the act of communicating information about a certain event, situation, or when certain conditions are met to a specific recipient.

[1987] "Real time" refers to processing or responding to an event at the moment or almost at the same time as the event occurs.

[1988] A "voice recognition module" is a device or software that receives voice input and converts it into a readable form and further digital data that can be analyzed.

[1989] "Built-in sensors" are sensors that are built into a device and are typically used to measure acceleration, gyroscope, location information, etc.

[1990] "Background recording" refers to the continuous, unobtrusive collection of necessary data while the user is using the device.

[1991] This invention is a system that accumulates and analyzes conversation and movement information of elderly people, and further evaluates the emotional state of the user using an emotion engine. The specific implementation method and procedures for this system are described in detail below.

[1992] System configuration:

[1993] Device (smartphone):

[1994] 1. Data collection methods:

[1995] Speech Recognition Module:

[1996] The device is equipped with a voice recognition module that automatically records conversations between elderly people. For example, if an elderly person says, "What was that thing that happened yesterday?", the audio will be recorded.

[1997] Built-in sensors:

[1998] Built-in sensors (accelerometer, gyro sensor, etc.) are used to record elderly people's walking data (step length, speed, balance fluctuations, etc.). For example, when the stride length becomes extremely short, this data is collected.

[1999] 2. Data Encryption Methods:

[2000] Collected conversation and movement information is encrypted within the device, and this encryption process protects the privacy and security of the data.

[2001] 3. Means of data transmission:

[2002] The encrypted data is sent to the cloud server using the HTTP or HTTPS protocol.

[2003] Server (Cloud Server):

[2004] 1. Data receiving means:

[2005] The cloud server receives the encrypted data sent from the terminal and decrypts the data.

[2006] 2. Data analysis methods:

[2007] The received data is analyzed using artificial intelligence models. Natural language processing (NLP) techniques are used on the conversational information to examine specific language patterns, such as frequent use of "this" and "that," or repetition.

[2008] Data analysis algorithms are used on the movement information to evaluate walking stability and changes in speed.

[2009] 3. Emotion Engine:

[2010] The emotion engine evaluates the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) from conversation data, and also evaluates the user's physical and emotional state from walking data to generate a health index.

[2011] 4. Methods for quantifying analysis results:

[2012] The analysis results are quantified and evaluated based on anomaly detection criteria, which determines whether the elderly person is showing signs of dementia.

[2013] 5. Means of notification:

[2014] The results are then sent to the user and their family via email or push notification, and include the analysis results and diagnostic recommendations.

[2015] User and his / her family (User):

[2016] 1. Check the results:

[2017] Users and their family members can open the app to check the analysis results, which may include a message such as, "Your father's recent behavioral patterns show signs of dementia. We recommend that you see a specialist."

[2018] 2. Collaboration with medical institutions:

[2019] Users and their families who receive the notification can make an appointment at a corresponding medical institution via a link within the app.

[2020] Examples and prompts:

[2021] Examples:

[2022] When elderly person A is spending time at home, conversation information (e.g., "I forgot what I did yesterday") and movement information (e.g., his stride suddenly shortened) are collected and analyzed on a cloud server. If an abnormality is detected, a push notification is sent to his family saying, "Abnormalities have been observed in A's behavior. Please check."

[2023] Example prompt sentence:

[2024] Write a Python program that collects and encrypts conversation and movement data of elderly people and sends it to a cloud server. The cloud server decrypts the data, analyzes it using an AI engine, and notifies family members if an abnormality is detected. Please include specific steps for data collection, encryption, and transmission to create a system.

[2025] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2026] Step 1:

[2027] The device collects voice and walking data:

[2028] The elderly person's conversation information is collected by a voice recognition module, and movement information is recorded using built-in sensors (accelerometer and gyro sensor). The input is the elderly person's conversation and walking patterns, and the output is a voice data file and a walking data file, respectively. Specifically, the device periodically collects voice and walking data in the background and stores it in temporary storage.

[2029] Step 2:

[2030] Encrypt data collected by the device:

[2031] Voice and walking data are encrypted. The input is the collected raw data, and the output is an encrypted data file. Specifically, the device generates an encryption key (or uses an existing key) and encrypts the data using the Fernet algorithm, etc. This protects privacy.

[2032] Step 3:

[2033] The device sends the encrypted data to the cloud server:

[2034] The encrypted data is sent to the cloud server using the HTTP or HTTPS protocol. The input is the encrypted data file, and the output is a transmission status code. Specifically, the device periodically sends an HTTP POST request to the cloud server endpoint to upload the data.

[2035] Step 4:

[2036] The server decrypts the received data:

[2037] The cloud server receives and decrypts the encrypted data sent from the device. The input is the encrypted data and the encryption key, and the output is the decrypted data. Specifically, the server decrypts the data that arrives and converts it into an analyzable format.

[2038] Step 5:

[2039] The server parses the data:

[2040] The server inputs the decoded conversation data and walking data into an artificial intelligence model for analysis. The input is the decoded data, and the output is the analysis results. Specifically, the server analyzes the conversation data using a natural language processing (NLP) algorithm and the walking data using a data analysis algorithm. For example, it evaluates specific language patterns and walking stability.

[2041] Step 6:

[2042] The server uses the emotion engine to assess the emotional state:

[2043] The server uses an emotion engine to evaluate the user's emotional state based on conversation data. It also evaluates emotions and physical condition from walking data. The input is analyzed conversation data and walking data, and the output is the emotion evaluation result. Specifically, the emotion engine analyzes the user's emotions based on their tone of voice and walking patterns, and generates a ...

Claims

1. A means of collecting conversation data from elderly people; A means of collecting gait data for older adults; means for encrypting the conversation data and walking data and transmitting the encrypted data to a cloud server; In the cloud server, a means for decoding the received conversation data and walking data and analyzing them using an artificial intelligence model; A means for quantifying the analysis results and making a determination based on an anomaly detection standard; A means for notifying the user and the user's family of the result of the determination; A system including:

2. The system of claim 1 , wherein the notification means generates and transmits a notification message including the analysis results and diagnostic recommendations to the user and family members.

3. 10. The system of claim 1, wherein the data collection means includes a voice recognition module and built-in sensors to periodically record voice conversation and gait data in the background.

Images