System

Abstract

To provide a system.SOLUTION: A system comprises: means for performing conversation with a user by voice input, upon voice input from the user; means for analyzing a state of the conversation with the user, and detecting an abnormality in a daily life of the user; means for reporting the detected abnormality to a contact address which is set in advance; means for analyzing in real time, a feeling of the user, and detecting a change of the feeling of the user; and means for adjusting a content of the conversation according to the feeling of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In an aging society, issues such as isolation among people living alone and early detection of dementia are becoming issues. There is also a need for appropriate advice on health management and the establishment of a monitoring system in cooperation with the local community. [Means for solving the problem]

[0005] This invention enables conversations via voice input, contributing to the user's mental stability and brain activation. It also analyzes the state of the conversation, detects abnormalities such as dementia trends, and notifies hospitals, facilities, and families of detected abnormalities. Furthermore, by linking with IoT and healthcare systems, it provides advice on improving health, helping to realize a society where people are monitored closely through collaboration with the local community. This will solve issues such as preventing isolation among people living alone, early detection of dementia, supporting health management, and strengthening collaboration with the local community. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15]FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention uses speech recognition technology to converse with the user. This conversation contributes to the user's mental stability and brain activity. Specifically, it receives voice input from the user, converts it into text, and generates an appropriate response. This response is then conveyed to the user using speech synthesis technology.

[0029] "Example 2"

[0030] The system of the present invention also analyzes the state of conversation. This analysis uses natural language processing and machine learning technologies. Specifically, it detects abnormalities such as dementia trends from the content, speech patterns, and frequency of the user's speech. Detected abnormalities are notified to hospitals, facilities, family members, etc.

[0031] "Example 3"

[0032] Furthermore, the system of the present invention provides advice on improving health by linking with IoT and healthcare systems. Specifically, it collects and analyzes the user's lifestyle and health data. Based on the results, it generates appropriate advice and delivers it to the user.

[0033] "Example 4"

[0034] Furthermore, the system of the present invention supports the realization of a society where people are monitored in cooperation with the local community. Specifically, the system shares information about the user's living situation and health status with relevant people in the local community. This prevents users from becoming isolated and builds a monitoring system that involves the entire community.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: Receive speech input from the user. This speech input is part of a conversation between the user and the system.

[0038] Step 2: Convert the received voice input into text using voice recognition technology.

[0039] Step 3: Analyze the converted text and generate an appropriate response, using natural language processing and machine learning techniques.

[0040] Step 4: The generated response is converted into speech and conveyed to the user. This speech conversion is achieved using speech synthesis technology.

[0041] "Example 2"

[0042] Step 1: Analyze the state of the conversation with the user. This analysis uses natural language processing and machine learning technologies.

[0043] Step 2: Detect abnormalities such as dementia trends from the analysis results.

[0044] Step 3: Notify the hospital, facility, family, etc. of the detected abnormality. This notification is made via communication methods such as email, messaging apps, and telephone.

[0045] "Example 3"

[0046] Step 1: Collect user lifestyle and health data from IoT and healthcare systems.

[0047] Step 2: Analyze the collected data. This analysis uses statistical methods and machine learning techniques.

[0048] Step 3: Based on the analysis results, appropriate advice for improving health is generated.

[0049] Step 4: The generated advice is communicated to the user using voice output, text output, or a graphical display.

[0050] "Example 4"

[0051] Step 1: Share your living situation and health status with relevant people in your community via email, messaging apps, phone calls, or other communication methods.

[0052] Step 2: Collaborate with relevant parties in the local community to establish a system for monitoring users. This collaboration will involve local public institutions, volunteer groups, and local residents.

[0053] Example 1

[0054] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0055] In modern society, there is a demand for methods to stabilize users' mental health and stimulate their brains. It is also important to detect abnormalities such as dementia early and take appropriate measures. However, systems to solve these issues have not yet been fully developed. Furthermore, there is a need for advice on improving health and for a society that monitors users through cooperation with the local community.

[0056] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation, a means for converting the user's voice into text using voice recognition technology, a means for generating an appropriate response using a generative AI model, a means for converting the generated response into voice using text-to-speech conversion technology, a means for analyzing the state of the conversation and detecting abnormalities such as dementia tendencies, a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, a means for providing advice on improving health through collaboration with IoT and healthcare systems, and a means for supporting the realization of a monitoring society through collaboration with the local community. This enables early detection of abnormalities and appropriate responses while promoting the user's mental stability and brain activation.

[0057] "Voice input" is a technology that captures the voice a user speaks through a microphone as digital data.

[0058] "Mental stability" refers to maintaining or improving the user's psychological stability.

[0059] "Brain activation" refers to stimulating the user's brain functions and improving their cognitive and thinking abilities.

[0060] "Speech recognition technology" is a technology that converts voice data into text data.

[0061] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input.

[0062] "Text-to-speech conversion technology" is a technology that converts text data into voice data.

[0063] "Means for analyzing the state of conversation" refers to technology that analyzes the content and patterns of user speech and detects abnormalities.

[0064] "Dementia tendency" refers to a state in which a user shows early symptoms of dementia.

[0065] "Means for detecting anomalies" refers to technology that discovers anomalies from users' statements and behavior.

[0066] "Means of notifying hospitals, facilities, families, etc." refers to technology that notifies relevant parties of detected abnormalities.

[0067] "IoT" is a technology that allows various devices to connect via the Internet.

[0068] A "healthcare system" is a system for managing and improving the health status of users.

[0069] "Health Improvement Advice" refers to providing advice to improve a user's health.

[0070] "Collaboration with the local community" refers to working with the local community in which the user lives to monitor and provide support.

[0071] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[0072] This invention is a system for promoting mental stability and brain activation for users, and is realized using voice input, voice recognition, generative AI models, and text-to-speech conversion technologies. Specific embodiments of this system are described below.

[0073] Hardware and Software Configuration

[0074] 1. Voice Input

[0075] The user inputs voice through a microphone. The device captures this voice and sends it to the server as digital data. The hardware used is a microphone and a voice capture device.

[0076] 2. Voice Recognition

[0077] The server uses speech recognition technology to convert the received voice data into text data. Specifically, it uses a speech recognition API (for example, a general speech recognition service). This API provides highly accurate speech recognition and accurately converts the user's speech into text.

[0078] 3. Response Generation

[0079] The server uses a generative AI model (e.g., a general generative AI model) to generate appropriate responses to the user's textual utterances. The model generates responses that include content that contributes to the user's mental stability and brain activity.

[0080] 4. Text to speech conversion

[0081] The generated response text is converted into audio data using text-to-speech conversion technology. Specifically, a speech synthesis API (e.g., a general speech synthesis service) is used. This API provides natural speech synthesis and generates audio that is easy for the user to understand.

[0082] 5. Providing voice responses

[0083] The terminal receives the voice data sent from the server and provides a response to the user through the speaker, allowing the user to continue the conversation with the system. The hardware used is a speaker and an audio playback device.

[0084] Specific examples

[0085] Example 1: When a user is stressed

[0086] 1. User: "I'm really stressed out today."

[0087] 2. Device: Captures audio with a microphone and sends it to the server.

[0088] 3. Server: Uses speech recognition API to convert "I'm very stressed today" into text.

[0089] 4. Server: Uses a generative AI model to generate a response such as "That's tough. Try taking a deep breath to relax."

[0090] 5. Server: Convert the response into speech using a speech synthesis API.

[0091] 6. Device: Through the speaker, the user is told, "That's tough. Try taking a deep breath to relax."

[0092] Example 2: When the user wants to stimulate their brain

[0093] 1. User: "I want to learn something new today."

[0094] 2. Device: Captures audio with a microphone and sends it to the server.

[0095] 3. Server: Use speech recognition API to convert "I want to learn something new today" into text.

[0096] 4. Server: Uses a generative AI model to generate a response like, "That's great! Want to talk about some recent advances in science and technology?"

[0097] 5. Server: Convert the response into speech using a speech synthesis API.

[0098] 6. Terminal: Through the speaker, the user is told, "That's great! Shall I tell you about some recent advances in science and technology?"

[0099] Prompt Sentence Examples

[0100] "How do I respond if a user is stressed?"

[0101] "When a user says they want to learn something new, how do I respond?"

[0102] This system combines voice recognition technology with generative AI models to enable natural conversations with users, contributing to mental stability and brain activation.

[0103] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0104] Step 1:

[0105] The user inputs voice through a microphone. The device captures this voice and stores it as digital data. Specifically, the device's microphone picks up the user's voice and stores it as voice data. This data is sent to the server in real time. The input is the user's voice, and the output is digital voice data.

[0106] Step 2:

[0107] The server uses voice recognition technology to convert the received voice data into text data. Specifically, the server sends the voice data to a voice recognition API, which analyzes the voice and converts it into text. The converted text is returned to the server and passed on to the next processing step. The input is digital voice data, and the output is text data.

[0108] Step 3:

[0109] The server uses a generative AI model to generate appropriate responses to the user's textual utterances. Specifically, the server inputs text data into the generative AI model, which then generates a response based on the user's utterances. This response includes content that contributes to the user's mental stability and brain activation. The input is text data, and the output is a response text.

[0110] Step 4:

[0111] The generated response text is converted into audio data using text-to-speech technology. Specifically, the server sends the response text to a speech synthesis API, which analyzes the text and generates natural-sounding audio. This audio data is returned to the server and passed on to the next processing step. The input is the response text, and the output is audio data.

[0112] Step 5:

[0113] The terminal receives the voice data sent from the server and provides a response to the user through the speaker. Specifically, the terminal receives the voice data and plays it through the speaker, allowing the user to continue a conversation with the system. The input is the voice data, and the output is a voice response to the user.

[0114] (Application example 1)

[0115] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] In modern society, there is a demand for methods to stabilize users' mental health and stimulate their brains. It is also important to detect abnormalities such as dementia early and take appropriate measures. Furthermore, there is a demand to improve the quality of life by enabling users to easily use food delivery services. However, it is difficult to realize these functions in a single system, and an efficient solution is needed.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0118] In this invention, the server includes: a means for conducting a conversation via voice input, contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia; a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; a means for providing health improvement advice through collaboration with IoT and healthcare systems; a means for supporting the realization of a watchful society through collaboration with the local community; a means for converting voice input into text, generating an appropriate response, and conveying it to the user using speech synthesis technology; a means for placing a food delivery order based on the user's voice input; and a means for generating an appropriate response based on the user's order using a generative AI model. This enables the early detection of abnormalities such as dementia and appropriate responses to be taken while promoting the user's mental stability and brain activation. It also allows users to easily use food delivery services, improving their quality of life.

[0119] "Voice input" refers to a user inputting voice through a device such as a microphone.

[0120] "Conversation" refers to the exchange of information between a user and a system via voice.

[0121] "Mental stability" refers to maintaining or improving the user's psychological stability.

[0122] "Brain activation" means stimulating and activating the user's brain functions.

[0123] "Conversational state" refers to the content and quality of the conversation between the user and the system.

[0124] "Dementia tendency" refers to a state in which the user may develop dementia.

[0125] An "abnormality" is a phenomenon or symptom that deviates from the normal state.

[0126] "Hospitals, facilities, families, etc." refers to medical institutions, nursing homes, families, etc. that are involved in the user's health condition.

[0127] "Notification" is the act of conveying specific information to others.

[0128] "IoT" is a technology that connects various devices to each other via the Internet.

[0129] A "healthcare system" is a system that supports users in managing their health.

[0130] "Health Improvement Advice" refers to advice or suggestions for improving the user's health.

[0131] "Local community" refers to the local community in which the user lives.

[0132] A "watchful society" is a system in which the entire local community watches over the safety and health of users.

[0133] "Speech synthesis technology" refers to the technology that converts text data into speech.

[0134] "Food delivery" refers to a service that delivers meals ordered by users to a specified location.

[0135] A "generative AI model" is a model that uses artificial intelligence to generate responses or results for a specific task.

[0136] An "appropriate response" is an appropriate and helpful reply to the user's input.

[0137] A system for implementing this invention provides a function that contributes to the user's mental stability and brain activation by conducting conversations via voice input. The system realizes interaction with the user using voice recognition technology, voice synthesis technology, and generative AI models.

[0138] Hardware and Software Configuration

[0139] The system uses the following hardware and software:

[0140] Hardware: microphone, speakers, smartphone or computer

[0141] Software: Speech recognition library, text-to-speech library, generative AI model

[0142] Processing flow

[0143] 1. Acquiring voice input:

[0144] The user provides voice input through a microphone, which is converted to text using a speech recognition library.

[0145] 2. Response generation using generative AI models:

[0146] The converted text is input to a generative AI model (e.g., OpenAI's API) to generate an appropriate response. An example of a prompt sentence is, "The user wants to order food delivery. Please generate an appropriate response based on the following order details: I would like to order pizza."

[0147] 3. Providing a response via speech synthesis:

[0148] The generated response is converted into audio using a text-to-speech library and provided to the user through the speaker.

[0149] Specific examples

[0150] For example, if a user says "I want to order a pizza," the system will do the following:

[0151] 1. The user's voice input "I would like to order pizza" is picked up by the microphone.

[0152] 2. The speech recognition library converts the speech into text: "I would like to order a pizza."

[0153] 3. The generative AI model generates a response based on the prompt: "The user wants to order food delivery. Please generate an appropriate response based on the following order: I would like to order pizza."

[0154] 4. The generated response, "What type and size of pizza do you want?" is converted into speech by the speech library and provided to the user through the speaker.

[0155] In this way, users can place food delivery orders through voice input, and the system will provide appropriate responses, contributing to the user's mental well-being and brain activity.

[0156] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0157] Step 1:

[0158] The user inputs voice through the microphone. The input is the user's voice, and the output is voice data. The terminal acquires this voice data and passes it to the next step.

[0159] Step 2:

[0160] The device converts voice data into text using a speech recognition library. The input is voice data and the output is text data. Specifically, the speech recognition engine analyzes the voice waveform and generates the corresponding text.

[0161] Step 3:

[0162] The server uses a generative AI model to generate an appropriate response based on the text data. The input is text data, and the output is response text. Specifically, the text data is input to the generative AI model as a prompt sentence, and the response text is obtained. An example of a prompt sentence is, "The user says they want to order food delivery. Please generate an appropriate response based on the following order details: I would like to order pizza."

[0163] Step 4:

[0164] The server converts the response text into speech using a text-to-speech library. The input is the response text and the output is audio data. Specifically, the response text is input to a speech synthesis engine, which generates a corresponding audio file.

[0165] Step 5:

[0166] The terminal provides the generated voice data to the user through a speaker. The input is the voice data, and the output is the voice that the user hears. Specifically, the terminal plays the voice file and conveys the response to the user.

[0167] In this way, users can place food delivery orders through voice input, and the system will provide appropriate responses, contributing to the user's mental well-being and brain activity.

[0168] Example 2

[0169] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0170] In an aging society, early detection of dementia and appropriate treatment are important. However, early detection is difficult because the signs of dementia are often overlooked in daily life. There is also a lack of effective means to stabilize users' mental health and activate their brains. Furthermore, there is a need for a system that can quickly and appropriately notify users when an abnormality is detected.

[0171] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0172] In this invention, the server includes a means for conducting a conversation via voice input, contributing to the user's mental stability and brain activation, a means for analyzing the state of the conversation and converting the user's speech into text data using natural language processing technology, a means for preprocessing the converted text data and inputting it into a machine learning model to analyze the user's speech patterns and frequency, a means for detecting abnormalities based on the analysis results, a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, a means for providing advice on improving health by linking with IoT and healthcare systems, and a means for supporting the realization of a monitoring society by linking with the local community, thereby enabling early detection of dementia and appropriate treatment.

[0173] "Voice input" is a technology that collects the user's spoken voice as digital data.

[0174] "Mental stability" refers to maintaining or improving the user's psychological stability.

[0175] "Brain activation" refers to stimulating the user's brain functions and improving cognitive ability and memory.

[0176] "Conversational state" refers to the overall state of a user's speech, including their speech content, speech patterns, and speech frequency.

[0177] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0178] "Text data" refers to data obtained by converting voice data into character information.

[0179] "Preprocessing" refers to the process of cleaning and normalizing data before inputting it into a machine learning model.

[0180] A "machine learning model" refers to an algorithm that learns patterns based on data and makes predictions and classifications.

[0181] "Speech patterns" refer to characteristic tendencies and repetitions in a user's speech.

[0182] "Utterance frequency" refers to the number of times a user speaks within a certain period of time.

[0183] "Abnormal" refers to deviations from normal speech patterns or frequencies, including signs of dementia.

[0184] "Notification" refers to the means of communicating information to inform relevant parties of a detected abnormality.

[0185] "IoT" refers to technology that connects physical devices and systems to each other through the Internet.

[0186] "Healthcare system" refers to a system for providing health management and medical services.

[0187] "Health Improvement Advice" refers to specific suggestions or instructions to improve a user's health.

[0188] "Community" refers to the geographic or social community to which a User belongs.

[0189] A "watchful society" refers to efforts by the entire local community to support the elderly and vulnerable and ensure their safety.

[0190] The present invention provides a system that collects user utterance data and analyzes the state of the conversation using natural language processing technology and machine learning technology. Specific embodiments of this system are described below.

[0191] First, a user uses a device (such as a smartphone or a PC with a microphone) to have a conversation. The device records the user's speech in real time and transmits the audio data to a server. The server then converts the received audio data into text data using speech recognition software (e.g., Google® Cloud Speech-to-Text API).

[0192] The server then preprocesses the converted text data, which includes removing unnecessary spaces and special characters, normalizing words (e.g., "run" -> "run"), etc. The preprocessed text data is then input to a machine learning model using a machine learning framework (e.g., TENSORFLOW® or PyTorch).

[0193] The server inputs the preprocessed text data into a machine learning model to analyze the user's speech patterns and frequency. This analysis detects abnormalities in the content, patterns, and frequency of the user's speech. For example, if the user's speech suddenly decreases or they start using certain words or phrases more frequently, this could be a sign of an abnormality, such as dementia.

[0194] Any detected abnormalities are notified to the hospital, facility, or family member via a notification service such as a cloud-based communication service. The notification includes details of the abnormality and recommended countermeasures.

[0195] As a concrete example, the following prompt sentence can be input to a generative AI model:

[0196] Example prompt sentence:

[0197] "A 70-year-old male user has recently started using certain words more frequently in conversations. The frequency of his speech has also decreased. This user's conversation data will be analyzed, and if any abnormalities are detected, his family will be notified."

[0198] Based on this prompt, the generative AI model generates the following output:

[0199] Example output of a generative AI model:

[0200] "Analysis of the user's conversation data has confirmed that certain words are being used frequently. In addition, the frequency of speech has decreased, suggesting a tendency toward dementia. This abnormality will be notified to the user's family."

[0201] In this way, the system can analyze the user's conversation state in detail, detect abnormalities, and provide appropriate notifications.

[0202] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0203] Step 1:

[0204] The user uses the terminal to have a conversation, and the terminal records the user's speech in real time and transmits the audio data to the server.

[0205] Input: User's speech

[0206] Output: Audio data sent to the server

[0207] Step 2:

[0208] The server converts the received voice data into text data using voice recognition software, for example, the Google Cloud Speech-to-Text API.

[0209] Input: Audio data

[0210] Output: Text data

[0211] Step 3:

[0212] The server preprocesses the converted text data, which includes removing unnecessary spaces and special characters, and normalizing words.

[0213] Input: Text data

[0214] Output: Preprocessed text data

[0215] Step 4:

[0216] The server then inputs the preprocessed text data into a machine learning model to analyze the user's speech patterns and frequency, using, for example, TensorFlow or PyTorch.

[0217] Input: Preprocessed text data

[0218] Output: Analysis of speech patterns and frequency

[0219] Step 5:

[0220] The server detects anomalies based on the output of the machine learning model. For example, if a user's speech suddenly decreases or they start using certain words or phrases more frequently, this could be a sign of dementia or other abnormalities.

[0221] Input: Analysis results of speech patterns and frequency

[0222] Output: Anomaly detection results

[0223] Step 6:

[0224] The server notifies the hospital, facility, or family of the detected abnormality using a notification service, such as a cloud-based communication service. The notification includes details of the abnormality and recommended countermeasures.

[0225] Input: Anomaly detection result

[0226] Output: Notification message

[0227] In this way, the system can analyze the user's conversation state in detail, detect abnormalities, and provide appropriate notifications.

[0228] (Application example 2)

[0229] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0230] In modern society, managing dementia and mental health conditions among the elderly is an important issue. In particular, early detection of dementia and appropriate responses are required, but this is difficult to achieve in daily life. Also, a method is needed to quickly notify appropriate institutions and family members when an abnormality is detected. Furthermore, it is important to safely store user conversation data so that it can be analyzed later.

[0231] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia tendencies; a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; a means for providing health improvement advice through collaboration with IoT and healthcare systems; a means for supporting the realization of a watchful society through collaboration with the local community; a means for monitoring the user's conversation in real time and analyzing the content, patterns, and frequency of the speech using natural language processing technology and machine learning technology; a means for sending notifications to pre-registered hospitals, facilities, and family members when an abnormality is detected; and a means for securely storing the user's conversation data so that it can be analyzed later. This enables management of the user's mental health status, early detection of dementia, and prompt response.

[0232] "Voice input" is a technology in which the user's voice is captured as digital data and the system analyzes that voice.

[0233] "Mental stability" is a state that aims to maintain the user's psychological stability and reduce stress and anxiety.

[0234] "Brain activation" is a state that aims to stimulate the user's brain functions and improve cognitive function and memory.

[0235] "Conversation state" refers to comprehensive information such as the content of a user's speech, speech patterns, and frequency of speech.

[0236] "Dementia trends" are signs or patterns that indicate whether a user is at risk of developing dementia.

[0237] "Abnormal" refers to user behavior or speech that deviates from normal, especially health and safety issues.

[0238] "Notification" refers to the act of transmitting information to pre-registered hospitals, facilities, family members, etc. when an abnormality is detected.

[0239] "IoT" is a technology that allows various devices to communicate with each other and exchange data via the Internet.

[0240] A "healthcare system" is a system for managing and improving a user's health status.

[0241] "Health Improvement Advice" refers to specific suggestions and guidance for improving the user's health.

[0242] "Local community" refers to the local community and social environment in which the user lives.

[0243] A "watchful society" refers to a society in which the entire local community supports the elderly and vulnerable and provides an environment in which they can live in peace of mind.

[0244] "Real-time monitoring" is a technology that monitors user conversations and behavior in real time and analyzes the data instantly.

[0245] "Natural language processing technology" is a technology that allows computers to understand and analyze human language.

[0246] "Machine learning technology" is a technology that allows computers to learn from data and make predictions and classifications.

[0247] "Speech content" refers to the specific words and sentences spoken by the user.

[0248] "Speech patterns" refer to the way a user speaks and the way they use words.

[0249] "Speech frequency" is an indicator of how often a user speaks.

[0250] "Data storage" is the act of safely storing acquired data so that it can be accessed later.

[0251] A system for implementing this invention has the following configuration: First, a user engages in conversation using a smartphone or other voice input device. The voice input device captures the user's speech in real time and transmits the voice data to a server.

[0252] When the server receives the voice data, it first converts it into text using speech recognition software (e.g., a speech recognition library). This text data is then analyzed using natural language processing techniques (e.g., NLTK or spaCy) and machine learning techniques (e.g., scikit-learn). As a result of the analysis, the content of the user's speech, speech patterns, and frequency of speech are extracted.

[0253] Next, the server uses a machine learning model (e.g., dementia detection model) to detect abnormalities, such as dementia trends, from the extracted data. If an abnormality is detected, the server sends a notification to pre-registered hospitals, facilities, and family members. This notification is sent via communication methods such as email or SMS.

[0254] Additionally, the server securely stores the user's conversation data for later analysis. This data is stored using a database system (e.g., MySQL (registered trademark) or PostgreSQL). The stored data can be used for later reanalysis and long-term health management.

[0255] For example, if a user says, "I've been getting more and more forgetful lately," the voice input device captures the utterance and sends it to the server. The server converts the speech into text and uses a machine learning model to detect anomalies. If an anomaly is detected, the server sends a notification to pre-registered family members saying, "An anomaly has been detected: I've been getting more and more forgetful lately."

[0256] Example prompt sentence:

[0257] Develop an application that monitors user conversations in real time and analyzes the content, patterns, and frequency of speech using natural language processing and machine learning technologies. Include a function to send notifications to pre-registered hospitals, facilities, and family members if an anomaly is detected.

[0258] In this way, the invention enables users to manage their mental health and detect dementia early and respond quickly.

[0259] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0260] Step 1:

[0261] A user uses a smartphone or other voice input device to have a conversation. The voice input device captures the user's speech in real time and sends the voice data to a server. The input is the user's voice, and the output is the voice data sent to the server.

[0262] Step 2:

[0263] When the server receives the voice data, it uses voice recognition software (e.g., a voice recognition library) to convert the voice data into text data. The input is voice data, and the output is text data. Specifically, the server calls the voice recognition library, analyzes the voice data, and converts it into text.

[0264] Step 3:

[0265] The server analyzes the text data using natural language processing techniques (e.g., NLTK and spaCy) and machine learning techniques (e.g., scikit-learn). The input is text data, and the output is the analysis results, such as the content of the utterance, speech patterns, and frequency of the utterance. Specifically, the server tokenizes the text data and analyzes the meaning and context of each token.

[0266] Step 4:

[0267] The server uses a machine learning model (e.g., dementia detection model) to detect abnormalities such as dementia trends from the analysis results. The input is the analysis results, and the output is data indicating whether or not there are abnormalities. Specifically, the server inputs the analysis results into the machine learning model and predicts whether or not there are abnormalities.

[0268] Step 5:

[0269] If an abnormality is detected, the server sends a notification to pre-registered hospitals, facilities, and family members. The input is data indicating whether or not an abnormality exists, and the output is a notification message. Specifically, the server generates a notification message and sends it using a communication method such as email or SMS.

[0270] Step 6:

[0271] The server securely stores user conversation data for later analysis. The input is the voice data and the analysis results, and the output is data stored in a database. Specifically, the server stores the data using a database system (e.g., MySQL or PostgreSQL).

[0272] Step 7:

[0273] The stored data will be used for later reanalysis and long-term health management. The input is the stored data, and the output is the reanalysis results and information for health management. Specifically, the server reanalyzes the stored data and provides feedback to the user or medical institution as needed.

[0274] Example 3

[0275] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0276] In modern society, there are a wide variety of systems that support users' health management and mental well-being. However, these systems are specialized in individual functions and lack comprehensive health management and collaboration with the local community. Furthermore, there are limited systems that effectively collect and analyze users' lifestyle and health data and provide appropriate advice. Furthermore, there are insufficient means to prevent users from becoming isolated and to establish a community-wide monitoring system. To address these issues, a system that utilizes IoT devices and generative AI models to provide integrated support for user health management and community monitoring is needed.

[0277] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0278] In this invention, the server includes: means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; means for analyzing the conversation and detecting abnormalities such as dementia; means for notifying medical institutions, facilities, family members, etc. of the detected abnormalities; means for providing health improvement advice through collaboration with IoT and healthcare systems; means for supporting the realization of a monitoring society through collaboration with the local community; means for collecting the user's lifestyle and health data; means for storing the collected data in cloud storage; means for analyzing the stored data using a data analysis tool; means for evaluating the user's health status using a generative AI model; means for generating appropriate health improvement advice based on the analysis results; means for notifying the user's device of the generated advice; means for sharing the user's data with relevant parties in the local community; and means for establishing a monitoring system for the entire local community based on the shared data. This provides comprehensive support for the user's health management and mental stability and enables the establishment of a monitoring system in collaboration with the local community.

[0279] "Voice input" is a means by which a user provides information to a system using speech.

[0280] "Mental stability" refers to stabilizing the user's mental state and reducing stress and anxiety.

[0281] "Brain activation" refers to improving the user's brain function and enhancing cognitive ability and memory.

[0282] "Conversation status" refers to information such as the content, tone, and frequency of a user's conversations.

[0283] "Dementia tendency" refers to a state in which a user may be showing early symptoms of dementia.

[0284] "Abnormal" refers to a state in which a user's health or behavior is different from normal and is deemed to be problematic.

[0285] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0286] "Facilities" refers to places that support users' health and lifestyles, such as nursing homes and rehabilitation facilities.

[0287] "Family" refers to people related to you by blood or marriage.

[0288] "IoT" refers to physical devices and sensors connected to each other through the internet.

[0289] "Healthcare system" refers to systems and services that assist users in managing their health.

[0290] "Health Improvement Advice" refers to specific instructions or suggestions for improving a user's health.

[0291] "Local community" refers to the residents and stakeholders of the area where the user lives.

[0292] A "watchful society" refers to a system in which the entire local community watches over and supports users' safety and health.

[0293] "Lifestyle data" refers to data related to the user's daily life (e.g., number of steps taken, sleep time, dietary content).

[0294] "Health Data" means data relating to a User's health condition (e.g., body temperature, blood pressure, heart rate).

[0295] "Cloud storage" refers to an online storage service for storing and managing data via the Internet.

[0296] "Data analysis tools" refers to software and platforms for analyzing collected data.

[0297] A "generative AI model" refers to a model that uses machine learning and artificial intelligence techniques to analyze data and make predictions and evaluations.

[0298] "Terminal" refers to the device used by the User (e.g., smartphone, PC).

[0299] "Stakeholders" refers to people involved in supporting the user's health and life (e.g., family members, medical professionals, community volunteers).

[0300] MODE FOR CARRYING OUT THE INVENTION

[0301] The system of the present invention provides integrated support for user health management and community monitoring, and is implemented using voice input, IoT devices, generative AI models, cloud storage, and data analysis tools.

[0302] Hardware and software used

[0303] Hardware: Smartwatches, fitness trackers, smartphones, computers

[0304] Software: Python (registered trademark), R, TensorFlow, PyTorch, MySQL

[0305] Specific operation of the system

[0306] 1. Data Collection

[0307] Users wear IoT devices such as smartwatches and fitness trackers.

[0308] The terminal collects the user's lifestyle data (number of steps, sleep time, heart rate, etc.) and health data (body temperature, blood pressure, etc.) from these devices in real time.

[0309] 2. Data Transmission

[0310] The device periodically sends the collected data to a server, for example, by using Bluetooth or Wi-Fi to upload the data to the cloud.

[0311] 3. Data storage

[0312] The server stores the received data in a database (e.g. MySQL), organized by user, for easy later access.

[0313] 4. Data Analysis

[0314] The server uses data analysis tools such as Python and R to analyze the stored data.

[0315] The server uses a generative AI model (e.g., TensorFlow or PyTorch) to assess the user's health status, for example by comparing it with past data to detect outliers.

[0316] 5. Advice Generation

[0317] Based on the analysis results, the server generates appropriate advice for improving one's physical condition, such as "You've been sleeping less recently, so I recommend you go to bed earlier."

[0318] 6. Advice Notice

[0319] The server then sends the generated advice to the user's smartphone or computer via push notification or email.

[0320] 7. Data Sharing

[0321] A user uses an application to record their living situation and health condition.

[0322] The terminal transmits the recorded data to the server.

[0323] The server shares the user's data with community stakeholders (e.g., family members, health care workers, community volunteers) using a secure communication protocol (e.g., HTTPS).

[0324] 8. Establishing a monitoring system

[0325] Based on the shared data, the server will provide information for building a community-wide monitoring system, such as suggesting a schedule for regular health checks.

[0326] Community stakeholders use the information provided to provide support to prevent users from becoming isolated, for example by visiting or calling them.

[0327] Examples of concrete examples and prompts

[0328] Examples:

[0329] The user wears a smartwatch and it records the number of steps taken and their heart rate each day. For example, if the user has a goal of taking 10,000 steps a day, the smartwatch will collect that data.

[0330] The device sends this data to a smartphone via Bluetooth, and the smartphone then uploads the data to the cloud using Wi-Fi.

[0331] The server receives the data and stores it in a MySQL database, organized by user ID.

[0332] The server runs a Python script and uses a TensorFlow model to analyze the data, detecting an outlier, for example, if the number of steps taken today is significantly lower than the average number of steps taken over the past week.

[0333] The server generates advice such as "You haven't taken many steps today, so you should walk a little more."

[0334] The server sends the generated advice to the user's smartphone via push notification.

[0335] Through the application, users can share their health status with local healthcare professionals and receive the necessary support. For example, healthcare professionals can review the user's data and suggest regular visits.

[0336] Example prompt sentence:

[0337] "Generate advice for improving health based on the user's step count and heart rate data."

[0338] "Please share your health status with local healthcare professionals and provide information to help them establish a monitoring system."

[0339] In this way, the system of the present invention utilizes IoT devices and generative AI models to support the health management of users and the monitoring system of local communities. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0340] Step 1:

[0341] Data collection

[0342] Users wear IoT devices such as smartwatches and fitness trackers.

[0343] The terminal collects the user's lifestyle data (number of steps, sleep time, heart rate, etc.) and health data (body temperature, blood pressure, etc.) from these devices in real time.

[0344] Input: Data from the user's IoT device

[0345] Output: Collected lifestyle and health data

[0346] Step 2:

[0347] Data transmission

[0348] The device periodically sends the collected data to a server, for example, by using Bluetooth or Wi-Fi to upload the data to the cloud.

[0349] Input: Collected lifestyle and health data

[0350] Output: Data sent to the server

[0351] Step 3:

[0352] Data storage

[0353] The server stores the received data in a database (e.g. MySQL), organized by user, for easy later access.

[0354] Input: Data sent to the server

[0355] Output: Data stored in the database

[0356] Step 4:

[0357] Data analysis

[0358] The server uses data analysis tools such as Python and R to analyze the stored data.

[0359] The server uses a generative AI model (e.g., TensorFlow or PyTorch) to assess the user's health status, for example by comparing it with past data to detect outliers.

[0360] Input: Data stored in the database

[0361] Output: Analysis results (such as outlier detection)

[0362] Step 5:

[0363] Advice Generation

[0364] Based on the analysis results, the server generates appropriate advice for improving one's physical condition, such as "You've been sleeping less recently, so I recommend you go to bed earlier."

[0365] Input: Analysis results

[0366] Output: Generated advice for improving your health

[0367] Step 6:

[0368] Advice Notice

[0369] The server then sends the generated advice to the user's smartphone or computer via push notification or email.

[0370] Input: Generated advice for improving your health

[0371] Output: Advice sent to the user's device

[0372] Step 7:

[0373] Data Sharing

[0374] A user uses an application to record their living situation and health condition.

[0375] The terminal transmits the recorded data to the server.

[0376] The server shares the user's data with community stakeholders (e.g., family members, health care workers, community volunteers) using a secure communication protocol (e.g., HTTPS).

[0377] Input: Recorded living and health status data

[0378] Output: Data shared with community stakeholders

[0379] Step 8:

[0380] Establishing a monitoring system

[0381] Based on the shared data, the server will provide information for building a community-wide monitoring system, such as suggesting a schedule for regular health checks.

[0382] Community stakeholders use the information provided to provide support to prevent users from becoming isolated, for example by visiting or calling them.

[0383] Input: Shared data

[0384] Output: Information necessary for establishing a monitoring system and implementation of support activities

[0385] (Application example 3)

[0386] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0387] Maintaining users' mental and physical health is an important issue in modern society. However, proper health management and monitoring are difficult in environments where users tend to be isolated. Furthermore, there is a lack of systems that can effectively collect and analyze users' lifestyle and health data and provide appropriate advice. Furthermore, there is no system in place for rapid response in emergencies. To solve these issues, a system that can comprehensively manage users' health status is needed, linking with IoT and healthcare systems and working with the local community.

[0388] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0389] In this invention, the server includes: means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; means for analyzing the conversation and detecting abnormalities such as dementia; means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; means for providing health improvement advice through collaboration with IoT and healthcare systems; means for supporting the realization of a monitoring society through collaboration with the local community; means for collecting and analyzing the user's lifestyle and health data; means for generating and providing advice for health improvement to the user based on the analysis results; means for sharing the collected data with relevant parties in the local community and establishing a monitoring system; and means for issuing alerts in the event of an emergency and encouraging a rapid response. This allows for comprehensive management of the user's health status, providing appropriate advice, and establishing a monitoring system in collaboration with the local community, enabling a rapid response in the event of an emergency.

[0390] "Voice input" is a technology that allows digital devices to recognize and process the words a user speaks.

[0391] "Mental stability" refers to stabilizing the user's mental state and reducing stress and anxiety.

[0392] "Brain activation" refers to improving the user's brain function and enhancing cognitive ability and memory.

[0393] "Conversation status" refers to information such as the content, tone, and frequency of a user's conversations.

[0394] "Dementia proneness" refers to the likelihood that a user is exhibiting early symptoms of dementia.

[0395] "Abnormal" refers to a state in which a user's health condition or behavior is different from normal.

[0396] "Hospitals, facilities, families, etc." refers to parties involved in managing and monitoring the user's health.

[0397] "IoT" refers to the technology that connects various devices via the Internet and exchanges data.

[0398] "Healthcare system" refers to a system for managing and improving the health status of users.

[0399] "Health Improvement Advice" refers to specific instructions or suggestions for improving a user's health.

[0400] "Local community" refers to the residents and stakeholders of the area where the user lives.

[0401] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[0402] "Lifestyle data" refers to information about a user's daily life (e.g., sleep, exercise, diet, etc.).

[0403] "Health Data" refers to information relating to a User's health status (e.g., heart rate, blood pressure, etc.).

[0404] "Analysis" refers to evaluating collected data and extracting meaningful information.

[0405] "Generating advice" refers to creating specific instructions or suggestions to provide to the user based on the analysis results.

[0406] "Sharing data" refers to exchanging collected information with other parties.

[0407] A "monitoring system" refers to a mechanism for continuously monitoring the safety and health of users and taking necessary measures.

[0408] "Emergency" means a situation that requires immediate attention from the User.

[0409] "Sending an alert" refers to sending a notification to relevant parties in the event of an emergency.

[0410] "Rapid response" refers to taking appropriate action quickly in an emergency.

[0411] The system for implementing this invention is composed of an IoT device for collecting and analyzing a user's lifestyle and health data, a server for processing this data, and a terminal for providing advice to the user. Specific embodiments are described below.

[0412] Hardware Configuration

[0413] IoT devices: Devices designed to collect user health data, such as heart rate monitors, blood pressure monitors, and smartwatches.

[0414] Server: A central processing unit for data collection, analysis, advice generation, and data sharing. It uses Python, the Requests library, and the JSON library.

[0415] Device: A device, such as a smartphone or tablet, that provides advice to users and receives alerts in the event of an emergency.

[0416] Software Configuration

[0417] Data collection module: collects data from IoT devices and sends it to the server.

[0418] Data analysis module: Analyzes collected data and evaluates the user's health status.

[0419] Advice generation module: Generates advice to be provided to the user based on the analysis results.

[0420] Data sharing module: Share collected data with local community stakeholders to build a monitoring system.

[0421] Alert module: Sends alerts in case of emergencies, encouraging rapid response.

[0422] Processing flow

[0423] 1. Data collection: IoT devices collect health data such as the user's heart rate and blood pressure and send it to a server.

[0424] 2. Data analysis: The server analyzes the collected data and evaluates the user's health condition. For example, if the heart rate is high, it generates advice such as "Your heart rate is high. Take a deep breath to relax."

[0425] 3. Providing advice: The generated advice is sent to the terminal and provided to the user.

[0426] 4. Data sharing: Collected data will be shared with relevant people in the local community (family members, medical professionals, local volunteers, etc.) to establish a monitoring system.

[0427] 5. Alerting: In the event of an emergency, alerts will be sent to encourage rapid response.

[0428] Specific examples

[0429] For example, if a user is wearing a heart rate monitor, if their heart rate exceeds 100, the server will generate a message saying, "Your heart rate is high. Take a deep breath to relax." This message will be sent to the device. At the same time, this data will be shared with family and medical professionals, allowing them to take the necessary measures promptly.

[0430] Prompt Sentence Examples

[0431] "Write a Python program that generates relaxation advice when a user's heart rate exceeds 100 and shares the data with family and healthcare professionals."

[0432] The above is a specific embodiment for carrying out the present invention.

[0433] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0434] Step 1:

[0435] Data collection

[0436] The server collects user health data from IoT devices (heart rate monitors, blood pressure monitors, smart watches, etc.). Specifically, the IoT devices periodically measure data such as heart rate and blood pressure and send it to the server. The input is the health data from the IoT devices, and the output is the raw data stored on the server.

[0437] Step 2:

[0438] Data analysis

[0439] The server analyzes the collected health data. Specifically, it uses a Python data analysis library to detect abnormal values ​​in heart rate and blood pressure. For example, if the heart rate exceeds 100, it is considered abnormal. The input is the health data collected in step 1, and the output is the analysis result (e.g., high heart rate).

[0440] Step 3:

[0441] Advice Generation

[0442] The server generates advice to provide to the user based on the results of the data analysis. Specifically, it uses a generative AI model to create appropriate advice. For example, if the heart rate is high, it generates the advice "Your heart rate is high. Take a deep breath to relax." The input is the analysis result from step 2, and the output is the generated advice.

[0443] Step 4:

[0444] Providing advice

[0445] The server sends the generated advice to the device (smartphone or tablet) and provides it to the user. Specifically, it sends the advice to the device using an HTTP request, and the device displays it to the user. The input is the advice generated in step 3, and the output is the advice displayed to the user.

[0446] Step 5:

[0447] Data Sharing

[0448] The server shares the collected health data with stakeholders in the local community (family members, medical professionals, local volunteers, etc.). Specifically, it exports the data in JSON format and sends it to the stakeholders. The input is the health data collected in step 1, and the output is the data shared with the stakeholders.

[0449] Step 6:

[0450] Alert sending

[0451] The server issues an alert in the event of an emergency, encouraging a prompt response. Specifically, if an abnormality is detected, it sends a notification to the relevant parties via SMS or email. The input is the analysis result from step 2, and the output is the issued alert.

[0452] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0453] "Example 1"

[0454] One embodiment of the present invention is a system that incorporates an emotion engine. This system receives a user's voice as input and recognizes the user's emotion from the voice. The emotion engine estimates the emotion based on features extracted from the voice (e.g., pitch, volume, speaking rate, etc.). The emotion engine can classify emotions using a machine learning model. This model is trained from a large amount of voice data and its corresponding emotion labels.

[0455] "Example 2"

[0456] The emotion engine can also adjust the content of the conversation based on the emotions it recognizes. For example, if it recognizes that the user is angry, the system will switch to a more gentle topic or make statements that will calm the user. Conversely, if it recognizes that the user is happy, the system will make statements that share that joy or change to a topic that will amplify that joy.

[0457] "Example 3"

[0458] Furthermore, the emotion engine can track long-term changes in a user's emotions, enabling early detection of changes in the user's mental state and providing necessary support. For example, if a user continues to show sad emotions over a certain period of time, the system will notify medical institutions or family members, enabling early intervention and appropriate care.

[0459] "Example 4"

[0460] The emotion engine can also provide well-being advice tailored to the user's emotions. For example, if the system recognizes that the user is feeling stressed, it can suggest relaxation or meditation techniques. Or, if the system recognizes that the user is feeling joy, it can suggest activities and habits to maintain that joy.

[0461] The processing flow of each embodiment will be described below.

[0462] "Example 1"

[0463] Step 1: The system receives the user's voice as input.

[0464] Step 2: Extract features from the audio (e.g., pitch, volume, speaking rate, etc.).

[0465] Step 3: The emotion engine estimates emotions based on the extracted features.

[0466] Step 4: The emotion engine classifies emotions using a machine learning model that is trained from a large amount of audio data and its corresponding emotion labels.

[0467] "Example 2"

[0468] Step 1: The emotion engine adjusts the content of the conversation based on the emotions it recognizes.

[0469] Step 2: For example, if the system recognizes that the user is angry, it will switch to a calmer topic or make remarks to calm the user.

[0470] Step 3: Conversely, if the system recognizes that the user is happy, it will make comments that share that happiness or switch to a topic that will amplify that happiness.

[0471] "Example 3"

[0472] Step 1: The emotion engine tracks the user's long-term emotional changes.

[0473] Step 2: If the user continues to show sad emotions over a period of time, the system will notify medical institutions and family members.

[0474] Step 3: This allows for early intervention and appropriate care.

[0475] "Example 4"

[0476] Step 1: The emotion engine provides health improvement advice adapted to the user's emotions.

[0477] Step 2: For example, if the system recognizes that the user is feeling stressed, it will suggest relaxation or meditation techniques.

[0478] Step 3: If the system also recognizes that the user is experiencing joy, it will suggest activities and habits to maintain that joy.

[0479] Example 1

[0480] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0481] In modern society, it is important to ensure users' mental stability and brain activity. However, with conventional systems, it was difficult to achieve this through natural conversation with the user. It was also difficult to accurately recognize the user's emotions and generate appropriate responses. Furthermore, there is a need to detect abnormalities such as dementia early and take appropriate action. In order to solve these problems, it is necessary to effectively utilize voice recognition technology and emotion recognition technology.

[0482] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0483] In this invention, the server includes: a means for conducting a conversation via voice input, contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia trends; a means for notifying a hospital, facility, family, etc. of the detected abnormality; a means for converting voice data into text data; a means for recognizing the user's emotions from the voice data; a means for generating an appropriate response based on the emotional information and text data; a means for converting the generated response into voice and conveying it to the user; a means for providing health improvement advice through collaboration with IoT and healthcare systems; and a means for supporting the realization of a monitoring society through collaboration with the local community. This enables mental stability and brain activation through natural conversation with the user, and furthermore, it can accurately recognize the user's emotions and generate appropriate responses. It can also detect abnormalities such as dementia trends early and take appropriate measures.

[0484] "Voice input" is the process of capturing user-generated speech into a system through a device such as a microphone.

[0485] "Mental stability" refers to maintaining or improving the user's psychological stability.

[0486] "Brain activation" refers to stimulating the user's brain functions and improving their cognitive and thinking abilities.

[0487] "Conversational state" refers to the content and quality of the interaction between the user and the system.

[0488] "Dementia tendency" refers to a state in which a user shows early symptoms or signs of dementia.

[0489] "Abnormal" refers to a user's behavior or health condition that deviates from normal.

[0490] "Notification" is the process of informing relevant parties such as hospitals, facilities, and family members of detected abnormalities.

[0491] "IoT" refers to a network of devices and systems connected via the internet.

[0492] "Healthcare system" refers to a system for managing and improving the health status of users.

[0493] "Health Improvement Advice" refers to specific suggestions or instructions to improve a user's health.

[0494] "Community" refers to the geographic or social community to which a User belongs.

[0495] A "watchful society" refers to a system in which the local community cooperates to watch over the safety and health of users.

[0496] "Voice data" refers to data that is a digital recording of a user's voice.

[0497] "Text data" refers to data obtained by converting voice data into character information.

[0498] "Emotion recognition" is the process of inferring emotions from a user's voice and behavior.

[0499] "Response generation" is the process of creating an appropriate reply to a user's input.

[0500] "Speech synthesis" refers to the technology of converting text data into speech.

[0501] This invention is a system that combines speech recognition technology and emotion recognition technology, and aims to stabilize the mind and activate the brain through natural conversation with the user. Specific embodiments of this system will be described below.

[0502] Hardware and software used

[0503] This system uses the following hardware and software:

[0504] Hardware: microphone, speaker, device (PC, smartphone, tablet, etc.)

[0505] Software: speech recognition APIs (e.g., Google Cloud Speech-to-Text), emotion recognition engines (e.g., TensorFlow, PyTorch), generative AI models (e.g., OpenAI GPT-3®), speech synthesis APIs (e.g., Google Cloud Text-to-Speech)

[0506] System processing flow

[0507] 1. Receiving voice input

[0508] The user speaks to the system, and the device captures this speech with a microphone. For example, if the user says, "I'm very tired today," the speech is input into the device's microphone.

[0509] 2. Converting voice data to text

[0510] The device sends the captured voice data to a voice recognition API. The server calls this API and converts the voice data into text data. For example, the voice saying "I'm very tired today" is converted into text "I'm very tired today."

[0511] 3. Emotion recognition

[0512] The server sends the text data and voice data to an emotion recognition engine. The emotion recognition engine extracts features such as pitch, volume, and speaking rate from the voice data and uses these features to estimate the user's emotion. For example, the emotion recognition engine recognizes the emotion "fatigue."

[0513] 4. Response Generation

[0514] The server sends a prompt to the generative AI model based on the emotional information and text data from the emotion recognition engine. The generative AI model generates an appropriate response. For example, if the prompt is "The user says, 'I'm very tired today,' and the emotion is 'fatigue.' Please generate an appropriate response," the generative AI model will generate the response, "Today was tough. Take a break and relax."

[0515] 5. Speech synthesis and response delivery

[0516] The server sends the generated response text to the device. The device then converts the text into speech using a speech synthesis API. For example, the text might say, "Today was tough. Let's take a break and relax." The device then transmits this speech to the user through the speaker.

[0517] Specific examples

[0518] When a user says, "I'm very tired today," the system processes as follows:

[0519] 1. Receiving voice input:

[0520] A user says, "I'm very tired today."

[0521] The device's microphone captures the audio.

[0522] 2. Audio to text conversion:

[0523] The device sends the captured voice data to the voice recognition API.

[0524] The server converts the speech data into text such as "I'm very tired today."

[0525] 3. Emotion recognition:

[0526] The server sends the text and voice data to the emotion recognition engine.

[0527] The emotion recognition engine extracts features such as pitch, volume, and speaking rate from the audio data.

[0528] The emotion recognition engine recognizes the emotion "fatigue."

[0529] 4. Response Generation:

[0530] The server sends prompts to the generative AI model based on emotional information and text data.

[0531] The generative AI model generates a response like, "You've had a hard day today. Take a break and relax."

[0532] 5. Speech synthesis and response delivery:

[0533] The server sends the generated response text to the terminal.

[0534] The device uses a speech synthesis API to convert the text into speech.

[0535] The device transmits the audio to the user through a speaker.

[0536] In this way, the system can have natural conversations with the user, contributing to the user's mental stability and brain activity.

[0537] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0538] Step 1:

[0539] Receiving voice input

[0540] The user speaks to the system. The device captures this voice with a microphone. For example, when a user says, "I'm very tired today," the voice is input into the device's microphone. The input is the user's voice, and the output is the captured voice data.

[0541] Step 2:

[0542] Converting audio data to text

[0543] The device sends the captured voice data to a voice recognition API. The server calls this API and converts the voice data into text data. For example, the voice saying "I'm very tired today" is converted into text "I'm very tired today." The input is voice data, and the output is text data.

[0544] Step 3:

[0545] emotion recognition

[0546] The server sends text data and voice data to an emotion recognition engine. The emotion recognition engine extracts features such as pitch, volume, and speaking rate from the voice data and uses these features to estimate the user's emotion. For example, the emotion recognition engine recognizes the emotion "fatigue." The input is voice data and text data, and the output is emotional information.

[0547] Step 4:

[0548] Response Generation

[0549] The server sends a prompt to the generative AI model based on the emotional information and text data from the emotion recognition engine. The generative AI model generates an appropriate response. For example, if the prompt is "The user says, 'I'm very tired today,' and the emotion is 'fatigue.' Please generate an appropriate response," the generative AI model generates the response, "Today was tough. Take a break and relax." The input is emotional information and text data, and the output is the response text.

[0550] Step 5:

[0551] Speech synthesis and response delivery

[0552] The server sends the generated response text to the device. The device converts the text into speech using a speech synthesis API. For example, the text "Today has been tough. Let's take a break and relax." is converted into speech. The device transmits this speech to the user through the speaker. The input is the response text, and the output is audio data.

[0553] (Application example 1)

[0554] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0555] While conventional voice recognition systems can contribute to the user's mental stability and brain activity, they lack the ability to recognize the user's emotions and generate appropriate responses accordingly. In addition, certain services, such as food delivery, require support for the ordering process that takes the user's emotions into account. This requires a system that allows users to use the service smoothly and without stress.

[0556] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation, a means for analyzing the state of the conversation and detecting abnormalities such as dementia tendencies, a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, a means for providing advice on improving health through collaboration with IoT and healthcare systems, a means for supporting the realization of a monitoring society through collaboration with the local community, a means for recognizing emotions from the user's voice and generating a response according to the emotion, and a means for supporting the user's ordering process using voice recognition and emotion analysis. This makes it possible to generate a response that takes the user's emotions into consideration and support the ordering process.

[0557] "Voice input" is a technology in which the user's voice is captured as a digital signal and the system analyzes the content.

[0558] A "conversation" is the process by which a user and a system exchange information through speech.

[0559] "Mental stability" refers to maintaining or improving the user's psychological stability.

[0560] "Brain activation" refers to stimulating the user's brain functions and improving their cognitive and thinking abilities.

[0561] "Conversational state" refers to the overall situation, including the content, tone, and emotion of the conversation between the user and the system.

[0562] "Abnormal" refers to a user's behavior, emotions, or health state that deviates from normal.

[0563] "Hospitals, facilities, families, etc." refers to external organizations and individuals who manage the health and safety of users.

[0564] "Notification" is the act of communicating information to external organizations or individuals when a system detects an abnormality.

[0565] "IoT" is a technology that allows various devices to be interconnected and exchange data via the Internet.

[0566] A "healthcare system" is a system for managing and improving a user's health status.

[0567] "Health Improvement Advice" refers to specific instructions or suggestions for improving a user's health.

[0568] "Local community" refers to the local community in which the user lives.

[0569] A "watchful society" refers to a system in which the local community cooperates to watch over the safety and health of users.

[0570] "Emotion recognition" is a technology that analyzes and identifies a user's emotions from their voice and behavior.

[0571] "Generating a response" is the process by which the system creates an appropriate reply to the user's input.

[0572] "Ordering process" refers to the series of steps a user takes to order a product or service.

[0573] "Support" refers to the system providing assistance to help the user with their actions and decisions.

[0574] The system for implementing this invention integrates the functions of voice input, voice recognition, emotion recognition, response generation, voice synthesis, and order process support. Specific embodiments are described below.

[0575] Hardware and Software Configuration

[0576] This system uses the following hardware and software:

[0577] Hardware: Microphone, Smartphone

[0578] Software: Python, speech recognition library, speech synthesis module, emotion recognition module, order processing module

[0579] Processing flow

[0580] 1. Acquiring voice input: The user speaks through a microphone. The smartphone captures this voice as a digital signal and analyzes it using a speech recognition library.

[0581] 2. Speech recognition: The captured voice is converted into text using Google's speech recognition API, and the user's speech is captured as text data.

[0582] 3. Emotion Recognition: Text data is input into the emotion recognition module to analyze the user's emotions. Emotion recognition is performed based on voice features such as pitch, volume, and speaking rate.

[0583] 4. Response generation: Generate an appropriate response based on the emotion recognition results. For example, if the user is feeling stressed, generate a response to help them relax.

[0584] 5. Speech synthesis: The generated response is converted into speech using a speech synthesis module and conveyed to the user.

[0585] 6. Ordering process support: When a user places a food delivery order, the system uses speech recognition and sentiment analysis to support the ordering process. For example, if a user says, "I'd like to order a pizza," the system converts the speech into text. If the sentiment engine determines that the user is stressed, the system responds, "Please relax. We'll take your order."

[0586] Specific examples

[0587] For example, if a user says, "I'd like to order a pizza," the system converts that speech into text, and if the emotion engine determines that the user is stressed, it responds, "Please relax. We'll take your order." In this way, it is possible to generate responses that take the user's emotions into account and support the ordering process.

[0588] Prompt Sentence Examples

[0589] If a user says, "I'd like to order a pizza," the system converts that speech to text, and if its emotion engine determines that the user is stressed, it responds, "Relax. We'll take your order."

[0590] In this way, a food delivery application can provide a voice interface that takes into account the user's emotions.

[0591] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0592] Step 1:

[0593] The user inputs voice through a microphone. The device captures this voice as a digital signal and analyzes it using a speech recognition library. The input is the user's voice, and the output is a digital signal. This digital signal is used for speech recognition in the next step.

[0594] Step 2:

[0595] The device converts the acquired digital signal into text using Google's speech recognition API. The input is a digital signal, and the output is text data. This text data represents the content of the user's speech and is used for emotion recognition in the next step.

[0596] Step 3:

[0597] The server inputs text data into an emotion recognition module to analyze the user's emotions. The input is text data, and the output is an emotion label (e.g., stress, joy, etc.). Emotion recognition is performed based on voice features such as pitch, volume, and speaking rate.

[0598] Step 4:

[0599] The server generates an appropriate response based on the emotion recognition results. The input is an emotion label, and the output is a response text. For example, if the user is feeling stressed, the server generates a response such as "Please relax. We will take your order."

[0600] Step 5:

[0601] The server converts the generated response text into speech using a speech synthesis module and transmits it to the user through the terminal. The input is the response text and the output is audio data. The terminal plays this audio data and transmits the response to the user.

[0602] Step 6:

[0603] When a user places a food delivery order, the server uses speech recognition and sentiment analysis to support the ordering process. The input is the user's order and an emotional label, and the output is a response confirming the order. For example, if a user says, "I'd like to order a pizza," the system converts the speech into text, and if the sentiment engine determines that the user is stressed, it responds, "Please relax. We'll take your order."

[0604] Example 2

[0605] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0606] In an aging society, early detection and prevention of dementia are becoming important issues. It is also necessary to care for the user's mental stability and emotions. Furthermore, if an abnormality is detected, a prompt and appropriate response is required. To solve these issues, a system is needed that can analyze the user's speech content and emotions in detail and respond appropriately.

[0607] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0608] In this invention, the server includes: means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; means for analyzing the state of the conversation and detecting abnormalities such as dementia trends from the user's speech content, speech patterns, and speech frequency using natural language processing technology and machine learning technology; means for notifying a hospital, facility, family, etc. of the detected abnormality; and means for recognizing the user's emotions using an emotion engine and adjusting the content of the conversation based on the recognized emotions, thereby enabling monitoring of the user's health condition and emotional care.

[0609] "Voice input" is a technology that collects user speech as voice data.

[0610] "Mental stability" refers to maintaining the user's psychological stability.

[0611] "Brain activation" refers to improving the user's brain function.

[0612] "Conversation state" refers to the content of a user's speech, speech patterns, frequency of speech, etc.

[0613] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[0614] "Machine learning technology" is a technology that allows computers to learn from data and make predictions and classifications.

[0615] "Speech content" refers to the specific words or sentences spoken by the user.

[0616] "Speech patterns" refer to the tendencies and characteristics of a user's speech.

[0617] "Utterance frequency" refers to how often a user says a particular word or phrase.

[0618] "Dementia proneness" refers to the likelihood that a user is exhibiting early symptoms of dementia.

[0619] "Abnormal" refers to a state or behavior that is different from the normal.

[0620] "Notification" refers to conveying information to relevant parties when an abnormality is detected.

[0621] An "emotion engine" is a technology that recognizes the user's emotions and responds appropriately based on them.

[0622] "Emotion" refers to the user's psychological state.

[0623] "Adjusting the conversation" refers to changing the topic and tone of the conversation depending on the user's emotions.

[0624] "IoT" is a technology that allows various devices to connect via the Internet.

[0625] A "healthcare system" is a system for managing and improving a user's health status.

[0626] "Health Improvement Advice" refers to specific suggestions and guidance to improve the user's health.

[0627] "Local community" refers to the local community in which the user lives.

[0628] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[0629] The present invention is a system that analyzes the content of a user's speech and emotions in detail and responds appropriately. A specific embodiment of this system is described below.

[0630] Hardware and software used

[0631] Hardware: Servers, devices (PCs, smartphones, tablets, etc.)

[0632] Software: Natural language processing libraries (e.g., NLTK, spaCy), machine learning frameworks (e.g., TensorFlow, PyTorch), sentiment analysis engines (e.g., IBM Watson®, Microsoft® Azure® Emotion API), speech recognition software (e.g., Google Speech-to-Text API)

[0633] Data processing and calculation

[0634] 1. Collecting user utterance data

[0635] The device collects the user's speech as voice data. Specifically, it uses the microphone of a smartphone or tablet to record the user's speech.

[0636] Example: A user says, "The weather is nice today."

[0637] 2. Converting voice data to text

[0638] The device sends the collected voice data to the server.

[0639] The server uses voice recognition software to convert the voice data into text data.

[0640] Example: Speech data such as "The weather is nice today" is converted into text data such as "The weather is nice today."

[0641] 3. Analysis of speech content

[0642] The server uses a natural language processing library to analyze the text data, specifically tagging words by part of speech and analyzing sentence structure.

[0643] Example: The word "weather" is tagged as a noun and the word "nice" is tagged as an adjective.

[0644] 4. Anomaly Detection

[0645] The server uses a machine learning model to detect abnormalities based on the analysis results. Specifically, the server inputs the user's speech patterns and frequency into the model to determine whether they are prone to dementia.

[0646] Example: If a user frequently says, "I've been forgetting things a lot lately," the system will determine that they are prone to dementia.

[0647] 5. Notification of abnormalities

[0648] The server will notify hospitals, facilities, families, etc. of any abnormalities it detects. Possible notification methods include email, SMS, or push notifications via a dedicated app.

[0649] Example: Send a notification to a family member saying, "The user has been frequently mentioning forgetfulness recently. This may indicate a tendency toward dementia."

[0650] 6. Emotional Recognition

[0651] The server uses a sentiment analysis engine to recognize the user's emotions. Specifically, it inputs text data into the sentiment analysis engine and identifies the type of emotion (joy, anger, sadness, etc.).

[0652] Example: If a user says, "I'm very happy today," the system will recognize it as "joy."

[0653] 7. Adjust the conversation

[0654] The server adjusts the content of the conversation based on the emotions it recognizes, specifically by using a generative AI model to generate appropriate utterances.

[0655] Example: If a user says, "I'm so happy today," the system responds, "That's great! Did something good happen?"

[0656] Prompt Sentence Examples

[0657] "The user has frequently mentioned forgetfulness in recent utterances. Please analyze whether they have a tendency toward dementia."

[0658] "User recognized as angry. Generate appropriate utterance to transition to a calmer topic."

[0659] In this way, the present invention realizes monitoring of the user's health condition and emotional care by analyzing the user's speech content and emotions in detail and taking appropriate measures.

[0660] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0661] Step 1:

[0662] Collecting user speech data

[0663] The device collects the user's speech as voice data. Specifically, it uses the microphone of a smartphone or tablet to record the user's speech.

[0664] Input: User utterance (e.g., "The weather is nice today.")

[0665] Output: Audio data (e.g., a recorded audio file)

[0666] What it does: The device's microphone captures the user's speech and saves it as an audio file.

[0667] Step 2:

[0668] Converting audio data to text

[0669] The device sends the collected voice data to the server.

[0670] The server uses voice recognition software to convert the voice data into text data.

[0671] Input: Audio data (e.g., a recorded audio file)

[0672] Output: Text data (e.g. "The weather is nice today")

[0673] What happens: The server calls speech recognition software to convert the audio file into text.

[0674] Step 3:

[0675] Analysis of speech content

[0676] The server uses a natural language processing library to analyze the text data, specifically tagging words by part of speech and analyzing sentence structure.

[0677] Input: Text data (e.g., "The weather is nice today")

[0678] Output: Analysis result (e.g. "weather" is tagged as a noun and "good" as an adjective)

[0679] Specific operation: The server uses a natural language processing library to tag the text data by part of speech and analyze the sentence structure.

[0680] Step 4:

[0681] Anomaly detection

[0682] The server uses a machine learning model to detect abnormalities based on the analysis results. Specifically, the server inputs the user's speech patterns and frequency into the model to determine whether they are prone to dementia.

[0683] Input: Analysis results (e.g., part-of-speech tagged text data)

[0684] Output: Anomaly detection result (e.g., tendency toward dementia)

[0685] Specific operation: The server inputs the analysis results into the machine learning model and determines whether there are any abnormalities.

[0686] Step 5:

[0687] Abnormality notification

[0688] The server will notify hospitals, facilities, families, etc. of any abnormalities it detects. Possible notification methods include email, SMS, or push notifications via a dedicated app.

[0689] Input: Anomaly detection result (e.g., tendency toward dementia)

[0690] Output: Notification message (e.g. "The user has frequently mentioned forgetfulness recently. He may be prone to dementia.")

[0691] Specific operation: The server generates a notification message based on the anomaly detection result and sends it to the relevant parties.

[0692] Step 6:

[0693] Emotion recognition

[0694] The server uses a sentiment analysis engine to recognize the user's emotions. Specifically, it inputs text data into the sentiment analysis engine and identifies the type of emotion (joy, anger, sadness, etc.).

[0695] Input: Text data (e.g., "I'm very happy today")

[0696] Output: Emotion recognition result (e.g., joy)

[0697] Specific operation: The server inputs text data into a sentiment analysis engine to identify the type of sentiment.

[0698] Step 7:

[0699] Adjusting the conversation

[0700] The server adjusts the content of the conversation based on the emotions it recognizes, specifically by using a generative AI model to generate appropriate utterances.

[0701] Input: Emotion recognition result (e.g., joy)

[0702] Output: Tailored conversation (e.g., "That's great! Did something good happen?")

[0703] Specific operation: The server inputs the emotion recognition results into the generated AI model, generates appropriate utterances, and sends them to the device.

[0704] (Application example 2)

[0705] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0706] Dementia and mental health issues among the elderly are serious problems in modern society. It is particularly difficult to monitor daily life and detect abnormalities early for elderly people who live alone or away from their families. There is also a need to detect changes in users' emotions and abnormal speech patterns in real time and respond appropriately. To solve these problems, a system is needed that analyzes the content of users' conversations and emotions, detects abnormalities, and sends appropriate notifications.

[0707] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0708] In this invention, the server includes means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation, means for analyzing the state of the conversation and detecting abnormalities such as dementia trends, means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, means for providing advice on improving health through collaboration with IoT and healthcare systems, means for supporting the realization of a watchful society through collaboration with the local community, means for analyzing the user's emotions in real time and detecting abnormal speech patterns and changes in emotions, means for notifying pre-set contacts of the detected abnormalities, and means for adjusting the content of the conversation according to the user's emotions.This makes it possible to maintain the user's mental health, detect abnormalities early, and take appropriate measures.

[0709] "Voice input" is a technology that captures a user's speech as digital data.

[0710] "Mental stability" refers to maintaining the user's psychological stability.

[0711] "Brain activation" refers to stimulating the user's brain functions and improving cognitive function.

[0712] "Conversational state" refers to the overall state of a user's speech, including their speech content, speech patterns, and speech frequency.

[0713] "Dementia tendency" refers to a state in which a user may develop dementia.

[0714] "Abnormal" refers to a user's speech or behavior that deviates from normal.

[0715] "Notification" refers to sending information to pre-defined contacts when an abnormality is detected.

[0716] "IoT" refers to technology that connects various devices via the Internet.

[0717] "Healthcare system" refers to a system for managing and improving the health status of users.

[0718] "Health Improvement Advice" refers to specific instructions or suggestions for improving a user's health.

[0719] "Local community" refers to the local community in which the user lives.

[0720] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[0721] "Real-time emotion analysis" refers to technology that instantly analyzes emotions from the content of a user's speech.

[0722] "Speech patterns" refer to characteristics such as the rhythm and frequency of a user's speech, and word choice.

[0723] "Emotional changes" refers to the user's emotional state changing over time.

[0724] "Pre-configured Contacts" refers to contacts configured to receive notifications when an anomaly is detected.

[0725] "Conversational content adjustment" refers to changing the topic and tone of a conversation depending on the user's emotional state.

[0726] The following system configuration is used as an embodiment of the present invention.

[0727] System Configuration

[0728] The system consists of a voice input device, a server, a notification device, and a linked device. The voice input device captures the user's speech and converts it into digital data. The server analyzes the captured voice data and analyzes the user's mental state and emotions in real time. The notification device sends a notification to pre-set contacts if an abnormality is detected. The linked device connects with IoT and healthcare systems to provide the user with advice on improving their health.

[0729] Hardware and software used

[0730] Hardware: smartphone, microphone, notification device (e.g. smartwatch)

[0731] Software: Google Cloud Natural Language API, Communication Services API, Python

[0732] Data processing and calculation

[0733] 1. Acquiring voice data: Using a voice input device (smartphone or microphone), the user's speech is acquired as digital data.

[0734] 2. Natural language processing: The server uses the Google Cloud Natural Language API to analyze the acquired voice data and calculate the user's emotion score.

[0735] 3. Anomaly detection: The server determines an anomaly if the emotion score exceeds a certain threshold.

[0736] 4. Send notifications: If an anomaly is detected, a notification is sent to pre-defined contacts using the communication service API.

[0737] 5. Conversation content adjustment: The server adjusts the conversation content according to the user's emotional state.

[0738] Specific examples

[0739] For example, if a user says, "I've been feeling unmotivated lately," the voice input device captures this utterance and sends it to the server. The server uses the Google Cloud Natural Language API to calculate the emotion score for this utterance, resulting in a score of -0.7. Because this score is deemed abnormal, the communication service API is used to send a notification to pre-defined contacts stating, "An abnormal speech pattern has been detected."

[0740] Prompt Sentence Examples

[0741] If the user says, "I just don't feel like doing anything these days," use the following prompt:

[0742] user_input = "I've been feeling unmotivated lately."

[0743] score, magnitude = analyze_text(user_input)

[0744] if score < -0.5:

[0745] send_alert("An unusual speech pattern has been detected.")

[0746] In this way, it is possible to realize a system that analyzes the content of user utterances in real time and takes appropriate action if an abnormality is detected.

[0747] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0748] Step 1:

[0749] The user speaks.

[0750] The user speaks using a smartphone or microphone, and this speech becomes input data for the system.

[0751] Step 2:

[0752] The terminal acquires the audio data.

[0753] The device (smartphone or microphone) captures the user's speech as digital voice data, which serves as input for the next processing step.

[0754] Step 3:

[0755] The terminal transmits the voice data to the server.

[0756] The terminal transmits the acquired voice data to the server, which then uses the data as input for processing.

[0757] Step 4:

[0758] The server converts the voice data into text data.

[0759] The server uses speech recognition technology to convert the voice data into text data, which serves as input for the next processing step.

[0760] Step 5:

[0761] The server parses the text data.

[0762] The server uses the Google Cloud Natural Language API to analyze the text data for sentiment scores and sentiment intensity, which serve as input for the next processing step.

[0763] Step 6:

[0764] The server detects an abnormality.

[0765] The server determines an abnormality if the emotion score of the analysis result exceeds a certain threshold. For example, if the emotion score is less than -0.5, it determines an abnormality. The result of this anomaly detection becomes the input for the next processing step.

[0766] Step 7:

[0767] The server sends a notification.

[0768] If an anomaly is detected, the server uses the communication service API to send a notification to pre-defined contacts, and the content of this notification becomes the input for the next processing step.

[0769] Step 8:

[0770] A notification device receives the notification.

[0771] The notification device (smartphone or smartwatch) receives the notification sent from the server, and this notification is output to notify the user or other relevant parties of the abnormality.

[0772] Step 9:

[0773] The server moderates the conversation.

[0774] The server adjusts the conversation content depending on the user's emotional state. For example, if the user is angry, it switches to a calmer topic. This adjusted conversation content becomes the input for the next processing step.

[0775] Step 10:

[0776] The terminal provides the adjusted conversation content to the user.

[0777] The terminal provides the user with the adjusted conversation content sent from the server, and this provided conversation content becomes the output for the user.

[0778] The above processing steps realize a system that analyzes the content of a user's speech in real time and takes appropriate action if an abnormality is detected.

[0779] Example 3

[0780] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0781] Modern society demands systems that comprehensively manage users' mental and physical health and provide appropriate advice. However, existing systems struggle to effectively collect and analyze users' lifestyle and health data and provide appropriate advice. Furthermore, collaboration with local communities and the collection and analysis of users' emotional data are insufficient, making it difficult to prevent users from becoming isolated or their mental health from deteriorating. To address these issues, systems with more advanced data analysis technology and collaboration capabilities are needed.

[0782] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0783] In this invention, the server includes a means for collecting a user's lifestyle and health data, a means for preprocessing the collected data, and a means for analyzing the preprocessed data using a generative AI model. This enables highly accurate analysis of the user's health status and lifestyle patterns and the provision of appropriate advice for improving their physical condition. Furthermore, by including a means for providing an interface for collecting the user's emotional data and a means for collecting the user's feedback, the server can comprehensively manage the user's mental health status and realize early intervention and appropriate care. Furthermore, by including a means for supporting the realization of a monitoring society through collaboration with the local community, it becomes possible to prevent user isolation and establish a monitoring system throughout the community.

[0784] "Voice input" is a means by which a user provides information to a system through speech.

[0785] "Mental stability" is a state that maintains the user's mental balance and reduces stress and anxiety.

[0786] "Brain activation" is a state that improves the user's brain function and enhances cognitive ability and memory.

[0787] "Conversational state" refers to the content and quality of the voice interaction a user has with a system.

[0788] "Dementia tendency" refers to a state in which the user may be showing early symptoms of dementia.

[0789] An "abnormality" is a state in which a user's health condition or behavior is deemed to be different from normal or problematic.

[0790] "Medical institution" refers to a facility that manages the health of users, such as a hospital or clinic.

[0791] A "facility" is a specialized institution that supports the lives and health of users.

[0792] "Family" refers to the User's immediate family and close relatives.

[0793] "IoT" is a network of physical devices and sensors connected via the internet.

[0794] A "healthcare system" is an information system that supports users in managing their health.

[0795] "Health Improvement Advice" refers to specific suggestions and guidance for improving the user's health.

[0796] "Local community" refers to the residents and stakeholders of the area where the user lives.

[0797] A "watchful society" is a system in which the entire community watches over the safety and health of users.

[0798] "Lifestyle data" is information about the user's daily life.

[0799] "Health Data" means information relating to a User's health status.

[0800] "Preprocessing" is the process of preparing collected data in a form that is easy to analyze.

[0801] A "generative AI model" is a model that uses machine learning algorithms to analyze data and make predictions and classifications.

[0802] "Analysis" is the process of evaluating the user's condition based on collected data.

[0803] An "interface" is the means by which a user interacts with a system.

[0804] "Feedback" refers to the evaluations and opinions that users provide to the system.

[0805] MODE FOR CARRYING OUT THE INVENTION

[0806] The present invention is a system for collecting lifestyle and health data of a user, analyzing the data, and providing appropriate advice for improving physical condition. A specific embodiment of this system will be described below.

[0807] server

[0808] The server collects users' lifestyle and health data, including heart rate, step count, and sleep patterns from IoT devices such as smartwatches and fitness trackers. It also obtains health checkup results and medical history from electronic medical record systems. These data are managed as data frames using the Python Pandas library.

[0809] Next, the server preprocesses the collected data, specifically by imputing missing values, removing outliers, normalizing the data, etc. For example, it uses the Pandas fillna method to impute missing values ​​with the mean value.

[0810] The preprocessed data is then analyzed using generative AI models. Machine learning models are trained using TensorFlow and PyTorch to analyze the user's health and lifestyle patterns. For example, a model is used to predict stress levels using the user's heart rate data as input.

[0811] Based on the analysis results, the server generates appropriate advice to improve health. For example, if the user is feeling stressed, it will suggest relaxation or meditation techniques. If the user is feeling joy, it will suggest activities and habits to maintain that joy.

[0812] The generated advice is sent to the user's device using a messaging service.

[0813] Terminal

[0814] The device receives the advice to improve one's physical condition sent from the server. This corresponds to the user's smartphone or tablet. The device then displays the received advice to the user. Specifically, the advice is displayed on the app's notification screen or on a dedicated dashboard. For example, the app may display a message saying, "Try deep breathing techniques for relaxation."

[0815] The device also provides an interface for collecting user emotional data. For example, the user can record their emotions in a diary app, and the data is sent to the server. Specifically, the app displays a prompt such as "Please choose your mood today."

[0816] user

[0817] Users receive advice on how to improve their physical condition through their device. For example, they can check their smartphone notifications and practice relaxation techniques. Users can also record their emotions and health status in a diary app or similar and send that data to a server. For example, they can record in the app that they felt stressed today.

[0818] If the user continues to display sad emotions over a period of time, the system will notify medical institutions and family members, allowing for early intervention and appropriate care.

[0819] Specific examples

[0820] Examples of prompts to input to a generative AI model include:

[0821] 1. "Analyze user's heart rate data and assess their stress level."

[0822] 2. "Analyze users' sleep patterns and provide advice for improvement."

[0823] 3. "Please suggest appropriate relaxation methods based on the user's recent emotions."

[0824] In this way, a system is realized that comprehensively manages the lifestyle habit data and health data of the user and provides appropriate advice for improving physical condition. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0825] Step 1:

[0826] The server collects users' lifestyle and health data. As input, it receives data such as heart rate, step count, and sleep patterns from smartwatches and fitness trackers, as well as health checkup results and medical history from electronic medical record systems. It stores this data in a database. Specifically, it retrieves data from devices and systems through APIs and converts it into a data frame using Python's Pandas library.

[0827] Step 2:

[0828] The server preprocesses the collected data. As input, it receives the raw data collected in step 1. It imputes missing values ​​in the data, removes outliers, and normalizes the data. Specifically, it uses Pandas' fillna method to impute missing values ​​with the mean value, and detects and removes outliers. It also uses Scikit-learn's StandardScaler to scale the data. As output, it obtains preprocessed, clean data.

[0829] Step 3:

[0830] The server analyzes the preprocessed data using a generative AI model. It receives the preprocessed data from step 2 as input. It uses a generative AI model (e.g., a machine learning model using TensorFlow or PyTorch) to analyze the user's health condition and lifestyle patterns. Specifically, it runs a model that predicts stress levels using heart rate data as input. It obtains the analysis results (e.g., stress level assessment) as output.

[0831] Step 4:

[0832] The server generates appropriate advice for improving physical condition based on the analysis results. It receives the analysis results obtained in step 3 as input. It uses the generative AI model to generate advice according to the user's condition. Specifically, if the stress level is high, it suggests relaxation or meditation techniques, and if the user is feeling joy, it suggests activities and habits to maintain that joy. The generated advice is obtained as output.

[0833] Step 5:

[0834] The server notifies the user's device of the generated advice. As input, it receives the advice generated in step 4. It uses a messaging service to send a push notification to the user's smartphone or tablet. Specifically, it creates and sends a notification message using the messaging service's API. As output, the notification is displayed on the user's device.

[0835] Step 6:

[0836] The device receives health improvement advice sent from the server and displays it to the user. As input, it receives a notification message from the server. Specific operations include displaying the advice using the smartphone's notification function and providing detailed information within the app. As output, the user can confirm the advice.

[0837] Step 7:

[0838] The device provides an interface for collecting user emotion data. As input, it receives input about the user's emotions. As specific operations, it provides a diary application and emotion recording function to allow the user to record their emotions. As output, it sends the recorded emotion data to a server.

[0839] Step 8:

[0840] The user receives advice on how to improve their health through their device and puts it into practice. As input, they receive a notification of the advice from their device. As a specific action, they check the notification and practice the suggested relaxation techniques or meditation methods. As output, the user's health condition improves.

[0841] Step 9:

[0842] The user records their emotions and health status in a diary app or similar and sends the data to a server. The input is information about daily emotions and health status. A specific action is to record "I felt stressed today" in the app. The output is the recorded data sent to the server.

[0843] Step 10:

[0844] The server collects user feedback and uses it to improve the system. As input, it receives feedback from users. Specific operations include providing feedback forms and survey functions so that users can report on the effectiveness of advice and areas for improvement. As output, it analyzes the collected feedback data and reflects it in improving the system.

[0845] (Application example 3)

[0846] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0847] In modern society, maintaining users' mental and physical health is an important issue. However, in environments where users tend to be isolated, it is difficult for them to receive appropriate support and advice. It is also difficult to detect changes in users' emotions and health status early and take appropriate measures. Furthermore, there is a need to strengthen cooperation with local communities and establish a monitoring system.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia trends; a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; a means for providing advice on improving health through collaboration with IoT and healthcare systems; a means for supporting the realization of a monitoring society through collaboration with the local community; a means for collecting and analyzing the user's lifestyle data and health data; a means for tracking changes in the user's emotions using an emotion engine and providing necessary support; a means for sharing the user's data with relevant people in the local community; and a means for notifying medical institutions of changes in the user's emotions. This makes it possible to comprehensively support the user's mental and physical health and strengthen collaboration with the local community.

[0849] "Voice input" is a technology that recognizes the words a user speaks as digital data and inputs it into a system.

[0850] "Mental stability" refers to stabilizing the user's mental state and reducing stress and anxiety.

[0851] "Brain activation" refers to enhancing the user's brain function and improving cognitive ability and memory.

[0852] "Conversation status" refers to information such as the content, tone, and frequency of a user's conversations.

[0853] "Dementia tendency" refers to a state in which a user may develop dementia.

[0854] "Abnormal" refers to a state in the user's health or emotional state that is different from normal and is deemed problematic.

[0855] "Hospitals, Facilities, and Families" refers to medical institutions, nursing homes, and family members of users who may receive information about their health and emotional state.

[0856] "IoT" refers to physical devices and sensors connected via the internet.

[0857] "Healthcare system" refers to a system for managing and improving the health status of users.

[0858] "Health Improvement Advice" refers to specific suggestions or instructions to improve a user's health.

[0859] "Local community" refers to the residents and stakeholders of the area where the user lives.

[0860] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[0861] "Lifestyle data" refers to data related to a user's daily life (e.g., diet, exercise, sleep, etc.).

[0862] "Health Data" refers to data relating to a User's health status (e.g., heart rate, blood pressure, body temperature, etc.).

[0863] "Emotion engine" refers to technology for analyzing a user's emotional state and tracking changes.

[0864] "Emotional changes" refers to the user's emotional state changing over time.

[0865] "Necessary support" refers to appropriate support and interventions provided based on the user's health and emotional state.

[0866] "Community stakeholders" refers to local residents and organizations involved in the lives and health of users.

[0867] "Medical institution" refers to a facility that receives information about a user's health condition and provides appropriate medical services.

[0868] The system for implementing this invention is designed to comprehensively support the mental and physical health of users and strengthen ties with the local community. Specific embodiments of the system are described below.

[0869] 1. System Configuration

[0870] The system consists of the following major components:

[0871] Server: The central location for data collection, analysis, advice generation, and notifications.

[0872] Device: The smartphone or IoT device (e.g., smartwatch, fitness tracker) used by the user.

[0873] User: An individual who uses the system.

[0874] 2. Data Collection

[0875] The device collects the user's lifestyle and health data, specifically heart rate, sleep data, exercise data, etc. from smartwatches and fitness trackers, and transmits this data to a server.

[0876] 3. Data analysis

[0877] The server analyzes the collected data using machine learning models and an emotion engine, which evaluates the user's health status and emotional changes and generates appropriate advice.

[0878] 4. Advice Generation

[0879] The server generates health improvement advice based on the analysis results. For example, if the user is feeling stressed, it will suggest relaxation or meditation techniques. If the user is feeling joyful, it will suggest activities and habits to maintain that joy.

[0880] 5. Data Sharing and Notification

[0881] The server shares user data with relevant parties in the local community, creating a community-wide monitoring system. It also notifies medical institutions of changes in the user's emotions, enabling early intervention and appropriate care.

[0882] 6. Hardware and Software Used

[0883] Hardware: smartphones, smartwatches, fitness trackers

[0884] Software: Python, APIs (e.g., healthcare API, community API, sentiment analysis API)

[0885] Specific examples

[0886] User A: Heart rate and sleep data are collected from the smartwatch, and if stress levels are determined to be high, relaxation advice is provided.

[0887] User B: The emotion engine detects prolonged sadness and notifies medical authorities.

[0888] Prompt Sentence Examples

[0889] Create a program that collects and analyzes users' lifestyle and health data to generate advice on how to improve their health. Also, create a monitoring system in collaboration with the local community and add features to track changes in emotions using an emotion engine.

[0890] The above is a specific embodiment for carrying out the present invention. This system makes it possible to comprehensively support the mental and physical health of users and strengthen ties with the local community.

[0891] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0892] Step 1:

[0893] The device collects the user's lifestyle and health data. Specifically, it obtains heart rate, sleep data, exercise data, etc. from a smartwatch or fitness tracker. This data is sent from the device to a server. The input is the user's biometric data, and the output is the data sent to the server.

[0894] Step 2:

[0895] The server saves the received data and stores it in a database. The input is the user's biometric data sent from the terminal, and the output is the data stored in the database. Specific operations include unifying the data format and storing it appropriately in the database.

[0896] Step 3:

[0897] The server analyzes the data stored in the database. It uses machine learning models and an emotion engine to evaluate changes in the user's health status and emotions. The input is the user's biometric data stored in the database, and the output is the analysis results. Specific operations include data preprocessing, feature extraction, input to the model, and obtaining prediction results.

[0898] Step 4:

[0899] The server generates advice to improve physical condition based on the analysis results. For example, if the user is feeling stressed, it will suggest relaxation or meditation techniques. The input is the analysis results, and the output is the generated advice. Specifically, it selects appropriate advice based on the analysis results and generates a message to provide to the user.

[0900] Step 5:

[0901] The server sends the generated advice to the terminal. The input is the generated advice, and the output is the advice sent to the terminal. Specific operations include converting the advice into a format that is easy for the user to understand and sending it to the terminal.

[0902] Step 6:

[0903] The terminal notifies the user of the received advice. The input is the advice sent from the server, and the output is the advice displayed to the user. Specific operations include displaying the advice to the user using the notification function.

[0904] Step 7:

[0905] The server shares the user's data with the community stakeholders. The input is the user's biometric data and the analysis results, and the output is the data sent to the community stakeholders. Specific operations include converting the data into an appropriate format and sending it to the stakeholders.

[0906] Step 8:

[0907] The server notifies the medical institution of changes in the user's emotions. The input is the analysis result by the emotion engine, and the output is a notification sent to the medical institution. Specific operations include detecting changes in emotions and notifying the medical institution at the appropriate time.

[0908] The above are the specific processing steps for carrying out the present invention, which make it possible to comprehensively support the mental and physical health of users and strengthen ties with the local community.

[0909] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0910] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0911] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0912] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0913] [Second embodiment]

[0914] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0915] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0916] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0917] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0918] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0919] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0920] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0921] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0922] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0923] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0924] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0925] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0926] "Example 1"

[0927] The system of the present invention uses speech recognition technology to converse with the user. This conversation contributes to the user's mental stability and brain activity. Specifically, it receives voice input from the user, converts it into text, and generates an appropriate response. This response is then conveyed to the user using speech synthesis technology.

[0928] "Example 2"

[0929] The system of the present invention also analyzes the state of conversation. This analysis uses natural language processing and machine learning technologies. Specifically, it detects abnormalities such as dementia trends from the content, speech patterns, and frequency of the user's speech. Detected abnormalities are notified to hospitals, facilities, family members, etc.

[0930] "Example 3"

[0931] Furthermore, the system of the present invention provides advice on improving health by linking with IoT and healthcare systems. Specifically, it collects and analyzes the user's lifestyle and health data. Based on the results, it generates appropriate advice and delivers it to the user.

[0932] "Example 4"

[0933] Furthermore, the system of the present invention supports the realization of a society where people are monitored in cooperation with the local community. Specifically, the system shares information about the user's living situation and health status with relevant people in the local community. This prevents users from becoming isolated and builds a monitoring system that involves the entire community.

[0934] The processing flow of each embodiment will be described below.

[0935] "Example 1"

[0936] Step 1: Receive speech input from the user. This speech input is part of a conversation between the user and the system.

[0937] Step 2: Convert the received voice input into text using voice recognition technology.

[0938] Step 3: Analyze the converted text and generate an appropriate response, using natural language processing and machine learning techniques.

[0939] Step 4: The generated response is converted into speech and conveyed to the user. This speech conversion is achieved using speech synthesis technology.

[0940] "Example 2"

[0941] Step 1: Analyze the state of the conversation with the user. This analysis uses natural language processing and machine learning technologies.

[0942] Step 2: Detect abnormalities such as dementia trends from the analysis results.

[0943] Step 3: Notify the hospital, facility, family, etc. of the detected abnormality. This notification is made via communication methods such as email, messaging apps, and telephone.

[0944] "Example 3"

[0945] Step 1: Collect user lifestyle and health data from IoT and healthcare systems.

[0946] Step 2: Analyze the collected data. This analysis uses statistical methods and machine learning techniques.

[0947] Step 3: Based on the analysis results, appropriate advice for improving health is generated.

[0948] Step 4: The generated advice is communicated to the user using voice output, text output, or a graphical display.

[0949] "Example 4"

[0950] Step 1: Share your living situation and health status with relevant people in your community via email, messaging apps, phone calls, or other communication methods.

[0951] Step 2: Collaborate with relevant parties in the local community to establish a system for monitoring users. This collaboration will involve local public institutions, volunteer groups, and local residents.

[0952] Example 1

[0953] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0954] In modern society, there is a demand for methods to stabilize users' mental health and stimulate their brains. It is also important to detect abnormalities such as dementia early and take appropriate measures. However, systems to solve these issues have not yet been fully developed. Furthermore, there is a need for advice on improving health and for a society that monitors users through cooperation with the local community.

[0955] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation, a means for converting the user's voice into text using voice recognition technology, a means for generating an appropriate response using a generative AI model, a means for converting the generated response into voice using text-to-speech conversion technology, a means for analyzing the state of the conversation and detecting abnormalities such as dementia tendencies, a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, a means for providing advice on improving health through collaboration with IoT and healthcare systems, and a means for supporting the realization of a monitoring society through collaboration with the local community. This enables early detection of abnormalities and appropriate responses while promoting the user's mental stability and brain activation.

[0956] "Voice input" is a technology that captures the voice a user speaks through a microphone as digital data.

[0957] "Mental stability" refers to maintaining or improving the user's psychological stability.

[0958] "Brain activation" refers to stimulating the user's brain functions and improving their cognitive and thinking abilities.

[0959] "Speech recognition technology" is a technology that converts voice data into text data.

[0960] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input.

[0961] "Text-to-speech conversion technology" is a technology that converts text data into voice data.

[0962] "Means for analyzing the state of conversation" refers to technology that analyzes the content and patterns of user speech and detects abnormalities.

[0963] "Dementia tendency" refers to a state in which a user shows early symptoms of dementia.

[0964] "Means for detecting anomalies" refers to technology that discovers anomalies from users' statements and behavior.

[0965] "Means of notifying hospitals, facilities, families, etc." refers to technology that notifies relevant parties of detected abnormalities.

[0966] "IoT" is a technology that allows various devices to connect via the Internet.

[0967] A "healthcare system" is a system for managing and improving the health status of users.

[0968] "Health Improvement Advice" refers to providing advice to improve a user's health.

[0969] "Collaboration with the local community" refers to working with the local community in which the user lives to monitor and provide support.

[0970] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[0971] This invention is a system for promoting mental stability and brain activation for users, and is realized using voice input, voice recognition, generative AI models, and text-to-speech conversion technologies. Specific embodiments of this system are described below.

[0972] Hardware and Software Configuration

[0973] 1. Voice Input

[0974] The user inputs voice through a microphone. The device captures this voice and sends it to the server as digital data. The hardware used is a microphone and a voice capture device.

[0975] 2. Voice Recognition

[0976] The server uses speech recognition technology to convert the received voice data into text data. Specifically, it uses a speech recognition API (for example, a general speech recognition service). This API provides highly accurate speech recognition and accurately converts the user's speech into text.

[0977] 3. Response Generation

[0978] The server uses a generative AI model (e.g., a general generative AI model) to generate appropriate responses to the user's textual utterances. The model generates responses that include content that contributes to the user's mental stability and brain activity.

[0979] 4. Text to speech conversion

[0980] The generated response text is converted into audio data using text-to-speech conversion technology. Specifically, a speech synthesis API (e.g., a general speech synthesis service) is used. This API provides natural speech synthesis and generates audio that is easy for the user to understand.

[0981] 5. Providing voice responses

[0982] The terminal receives the voice data sent from the server and provides a response to the user through the speaker, allowing the user to continue the conversation with the system. The hardware used is a speaker and an audio playback device.

[0983] Specific examples

[0984] Example 1: When a user is stressed

[0985] 1. User: "I'm really stressed out today."

[0986] 2. Device: Captures audio with a microphone and sends it to the server.

[0987] 3. Server: Uses speech recognition API to convert "I'm very stressed today" into text.

[0988] 4. Server: Uses a generative AI model to generate a response such as "That's tough. Try taking a deep breath to relax."

[0989] 5. Server: Convert the response into speech using a speech synthesis API.

[0990] 6. Device: Through the speaker, the user is told, "That's tough. Try taking a deep breath to relax."

[0991] Example 2: When the user wants to stimulate their brain

[0992] 1. User: "I want to learn something new today."

[0993] 2. Device: Captures audio with a microphone and sends it to the server.

[0994] 3. Server: Use speech recognition API to convert "I want to learn something new today" into text.

[0995] 4. Server: Uses a generative AI model to generate a response like, "That's great! Want to talk about some recent advances in science and technology?"

[0996] 5. Server: Convert the response into speech using a speech synthesis API.

[0997] 6. Terminal: Through the speaker, the user is told, "That's great! Shall I tell you about some recent advances in science and technology?"

[0998] Prompt Sentence Examples

[0999] "How do I respond if a user is stressed?"

[1000] "When a user says they want to learn something new, how do I respond?"

[1001] This system combines voice recognition technology with generative AI models to enable natural conversations with users, contributing to mental stability and brain activation.

[1002] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1003] Step 1:

[1004] The user inputs voice through a microphone. The device captures this voice and stores it as digital data. Specifically, the device's microphone picks up the user's voice and stores it as voice data. This data is sent to the server in real time. The input is the user's voice, and the output is digital voice data.

[1005] Step 2:

[1006] The server uses voice recognition technology to convert the received voice data into text data. Specifically, the server sends the voice data to a voice recognition API, which analyzes the voice and converts it into text. The converted text is returned to the server and passed on to the next processing step. The input is digital voice data, and the output is text data.

[1007] Step 3:

[1008] The server uses a generative AI model to generate appropriate responses to the user's textual utterances. Specifically, the server inputs text data into the generative AI model, which then generates a response based on the user's utterances. This response includes content that contributes to the user's mental stability and brain activation. The input is text data, and the output is a response text.

[1009] Step 4:

[1010] The generated response text is converted into audio data using text-to-speech technology. Specifically, the server sends the response text to a speech synthesis API, which analyzes the text and generates natural-sounding audio. This audio data is returned to the server and passed on to the next processing step. The input is the response text, and the output is audio data.

[1011] Step 5:

[1012] The terminal receives the voice data sent from the server and provides a response to the user through the speaker. Specifically, the terminal receives the voice data and plays it through the speaker, allowing the user to continue a conversation with the system. The input is the voice data, and the output is a voice response to the user.

[1013] (Application example 1)

[1014] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1015] In modern society, there is a demand for methods to stabilize users' mental health and stimulate their brains. It is also important to detect abnormalities such as dementia early and take appropriate measures. Furthermore, there is a demand to improve the quality of life by enabling users to easily use food delivery services. However, it is difficult to realize these functions in a single system, and an efficient solution is needed.

[1016] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1017] In this invention, the server includes: a means for conducting a conversation via voice input, contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia; a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; a means for providing health improvement advice through collaboration with IoT and healthcare systems; a means for supporting the realization of a watchful society through collaboration with the local community; a means for converting voice input into text, generating an appropriate response, and conveying it to the user using speech synthesis technology; a means for placing a food delivery order based on the user's voice input; and a means for generating an appropriate response based on the user's order using a generative AI model. This enables the early detection of abnormalities such as dementia and appropriate responses to be taken while promoting the user's mental stability and brain activation. It also allows users to easily use food delivery services, improving their quality of life.

[1018] "Voice input" refers to a user inputting voice through a device such as a microphone.

[1019] "Conversation" refers to the exchange of information between a user and a system via voice.

[1020] "Mental stability" refers to maintaining or improving the user's psychological stability.

[1021] "Brain activation" means stimulating and activating the user's brain functions.

[1022] "Conversational state" refers to the content and quality of the conversation between the user and the system.

[1023] "Dementia tendency" refers to a state in which the user may develop dementia.

[1024] An "abnormality" is a phenomenon or symptom that deviates from the normal state.

[1025] "Hospitals, facilities, families, etc." refers to medical institutions, nursing homes, families, etc. that are involved in the user's health condition.

[1026] "Notification" is the act of conveying specific information to others.

[1027] "IoT" is a technology that connects various devices to each other via the Internet.

[1028] A "healthcare system" is a system that supports users in managing their health.

[1029] "Health Improvement Advice" refers to advice or suggestions for improving the user's health.

[1030] "Local community" refers to the local community in which the user lives.

[1031] A "watchful society" is a system in which the entire local community watches over the safety and health of users.

[1032] "Speech synthesis technology" refers to the technology that converts text data into speech.

[1033] "Food delivery" refers to a service that delivers meals ordered by users to a specified location.

[1034] A "generative AI model" is a model that uses artificial intelligence to generate responses or results for a specific task.

[1035] An "appropriate response" is an appropriate and helpful reply to the user's input.

[1036] A system for implementing this invention provides a function that contributes to the user's mental stability and brain activation by conducting conversations via voice input. The system realizes interaction with the user using voice recognition technology, voice synthesis technology, and generative AI models.

[1037] Hardware and Software Configuration

[1038] The system uses the following hardware and software:

[1039] Hardware: microphone, speakers, smartphone or computer

[1040] Software: Speech recognition library, text-to-speech library, generative AI model

[1041] Processing flow

[1042] 1. Acquiring voice input:

[1043] The user provides voice input through a microphone, which is converted to text using a speech recognition library.

[1044] 2. Response generation using generative AI models:

[1045] The converted text is then fed into a generative AI model (e.g., OpenAI's API) to generate an appropriate response. An example prompt is "The user wants to order food delivery. Please generate an appropriate response based on the following order details: I would like to order pizza."

[1046] 3. Providing a response via speech synthesis:

[1047] The generated response is converted into audio using a text-to-speech library and provided to the user through the speaker.

[1048] Specific examples

[1049] For example, if a user says "I want to order a pizza," the system will do the following:

[1050] 1. The user's voice input "I would like to order pizza" is picked up by the microphone.

[1051] 2. The speech recognition library converts the speech into text: "I would like to order a pizza."

[1052] 3. The generative AI model generates a response based on the prompt: "The user wants to order food delivery. Please generate an appropriate response based on the following order: I would like to order pizza."

[1053] 4. The generated response, "What type and size of pizza do you want?" is converted into speech by the speech library and provided to the user through the speaker.

[1054] In this way, users can place food delivery orders through voice input, and the system will provide appropriate responses, contributing to the user's mental well-being and brain activity.

[1055] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1056] Step 1:

[1057] The user inputs voice through the microphone. The input is the user's voice, and the output is voice data. The terminal acquires this voice data and passes it to the next step.

[1058] Step 2:

[1059] The device converts voice data into text using a speech recognition library. The input is voice data and the output is text data. Specifically, the speech recognition engine analyzes the voice waveform and generates the corresponding text.

[1060] Step 3:

[1061] The server uses a generative AI model to generate an appropriate response based on the text data. The input is text data, and the output is response text. Specifically, the text data is input to the generative AI model as a prompt sentence, and the response text is obtained. An example of a prompt sentence is, "The user says they want to order food delivery. Please generate an appropriate response based on the following order details: I would like to order pizza."

[1062] Step 4:

[1063] The server converts the response text into speech using a text-to-speech library. The input is the response text and the output is audio data. Specifically, the response text is input to a speech synthesis engine, which generates a corresponding audio file.

[1064] Step 5:

[1065] The terminal provides the generated voice data to the user through a speaker. The input is the voice data, and the output is the voice that the user hears. Specifically, the terminal plays the voice file and conveys the response to the user.

[1066] In this way, users can place food delivery orders through voice input, and the system will provide appropriate responses, contributing to the user's mental well-being and brain activity.

[1067] Example 2

[1068] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1069] In an aging society, early detection of dementia and appropriate treatment are important. However, early detection is difficult because the signs of dementia are often overlooked in daily life. There is also a lack of effective means to stabilize users' mental health and activate their brains. Furthermore, there is a need for a system that can quickly and appropriately notify users when an abnormality is detected.

[1070] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1071] In this invention, the server includes a means for conducting a conversation via voice input, contributing to the user's mental stability and brain activation, a means for analyzing the state of the conversation and converting the user's speech into text data using natural language processing technology, a means for preprocessing the converted text data and inputting it into a machine learning model to analyze the user's speech patterns and frequency, a means for detecting abnormalities based on the analysis results, a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, a means for providing advice on improving health by linking with IoT and healthcare systems, and a means for supporting the realization of a monitoring society by linking with the local community, thereby enabling early detection of dementia and appropriate treatment.

[1072] "Voice input" is a technology that collects the user's spoken voice as digital data.

[1073] "Mental stability" refers to maintaining or improving the user's psychological stability.

[1074] "Brain activation" refers to stimulating the user's brain functions and improving cognitive ability and memory.

[1075] "Conversational state" refers to the overall state of a user's speech, including their speech content, speech patterns, and speech frequency.

[1076] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1077] "Text data" refers to data obtained by converting voice data into character information.

[1078] "Preprocessing" refers to the process of cleaning and normalizing data before inputting it into a machine learning model.

[1079] A "machine learning model" refers to an algorithm that learns patterns based on data and makes predictions and classifications.

[1080] "Speech patterns" refer to characteristic tendencies and repetitions in a user's speech.

[1081] "Utterance frequency" refers to the number of times a user speaks within a certain period of time.

[1082] "Abnormal" refers to deviations from normal speech patterns or frequencies, including signs of dementia.

[1083] "Notification" refers to the means of communicating information to inform relevant parties of a detected abnormality.

[1084] "IoT" refers to technology that connects physical devices and systems to each other through the Internet.

[1085] "Healthcare system" refers to a system for providing health management and medical services.

[1086] "Health Improvement Advice" refers to specific suggestions or instructions to improve a user's health.

[1087] "Community" refers to the geographic or social community to which a User belongs.

[1088] A "watchful society" refers to efforts by the entire local community to support the elderly and vulnerable and ensure their safety.

[1089] The present invention provides a system that collects user utterance data and analyzes the state of the conversation using natural language processing technology and machine learning technology. Specific embodiments of this system are described below.

[1090] First, the user speaks using a device (such as a smartphone or a PC with a microphone). The device records the user's speech in real time and sends the audio data to the server. The server then converts the received audio data into text using speech recognition software (such as the Google Cloud Speech-to-Text API).

[1091] The server then preprocesses the converted text data, which includes removing unnecessary spaces and special characters, normalizing words (e.g., "run" -> "run"), etc. The preprocessed text data is then input to a machine learning model using a machine learning framework (e.g., TensorFlow or PyTorch).

[1092] The server inputs the preprocessed text data into a machine learning model to analyze the user's speech patterns and frequency. This analysis detects abnormalities in the content, patterns, and frequency of the user's speech. For example, if the user's speech suddenly decreases or they start using certain words or phrases more frequently, this could be a sign of an abnormality, such as dementia.

[1093] Any detected abnormalities are notified to the hospital, facility, or family member via a notification service such as a cloud-based communication service. The notification includes details of the abnormality and recommended countermeasures.

[1094] As a concrete example, the following prompt sentence can be input to a generative AI model:

[1095] Example prompt sentence:

[1096] "A 70-year-old male user has recently started using certain words more frequently in conversations. The frequency of his speech has also decreased. This user's conversation data will be analyzed, and if any abnormalities are detected, his family will be notified."

[1097] Based on this prompt, the generative AI model generates the following output:

[1098] Example output of a generative AI model:

[1099] "Analysis of the user's conversation data has confirmed that certain words are being used frequently. In addition, the frequency of speech has decreased, suggesting a tendency toward dementia. This abnormality will be notified to the user's family."

[1100] In this way, the system can analyze the user's conversation state in detail, detect abnormalities, and provide appropriate notifications.

[1101] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1102] Step 1:

[1103] The user uses the terminal to have a conversation, and the terminal records the user's speech in real time and transmits the audio data to the server.

[1104] Input: User's speech

[1105] Output: Audio data sent to the server

[1106] Step 2:

[1107] The server converts the received voice data into text data using voice recognition software, for example, the Google Cloud Speech-to-Text API.

[1108] Input: Audio data

[1109] Output: Text data

[1110] Step 3:

[1111] The server preprocesses the converted text data, which includes removing unnecessary spaces and special characters, and normalizing words.

[1112] Input: Text data

[1113] Output: Preprocessed text data

[1114] Step 4:

[1115] The server then inputs the preprocessed text data into a machine learning model to analyze the user's speech patterns and frequency, using, for example, TensorFlow or PyTorch.

[1116] Input: Preprocessed text data

[1117] Output: Analysis of speech patterns and frequency

[1118] Step 5:

[1119] The server detects anomalies based on the output of the machine learning model. For example, if a user's speech suddenly decreases or they start using certain words or phrases more frequently, this could be a sign of dementia or other abnormalities.

[1120] Input: Analysis results of speech patterns and frequency

[1121] Output: Anomaly detection results

[1122] Step 6:

[1123] The server notifies the hospital, facility, or family of the detected abnormality using a notification service, such as a cloud-based communication service. The notification includes details of the abnormality and recommended countermeasures.

[1124] Input: Anomaly detection result

[1125] Output: Notification message

[1126] In this way, the system can analyze the user's conversation state in detail, detect abnormalities, and provide appropriate notifications.

[1127] (Application example 2)

[1128] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1129] In modern society, managing dementia and mental health conditions among the elderly is an important issue. In particular, early detection of dementia and appropriate responses are required, but this is difficult to achieve in daily life. Also, a method is needed to quickly notify appropriate institutions and family members when an abnormality is detected. Furthermore, it is important to safely store user conversation data so that it can be analyzed later.

[1130] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia tendencies; a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; a means for providing health improvement advice through collaboration with IoT and healthcare systems; a means for supporting the realization of a watchful society through collaboration with the local community; a means for monitoring the user's conversation in real time and analyzing the content, patterns, and frequency of the speech using natural language processing technology and machine learning technology; a means for sending notifications to pre-registered hospitals, facilities, and family members when an abnormality is detected; and a means for securely storing the user's conversation data so that it can be analyzed later. This enables management of the user's mental health status, early detection of dementia, and prompt response.

[1131] "Voice input" is a technology in which the user's voice is captured as digital data and the system analyzes that voice.

[1132] "Mental stability" is a state that aims to maintain the user's psychological stability and reduce stress and anxiety.

[1133] "Brain activation" is a state that aims to stimulate the user's brain functions and improve cognitive function and memory.

[1134] "Conversation state" refers to comprehensive information such as the content of a user's speech, speech patterns, and frequency of speech.

[1135] "Dementia trends" are signs or patterns that indicate whether a user is at risk of developing dementia.

[1136] "Abnormal" refers to user behavior or speech that deviates from normal, especially health and safety issues.

[1137] "Notification" refers to the act of transmitting information to pre-registered hospitals, facilities, family members, etc. when an abnormality is detected.

[1138] "IoT" is a technology that allows various devices to communicate with each other and exchange data via the Internet.

[1139] A "healthcare system" is a system for managing and improving a user's health status.

[1140] "Health Improvement Advice" refers to specific suggestions and guidance for improving the user's health.

[1141] "Local community" refers to the local community and social environment in which the user lives.

[1142] A "watchful society" refers to a society in which the entire local community supports the elderly and vulnerable and provides an environment in which they can live in peace of mind.

[1143] "Real-time monitoring" is a technology that monitors user conversations and behavior in real time and analyzes the data instantly.

[1144] "Natural language processing technology" is a technology that allows computers to understand and analyze human language.

[1145] "Machine learning technology" is a technology that allows computers to learn from data and make predictions and classifications.

[1146] "Speech content" refers to the specific words and sentences spoken by the user.

[1147] "Speech patterns" refer to the way a user speaks and the way they use words.

[1148] "Speech frequency" is an indicator of how often a user speaks.

[1149] "Data storage" is the act of safely storing acquired data so that it can be accessed later.

[1150] A system for implementing this invention has the following configuration: First, a user engages in conversation using a smartphone or other voice input device. The voice input device captures the user's speech in real time and transmits the voice data to a server.

[1151] When the server receives the voice data, it first converts it into text using speech recognition software (e.g., a speech recognition library). This text data is then analyzed using natural language processing techniques (e.g., NLTK or spaCy) and machine learning techniques (e.g., scikit-learn). As a result of the analysis, the content of the user's speech, speech patterns, and frequency of speech are extracted.

[1152] Next, the server uses a machine learning model (e.g., dementia detection model) to detect abnormalities, such as dementia trends, from the extracted data. If an abnormality is detected, the server sends a notification to pre-registered hospitals, facilities, and family members. This notification is sent via communication methods such as email or SMS.

[1153] Additionally, the server securely stores user conversation data for later analysis using a database system (e.g., MySQL or PostgreSQL). The stored data can be used for later reanalysis and long-term health management.

[1154] For example, if a user says, "I've been getting more and more forgetful lately," the voice input device captures the utterance and sends it to the server. The server converts the speech into text and uses a machine learning model to detect anomalies. If an anomaly is detected, the server sends a notification to pre-registered family members saying, "An anomaly has been detected: I've been getting more and more forgetful lately."

[1155] Example prompt sentence:

[1156] Develop an application that monitors user conversations in real time and analyzes the content, patterns, and frequency of speech using natural language processing and machine learning technologies. Include a function to send notifications to pre-registered hospitals, facilities, and family members if an anomaly is detected.

[1157] In this way, the invention enables users to manage their mental health and detect dementia early and respond quickly.

[1158] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1159] Step 1:

[1160] A user uses a smartphone or other voice input device to have a conversation. The voice input device captures the user's speech in real time and sends the voice data to a server. The input is the user's voice, and the output is the voice data sent to the server.

[1161] Step 2:

[1162] When the server receives the voice data, it uses voice recognition software (e.g., a voice recognition library) to convert the voice data into text data. The input is voice data, and the output is text data. Specifically, the server calls the voice recognition library, analyzes the voice data, and converts it into text.

[1163] Step 3:

[1164] The server analyzes the text data using natural language processing techniques (e.g., NLTK and spaCy) and machine learning techniques (e.g., scikit-learn). The input is text data, and the output is the analysis results, such as the content of the utterance, speech patterns, and frequency of the utterance. Specifically, the server tokenizes the text data and analyzes the meaning and context of each token.

[1165] Step 4:

[1166] The server uses a machine learning model (e.g., dementia detection model) to detect abnormalities such as dementia trends from the analysis results. The input is the analysis results, and the output is data indicating whether or not there are abnormalities. Specifically, the server inputs the analysis results into the machine learning model and predicts whether or not there are abnormalities.

[1167] Step 5:

[1168] If an abnormality is detected, the server sends a notification to pre-registered hospitals, facilities, and family members. The input is data indicating whether or not an abnormality exists, and the output is a notification message. Specifically, the server generates a notification message and sends it using a communication method such as email or SMS.

[1169] Step 6:

[1170] The server securely stores user conversation data for later analysis. The input is the voice data and the analysis results, and the output is data stored in a database. Specifically, the server stores the data using a database system (e.g., MySQL or PostgreSQL).

[1171] Step 7:

[1172] The stored data will be used for later reanalysis and long-term health management. The input is the stored data, and the output is the reanalysis results and information for health management. Specifically, the server reanalyzes the stored data and provides feedback to the user or medical institution as needed.

[1173] Example 3

[1174] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1175] In modern society, there are a wide variety of systems that support users' health management and mental well-being. However, these systems are specialized in individual functions and lack comprehensive health management and collaboration with the local community. Furthermore, there are limited systems that effectively collect and analyze users' lifestyle and health data and provide appropriate advice. Furthermore, there are insufficient means to prevent users from becoming isolated and to establish a community-wide monitoring system. To address these issues, a system that utilizes IoT devices and generative AI models to provide integrated support for user health management and community monitoring is needed.

[1176] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1177] In this invention, the server includes: means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; means for analyzing the conversation and detecting abnormalities such as dementia; means for notifying medical institutions, facilities, family members, etc. of the detected abnormalities; means for providing health improvement advice through collaboration with IoT and healthcare systems; means for supporting the realization of a monitoring society through collaboration with the local community; means for collecting the user's lifestyle and health data; means for storing the collected data in cloud storage; means for analyzing the stored data using a data analysis tool; means for evaluating the user's health status using a generative AI model; means for generating appropriate health improvement advice based on the analysis results; means for notifying the user's device of the generated advice; means for sharing the user's data with relevant parties in the local community; and means for establishing a monitoring system for the entire local community based on the shared data. This provides comprehensive support for the user's health management and mental stability and enables the establishment of a monitoring system in collaboration with the local community.

[1178] "Voice input" is a means by which a user provides information to a system using speech.

[1179] "Mental stability" refers to stabilizing the user's mental state and reducing stress and anxiety.

[1180] "Brain activation" refers to improving the user's brain function and enhancing cognitive ability and memory.

[1181] "Conversation status" refers to information such as the content, tone, and frequency of a user's conversations.

[1182] "Dementia tendency" refers to a state in which a user may be showing early symptoms of dementia.

[1183] "Abnormal" refers to a state in which a user's health or behavior is different from normal and is deemed to be problematic.

[1184] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1185] "Facilities" refers to places that support users' health and lifestyles, such as nursing homes and rehabilitation facilities.

[1186] "Family" refers to people related to you by blood or marriage.

[1187] "IoT" refers to physical devices and sensors connected to each other through the internet.

[1188] "Healthcare system" refers to systems and services that assist users in managing their health.

[1189] "Health Improvement Advice" refers to specific instructions or suggestions for improving a user's health.

[1190] "Local community" refers to the residents and stakeholders of the area where the user lives.

[1191] A "watchful society" refers to a system in which the entire local community watches over and supports users' safety and health.

[1192] "Lifestyle data" refers to data related to the user's daily life (e.g., number of steps taken, sleep time, dietary content).

[1193] "Health Data" means data relating to a User's health condition (e.g., body temperature, blood pressure, heart rate).

[1194] "Cloud storage" refers to an online storage service for storing and managing data via the Internet.

[1195] "Data analysis tools" refers to software and platforms for analyzing collected data.

[1196] A "generative AI model" refers to a model that uses machine learning and artificial intelligence techniques to analyze data and make predictions and evaluations.

[1197] "Terminal" refers to the device used by the User (e.g., smartphone, PC).

[1198] "Stakeholders" refers to people involved in supporting the user's health and life (e.g., family members, medical professionals, community volunteers).

[1199] MODE FOR CARRYING OUT THE INVENTION

[1200] The system of the present invention provides integrated support for user health management and community monitoring, and is implemented using voice input, IoT devices, generative AI models, cloud storage, and data analysis tools.

[1201] Hardware and software used

[1202] Hardware: Smartwatches, fitness trackers, smartphones, computers

[1203] Software: Python, R, TensorFlow, PyTorch, MySQL

[1204] Specific operation of the system

[1205] 1. Data Collection

[1206] Users wear IoT devices such as smartwatches and fitness trackers.

[1207] The terminal collects the user's lifestyle data (number of steps, sleep time, heart rate, etc.) and health data (body temperature, blood pressure, etc.) from these devices in real time.

[1208] 2. Data Transmission

[1209] The device periodically sends the collected data to a server, for example, by using Bluetooth or Wi-Fi to upload the data to the cloud.

[1210] 3. Data storage

[1211] The server stores the received data in a database (e.g. MySQL), organized by user, for easy later access.

[1212] 4. Data Analysis

[1213] The server uses data analysis tools such as Python and R to analyze the stored data.

[1214] The server uses a generative AI model (e.g., TensorFlow or PyTorch) to assess the user's health status, for example by comparing it with past data to detect outliers.

[1215] 5. Advice Generation

[1216] Based on the analysis results, the server generates appropriate advice for improving one's physical condition, such as "You've been sleeping less recently, so I recommend you go to bed earlier."

[1217] 6. Advice Notice

[1218] The server then sends the generated advice to the user's smartphone or computer via push notification or email.

[1219] 7. Data Sharing

[1220] A user uses an application to record their living situation and health condition.

[1221] The terminal transmits the recorded data to the server.

[1222] The server shares the user's data with community stakeholders (e.g., family members, health care workers, community volunteers) using a secure communication protocol (e.g., HTTPS).

[1223] 8. Establishing a monitoring system

[1224] Based on the shared data, the server will provide information for building a community-wide monitoring system, such as suggesting a schedule for regular health checks.

[1225] Community stakeholders use the information provided to provide support to prevent users from becoming isolated, for example by visiting or calling them.

[1226] Examples of concrete examples and prompts

[1227] Examples:

[1228] The user wears a smartwatch and it records the number of steps taken and their heart rate each day. For example, if the user has a goal of taking 10,000 steps a day, the smartwatch will collect that data.

[1229] The device sends this data to a smartphone via Bluetooth, and the smartphone then uploads the data to the cloud using Wi-Fi.

[1230] The server receives the data and stores it in a MySQL database, organized by user ID.

[1231] The server runs a Python script and uses a TensorFlow model to analyze the data, detecting an outlier, for example, if the number of steps taken today is significantly lower than the average number of steps taken over the past week.

[1232] The server generates advice such as "You haven't taken many steps today, so you should walk a little more."

[1233] The server sends the generated advice to the user's smartphone via push notification.

[1234] Through the application, users can share their health status with local healthcare professionals and receive the necessary support. For example, healthcare professionals can review the user's data and suggest regular visits.

[1235] Example prompt sentence:

[1236] "Generate advice for improving health based on the user's step count and heart rate data."

[1237] "Please share your health status with local healthcare professionals and provide information to help them establish a monitoring system."

[1238] In this way, the system of the present invention utilizes IoT devices and generative AI models to support the health management of users and the monitoring system of local communities. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1239] Step 1:

[1240] Data collection

[1241] Users wear IoT devices such as smartwatches and fitness trackers.

[1242] The terminal collects the user's lifestyle data (number of steps, sleep time, heart rate, etc.) and health data (body temperature, blood pressure, etc.) from these devices in real time.

[1243] Input: Data from the user's IoT device

[1244] Output: Collected lifestyle and health data

[1245] Step 2:

[1246] Data transmission

[1247] The device periodically sends the collected data to a server, for example, by using Bluetooth or Wi-Fi to upload the data to the cloud.

[1248] Input: Collected lifestyle and health data

[1249] Output: Data sent to the server

[1250] Step 3:

[1251] Data storage

[1252] The server stores the received data in a database (e.g. MySQL), organized by user, for easy later access.

[1253] Input: Data sent to the server

[1254] Output: Data stored in the database

[1255] Step 4:

[1256] Data analysis

[1257] The server uses data analysis tools such as Python and R to analyze the stored data.

[1258] The server uses a generative AI model (e.g., TensorFlow or PyTorch) to assess the user's health status, for example by comparing it with past data to detect outliers.

[1259] Input: Data stored in the database

[1260] Output: Analysis results (such as outlier detection)

[1261] Step 5:

[1262] Advice Generation

[1263] Based on the analysis results, the server generates appropriate advice for improving one's physical condition, such as "You've been sleeping less recently, so I recommend you go to bed earlier."

[1264] Input: Analysis results

[1265] Output: Generated advice for improving your health

[1266] Step 6:

[1267] Advice Notice

[1268] The server then sends the generated advice to the user's smartphone or computer via push notification or email.

[1269] Input: Generated advice for improving your health

[1270] Output: Advice sent to the user's device

[1271] Step 7:

[1272] Data Sharing

[1273] A user uses an application to record their living situation and health condition.

[1274] The terminal transmits the recorded data to the server.

[1275] The server shares the user's data with community stakeholders (e.g., family members, health care workers, community volunteers) using a secure communication protocol (e.g., HTTPS).

[1276] Input: Recorded living and health status data

[1277] Output: Data shared with community stakeholders

[1278] Step 8:

[1279] Establishing a monitoring system

[1280] Based on the shared data, the server will provide information for building a community-wide monitoring system, such as suggesting a schedule for regular health checks.

[1281] Community stakeholders use the information provided to provide support to prevent users from becoming isolated, for example by visiting or calling them.

[1282] Input: Shared data

[1283] Output: Information necessary for establishing a monitoring system and implementation of support activities

[1284] (Application example 3)

[1285] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1286] Maintaining users' mental and physical health is an important issue in modern society. However, proper health management and monitoring are difficult in environments where users tend to be isolated. Furthermore, there is a lack of systems that can effectively collect and analyze users' lifestyle and health data and provide appropriate advice. Furthermore, there is no system in place for rapid response in emergencies. To solve these issues, a system that can comprehensively manage users' health status is needed, linking with IoT and healthcare systems and working with the local community.

[1287] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1288] In this invention, the server includes: means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; means for analyzing the conversation and detecting abnormalities such as dementia; means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; means for providing health improvement advice through collaboration with IoT and healthcare systems; means for supporting the realization of a monitoring society through collaboration with the local community; means for collecting and analyzing the user's lifestyle and health data; means for generating and providing advice for health improvement to the user based on the analysis results; means for sharing the collected data with relevant parties in the local community and establishing a monitoring system; and means for issuing alerts in the event of an emergency and encouraging a rapid response. This allows for comprehensive management of the user's health status, providing appropriate advice, and establishing a monitoring system in collaboration with the local community, enabling a rapid response in the event of an emergency.

[1289] "Voice input" is a technology that allows digital devices to recognize and process the words a user speaks.

[1290] "Mental stability" refers to stabilizing the user's mental state and reducing stress and anxiety.

[1291] "Brain activation" refers to improving the user's brain function and enhancing cognitive ability and memory.

[1292] "Conversation status" refers to information such as the content, tone, and frequency of a user's conversations.

[1293] "Dementia proneness" refers to the likelihood that a user is exhibiting early symptoms of dementia.

[1294] "Abnormal" refers to a state in which a user's health condition or behavior is different from normal.

[1295] "Hospitals, facilities, families, etc." refers to parties involved in managing and monitoring the user's health.

[1296] "IoT" refers to the technology that connects various devices via the Internet and exchanges data.

[1297] "Healthcare system" refers to a system for managing and improving the health status of users.

[1298] "Health Improvement Advice" refers to specific instructions or suggestions for improving a user's health.

[1299] "Local community" refers to the residents and stakeholders of the area where the user lives.

[1300] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[1301] "Lifestyle data" refers to information about a user's daily life (e.g., sleep, exercise, diet, etc.).

[1302] "Health Data" refers to information relating to a User's health status (e.g., heart rate, blood pressure, etc.).

[1303] "Analysis" refers to evaluating collected data and extracting meaningful information.

[1304] "Generating advice" refers to creating specific instructions or suggestions to provide to the user based on the analysis results.

[1305] "Sharing data" refers to exchanging collected information with other parties.

[1306] A "monitoring system" refers to a mechanism for continuously monitoring the safety and health of users and taking necessary measures.

[1307] "Emergency" means a situation that requires immediate attention from the User.

[1308] "Sending an alert" refers to sending a notification to relevant parties in the event of an emergency.

[1309] "Rapid response" refers to taking appropriate action quickly in an emergency.

[1310] The system for implementing this invention is composed of an IoT device for collecting and analyzing a user's lifestyle and health data, a server for processing this data, and a terminal for providing advice to the user. Specific embodiments are described below.

[1311] Hardware Configuration

[1312] IoT devices: Devices designed to collect user health data, such as heart rate monitors, blood pressure monitors, and smartwatches.

[1313] Server: A central processing unit for data collection, analysis, advice generation, and data sharing. It uses Python, the Requests library, and the JSON library.

[1314] Device: A device, such as a smartphone or tablet, that provides advice to users and receives alerts in the event of an emergency.

[1315] Software Configuration

[1316] Data collection module: collects data from IoT devices and sends it to the server.

[1317] Data analysis module: Analyzes collected data and evaluates the user's health status.

[1318] Advice generation module: Generates advice to be provided to the user based on the analysis results.

[1319] Data sharing module: Share collected data with local community stakeholders to build a monitoring system.

[1320] Alert module: Sends alerts in case of emergencies, encouraging rapid response.

[1321] Processing flow

[1322] 1. Data collection: IoT devices collect health data such as the user's heart rate and blood pressure and send it to a server.

[1323] 2. Data analysis: The server analyzes the collected data and evaluates the user's health condition. For example, if the heart rate is high, it generates advice such as "Your heart rate is high. Take a deep breath to relax."

[1324] 3. Providing advice: The generated advice is sent to the terminal and provided to the user.

[1325] 4. Data sharing: Collected data will be shared with relevant people in the local community (family members, medical professionals, local volunteers, etc.) to establish a monitoring system.

[1326] 5. Alerting: In the event of an emergency, alerts will be sent to encourage rapid response.

[1327] Specific examples

[1328] For example, if a user is wearing a heart rate monitor, if their heart rate exceeds 100, the server will generate a message saying, "Your heart rate is high. Take a deep breath to relax." This message will be sent to the device. At the same time, this data will be shared with family and medical professionals, allowing them to take the necessary measures promptly.

[1329] Prompt Sentence Examples

[1330] "Write a Python program that generates relaxation advice when a user's heart rate exceeds 100 and shares the data with family and healthcare professionals."

[1331] The above is a specific embodiment for carrying out the present invention.

[1332] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1333] Step 1:

[1334] Data collection

[1335] The server collects user health data from IoT devices (heart rate monitors, blood pressure monitors, smart watches, etc.). Specifically, the IoT devices periodically measure data such as heart rate and blood pressure and send it to the server. The input is the health data from the IoT devices, and the output is the raw data stored on the server.

[1336] Step 2:

[1337] Data analysis

[1338] The server analyzes the collected health data. Specifically, it uses a Python data analysis library to detect abnormal values ​​in heart rate and blood pressure. For example, if the heart rate exceeds 100, it is considered abnormal. The input is the health data collected in step 1, and the output is the analysis result (e.g., high heart rate).

[1339] Step 3:

[1340] Advice Generation

[1341] The server generates advice to provide to the user based on the results of the data analysis. Specifically, it uses a generative AI model to create appropriate advice. For example, if the heart rate is high, it generates the advice "Your heart rate is high. Take a deep breath to relax." The input is the analysis result from step 2, and the output is the generated advice.

[1342] Step 4:

[1343] Providing advice

[1344] The server sends the generated advice to the device (smartphone or tablet) and provides it to the user. Specifically, it sends the advice to the device using an HTTP request, and the device displays it to the user. The input is the advice generated in step 3, and the output is the advice displayed to the user.

[1345] Step 5:

[1346] Data Sharing

[1347] The server shares the collected health data with stakeholders in the local community (family members, medical professionals, local volunteers, etc.). Specifically, it exports the data in JSON format and sends it to the stakeholders. The input is the health data collected in step 1, and the output is the data shared with the stakeholders.

[1348] Step 6:

[1349] Alert sending

[1350] The server issues an alert in the event of an emergency, encouraging a prompt response. Specifically, if an abnormality is detected, it sends a notification to the relevant parties via SMS or email. The input is the analysis result from step 2, and the output is the issued alert.

[1351] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1352] "Example 1"

[1353] One embodiment of the present invention is a system that incorporates an emotion engine. This system receives a user's voice as input and recognizes the user's emotion from the voice. The emotion engine estimates the emotion based on features extracted from the voice (e.g., pitch, volume, speaking rate, etc.). The emotion engine can classify emotions using a machine learning model. This model is trained from a large amount of voice data and its corresponding emotion labels.

[1354] "Example 2"

[1355] The emotion engine can also adjust the content of the conversation based on the emotions it recognizes. For example, if it recognizes that the user is angry, the system will switch to a more gentle topic or make statements that will calm the user. Conversely, if it recognizes that the user is happy, the system will make statements that share that joy or change to a topic that will amplify that joy.

[1356] "Example 3"

[1357] Furthermore, the emotion engine can track long-term changes in a user's emotions, enabling early detection of changes in the user's mental state and providing necessary support. For example, if a user continues to show sad emotions over a certain period of time, the system will notify medical institutions or family members, enabling early intervention and appropriate care.

[1358] "Example 4"

[1359] The emotion engine can also provide well-being advice tailored to the user's emotions. For example, if the system recognizes that the user is feeling stressed, it can suggest relaxation or meditation techniques. Or, if the system recognizes that the user is feeling joy, it can suggest activities and habits to maintain that joy.

[1360] The processing flow of each embodiment will be described below.

[1361] "Example 1"

[1362] Step 1: The system receives the user's voice as input.

[1363] Step 2: Extract features from the audio (e.g., pitch, volume, speaking rate, etc.).

[1364] Step 3: The emotion engine estimates emotions based on the extracted features.

[1365] Step 4: The emotion engine classifies emotions using a machine learning model that is trained from a large amount of audio data and its corresponding emotion labels.

[1366] "Example 2"

[1367] Step 1: The emotion engine adjusts the content of the conversation based on the emotions it recognizes.

[1368] Step 2: For example, if the system recognizes that the user is angry, it will switch to a calmer topic or make remarks to calm the user.

[1369] Step 3: Conversely, if the system recognizes that the user is happy, it will make comments that share that happiness or switch to a topic that will amplify that happiness.

[1370] "Example 3"

[1371] Step 1: The emotion engine tracks the user's long-term emotional changes.

[1372] Step 2: If the user continues to show sad emotions over a period of time, the system will notify medical institutions and family members.

[1373] Step 3: This allows for early intervention and appropriate care.

[1374] "Example 4"

[1375] Step 1: The emotion engine provides health improvement advice adapted to the user's emotions.

[1376] Step 2: For example, if the system recognizes that the user is feeling stressed, it will suggest relaxation or meditation techniques.

[1377] Step 3: If the system also recognizes that the user is experiencing joy, it will suggest activities and habits to maintain that joy.

[1378] Example 1

[1379] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1380] In modern society, it is important to ensure users' mental stability and brain activity. However, with conventional systems, it was difficult to achieve this through natural conversation with the user. It was also difficult to accurately recognize the user's emotions and generate appropriate responses. Furthermore, there is a need to detect abnormalities such as dementia early and take appropriate action. In order to solve these problems, it is necessary to effectively utilize voice recognition technology and emotion recognition technology.

[1381] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1382] In this invention, the server includes: a means for conducting a conversation via voice input, contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia trends; a means for notifying a hospital, facility, family, etc. of the detected abnormality; a means for converting voice data into text data; a means for recognizing the user's emotions from the voice data; a means for generating an appropriate response based on the emotional information and text data; a means for converting the generated response into voice and conveying it to the user; a means for providing health improvement advice through collaboration with IoT and healthcare systems; and a means for supporting the realization of a monitoring society through collaboration with the local community. This enables mental stability and brain activation through natural conversation with the user, and furthermore, it can accurately recognize the user's emotions and generate appropriate responses. It can also detect abnormalities such as dementia trends early and take appropriate measures.

[1383] "Voice input" is the process of capturing user-generated speech into a system through a device such as a microphone.

[1384] "Mental stability" refers to maintaining or improving the user's psychological stability.

[1385] "Brain activation" refers to stimulating the user's brain functions and improving their cognitive and thinking abilities.

[1386] "Conversational state" refers to the content and quality of the interaction between the user and the system.

[1387] "Dementia tendency" refers to a state in which a user shows early symptoms or signs of dementia.

[1388] "Abnormal" refers to a user's behavior or health condition that deviates from normal.

[1389] "Notification" is the process of informing relevant parties such as hospitals, facilities, and family members of detected abnormalities.

[1390] "IoT" refers to a network of devices and systems connected via the internet.

[1391] "Healthcare system" refers to a system for managing and improving the health status of users.

[1392] "Health Improvement Advice" refers to specific suggestions or instructions to improve a user's health.

[1393] "Community" refers to the geographic or social community to which a User belongs.

[1394] A "watchful society" refers to a system in which the local community cooperates to watch over the safety and health of users.

[1395] "Voice data" refers to data that is a digital recording of a user's voice.

[1396] "Text data" refers to data obtained by converting voice data into character information.

[1397] "Emotion recognition" is the process of inferring emotions from a user's voice and behavior.

[1398] "Response generation" is the process of creating an appropriate reply to a user's input.

[1399] "Speech synthesis" refers to the technology of converting text data into speech.

[1400] This invention is a system that combines speech recognition technology and emotion recognition technology, and aims to stabilize the mind and activate the brain through natural conversation with the user. Specific embodiments of this system will be described below.

[1401] Hardware and software used

[1402] This system uses the following hardware and software:

[1403] Hardware: microphone, speaker, device (PC, smartphone, tablet, etc.)

[1404] Software: Speech recognition APIs (e.g., Google Cloud Speech-to-Text), emotion recognition engines (e.g., TensorFlow, PyTorch), generative AI models (e.g., OpenAI GPT-3), and speech synthesis APIs (e.g., Google Cloud Text-to-Speech)

[1405] System processing flow

[1406] 1. Receiving voice input

[1407] The user speaks to the system, and the device captures this speech with a microphone. For example, if the user says, "I'm very tired today," the speech is input into the device's microphone.

[1408] 2. Converting voice data to text

[1409] The device sends the captured voice data to a voice recognition API. The server calls this API and converts the voice data into text data. For example, the voice saying "I'm very tired today" is converted into text "I'm very tired today."

[1410] 3. Emotion recognition

[1411] The server sends the text data and voice data to an emotion recognition engine. The emotion recognition engine extracts features such as pitch, volume, and speaking rate from the voice data and uses these features to estimate the user's emotion. For example, the emotion recognition engine recognizes the emotion "fatigue."

[1412] 4. Response Generation

[1413] The server sends a prompt to the generative AI model based on the emotional information and text data from the emotion recognition engine. The generative AI model generates an appropriate response. For example, if the prompt is "The user says, 'I'm very tired today,' and the emotion is 'fatigue.' Please generate an appropriate response," the generative AI model will generate the response, "Today was tough. Take a break and relax."

[1414] 5. Speech synthesis and response delivery

[1415] The server sends the generated response text to the device. The device then converts the text into speech using a speech synthesis API. For example, the text might say, "Today was tough. Let's take a break and relax." The device then transmits this speech to the user through the speaker.

[1416] Specific examples

[1417] When a user says, "I'm very tired today," the system processes as follows:

[1418] 1. Receiving voice input:

[1419] A user says, "I'm very tired today."

[1420] The device's microphone captures the audio.

[1421] 2. Audio to text conversion:

[1422] The device sends the captured voice data to the voice recognition API.

[1423] The server converts the speech data into text such as "I'm very tired today."

[1424] 3. Emotion recognition:

[1425] The server sends the text and voice data to the emotion recognition engine.

[1426] The emotion recognition engine extracts features such as pitch, volume, and speaking rate from the audio data.

[1427] The emotion recognition engine recognizes the emotion "fatigue."

[1428] 4. Response Generation:

[1429] The server sends prompts to the generative AI model based on emotional information and text data.

[1430] The generative AI model generates a response like, "You've had a hard day today. Take a break and relax."

[1431] 5. Speech synthesis and response delivery:

[1432] The server sends the generated response text to the terminal.

[1433] The device uses a speech synthesis API to convert the text into speech.

[1434] The device transmits the audio to the user through a speaker.

[1435] In this way, the system can have natural conversations with the user, contributing to the user's mental stability and brain activity.

[1436] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1437] Step 1:

[1438] Receiving voice input

[1439] The user speaks to the system. The device captures this voice with a microphone. For example, when a user says, "I'm very tired today," the voice is input into the device's microphone. The input is the user's voice, and the output is the captured voice data.

[1440] Step 2:

[1441] Converting audio data to text

[1442] The device sends the captured voice data to a voice recognition API. The server calls this API and converts the voice data into text data. For example, the voice saying "I'm very tired today" is converted into text "I'm very tired today." The input is voice data, and the output is text data.

[1443] Step 3:

[1444] emotion recognition

[1445] The server sends text data and voice data to an emotion recognition engine. The emotion recognition engine extracts features such as pitch, volume, and speaking rate from the voice data and uses these features to estimate the user's emotion. For example, the emotion recognition engine recognizes the emotion "fatigue." The input is voice data and text data, and the output is emotional information.

[1446] Step 4:

[1447] Response Generation

[1448] The server sends a prompt to the generative AI model based on the emotional information and text data from the emotion recognition engine. The generative AI model generates an appropriate response. For example, if the prompt is "The user says, 'I'm very tired today,' and the emotion is 'fatigue.' Please generate an appropriate response," the generative AI model generates the response, "Today was tough. Take a break and relax." The input is emotional information and text data, and the output is the response text.

[1449] Step 5:

[1450] Speech synthesis and response delivery

[1451] The server sends the generated response text to the device. The device converts the text into speech using a speech synthesis API. For example, the text "Today has been tough. Let's take a break and relax." is converted into speech. The device transmits this speech to the user through the speaker. The input is the response text, and the output is audio data.

[1452] (Application example 1)

[1453] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1454] While conventional voice recognition systems can contribute to the user's mental stability and brain activity, they lack the ability to recognize the user's emotions and generate appropriate responses accordingly. In addition, certain services, such as food delivery, require support for the ordering process that takes the user's emotions into account. This requires a system that allows users to use the service smoothly and without stress.

[1455] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation, a means for analyzing the state of the conversation and detecting abnormalities such as dementia tendencies, a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, a means for providing advice on improving health through collaboration with IoT and healthcare systems, a means for supporting the realization of a monitoring society through collaboration with the local community, a means for recognizing emotions from the user's voice and generating a response according to the emotion, and a means for supporting the user's ordering process using voice recognition and emotion analysis. This makes it possible to generate a response that takes the user's emotions into consideration and support the ordering process.

[1456] "Voice input" is a technology in which the user's voice is captured as a digital signal and the system analyzes the content.

[1457] A "conversation" is the process by which a user and a system exchange information through speech.

[1458] "Mental stability" refers to maintaining or improving the user's psychological stability.

[1459] "Brain activation" refers to stimulating the user's brain functions and improving their cognitive and thinking abilities.

[1460] "Conversational state" refers to the overall situation, including the content, tone, and emotion of the conversation between the user and the system.

[1461] "Abnormal" refers to a user's behavior, emotions, or health state that deviates from normal.

[1462] "Hospitals, facilities, families, etc." refers to external organizations and individuals who manage the health and safety of users.

[1463] "Notification" is the act of communicating information to external organizations or individuals when a system detects an abnormality.

[1464] "IoT" is a technology that allows various devices to be interconnected and exchange data via the Internet.

[1465] A "healthcare system" is a system for managing and improving a user's health status.

[1466] "Health Improvement Advice" refers to specific instructions or suggestions for improving a user's health.

[1467] "Local community" refers to the local community in which the user lives.

[1468] A "watchful society" refers to a system in which the local community cooperates to watch over the safety and health of users.

[1469] "Emotion recognition" is a technology that analyzes and identifies a user's emotions from their voice and behavior.

[1470] "Generating a response" is the process by which the system creates an appropriate reply to the user's input.

[1471] "Ordering process" refers to the series of steps a user takes to order a product or service.

[1472] "Support" refers to the system providing assistance to help the user with their actions and decisions.

[1473] The system for implementing this invention integrates the functions of voice input, voice recognition, emotion recognition, response generation, voice synthesis, and order process support. Specific embodiments are described below.

[1474] Hardware and Software Configuration

[1475] This system uses the following hardware and software:

[1476] Hardware: Microphone, Smartphone

[1477] Software: Python, speech recognition library, speech synthesis module, emotion recognition module, order processing module

[1478] Processing flow

[1479] 1. Acquiring voice input: The user speaks through a microphone. The smartphone captures this voice as a digital signal and analyzes it using a speech recognition library.

[1480] 2. Speech recognition: The captured voice is converted into text using Google's speech recognition API, and the user's speech is captured as text data.

[1481] 3. Emotion Recognition: Text data is input into the emotion recognition module to analyze the user's emotions. Emotion recognition is performed based on voice features such as pitch, volume, and speaking rate.

[1482] 4. Response generation: Generate an appropriate response based on the emotion recognition results. For example, if the user is feeling stressed, generate a response to help them relax.

[1483] 5. Speech synthesis: The generated response is converted into speech using a speech synthesis module and conveyed to the user.

[1484] 6. Ordering process support: When a user places a food delivery order, the system uses speech recognition and sentiment analysis to support the ordering process. For example, if a user says, "I'd like to order a pizza," the system converts the speech into text. If the sentiment engine determines that the user is stressed, the system responds, "Please relax. We'll take your order."

[1485] Specific examples

[1486] For example, if a user says, "I'd like to order a pizza," the system converts that speech into text, and if the emotion engine determines that the user is stressed, it responds, "Please relax. We'll take your order." In this way, it is possible to generate responses that take the user's emotions into account and support the ordering process.

[1487] Prompt Sentence Examples

[1488] If a user says, "I'd like to order a pizza," the system converts that speech to text, and if its emotion engine determines that the user is stressed, it responds, "Relax. We'll take your order."

[1489] In this way, a food delivery application can provide a voice interface that takes into account the user's emotions.

[1490] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1491] Step 1:

[1492] The user inputs voice through a microphone. The device captures this voice as a digital signal and analyzes it using a speech recognition library. The input is the user's voice, and the output is a digital signal. This digital signal is used for speech recognition in the next step.

[1493] Step 2:

[1494] The device converts the acquired digital signal into text using Google's speech recognition API. The input is a digital signal, and the output is text data. This text data represents the content of the user's speech and is used for emotion recognition in the next step.

[1495] Step 3:

[1496] The server inputs text data into an emotion recognition module to analyze the user's emotions. The input is text data, and the output is an emotion label (e.g., stress, joy, etc.). Emotion recognition is performed based on voice features such as pitch, volume, and speaking rate.

[1497] Step 4:

[1498] The server generates an appropriate response based on the emotion recognition results. The input is an emotion label, and the output is a response text. For example, if the user is feeling stressed, the server generates a response such as "Please relax. We will take your order."

[1499] Step 5:

[1500] The server converts the generated response text into speech using a speech synthesis module and transmits it to the user through the terminal. The input is the response text and the output is audio data. The terminal plays this audio data and transmits the response to the user.

[1501] Step 6:

[1502] When a user places a food delivery order, the server uses speech recognition and sentiment analysis to support the ordering process. The input is the user's order and an emotional label, and the output is a response confirming the order. For example, if a user says, "I'd like to order a pizza," the system converts the speech into text, and if the sentiment engine determines that the user is stressed, it responds, "Please relax. We'll take your order."

[1503] Example 2

[1504] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1505] In an aging society, early detection and prevention of dementia are becoming important issues. It is also necessary to care for the user's mental stability and emotions. Furthermore, if an abnormality is detected, a prompt and appropriate response is required. To solve these issues, a system is needed that can analyze the user's speech content and emotions in detail and respond appropriately.

[1506] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1507] In this invention, the server includes: means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; means for analyzing the state of the conversation and detecting abnormalities such as dementia trends from the user's speech content, speech patterns, and speech frequency using natural language processing technology and machine learning technology; means for notifying a hospital, facility, family, etc. of the detected abnormality; and means for recognizing the user's emotions using an emotion engine and adjusting the content of the conversation based on the recognized emotions, thereby enabling monitoring of the user's health condition and emotional care.

[1508] "Voice input" is a technology that collects user speech as voice data.

[1509] "Mental stability" refers to maintaining the user's psychological stability.

[1510] "Brain activation" refers to improving the user's brain function.

[1511] "Conversation state" refers to the content of a user's speech, speech patterns, frequency of speech, etc.

[1512] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[1513] "Machine learning technology" is a technology that allows computers to learn from data and make predictions and classifications.

[1514] "Speech content" refers to the specific words or sentences spoken by the user.

[1515] "Speech patterns" refer to the tendencies and characteristics of a user's speech.

[1516] "Utterance frequency" refers to how often a user says a particular word or phrase.

[1517] "Dementia proneness" refers to the likelihood that a user is exhibiting early symptoms of dementia.

[1518] "Abnormal" refers to a state or behavior that is different from the normal.

[1519] "Notification" refers to conveying information to relevant parties when an abnormality is detected.

[1520] An "emotion engine" is a technology that recognizes the user's emotions and responds appropriately based on them.

[1521] "Emotion" refers to the user's psychological state.

[1522] "Adjusting the conversation" refers to changing the topic and tone of the conversation depending on the user's emotions.

[1523] "IoT" is a technology that allows various devices to connect via the Internet.

[1524] A "healthcare system" is a system for managing and improving a user's health status.

[1525] "Health Improvement Advice" refers to specific suggestions and guidance to improve the user's health.

[1526] "Local community" refers to the local community in which the user lives.

[1527] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[1528] The present invention is a system that analyzes the content of a user's speech and emotions in detail and responds appropriately. A specific embodiment of this system is described below.

[1529] Hardware and software used

[1530] Hardware: Servers, devices (PCs, smartphones, tablets, etc.)

[1531] Software: Natural language processing libraries (e.g., NLTK, spaCy), machine learning frameworks (e.g., TensorFlow, PyTorch), sentiment analysis engines (e.g., IBM Watson, Microsoft Azure Emotion API), speech recognition software (e.g., Google Speech-to-Text API)

[1532] Data processing and calculation

[1533] 1. Collecting user utterance data

[1534] The device collects the user's speech as voice data. Specifically, it uses the microphone of a smartphone or tablet to record the user's speech.

[1535] Example: A user says, "The weather is nice today."

[1536] 2. Converting voice data to text

[1537] The device sends the collected voice data to the server.

[1538] The server uses voice recognition software to convert the voice data into text data.

[1539] Example: Speech data such as "The weather is nice today" is converted into text data such as "The weather is nice today."

[1540] 3. Analysis of speech content

[1541] The server uses a natural language processing library to analyze the text data, specifically tagging words by part of speech and analyzing sentence structure.

[1542] Example: The word "weather" is tagged as a noun and the word "nice" is tagged as an adjective.

[1543] 4. Anomaly Detection

[1544] The server uses a machine learning model to detect abnormalities based on the analysis results. Specifically, the server inputs the user's speech patterns and frequency into the model to determine whether they are prone to dementia.

[1545] Example: If a user frequently says, "I've been forgetting things a lot lately," the system will determine that they are prone to dementia.

[1546] 5. Notification of abnormalities

[1547] The server will notify hospitals, facilities, families, etc. of any abnormalities it detects. Possible notification methods include email, SMS, or push notifications via a dedicated app.

[1548] Example: Send a notification to a family member saying, "The user has been frequently mentioning forgetfulness recently. This may indicate a tendency toward dementia."

[1549] 6. Emotional Recognition

[1550] The server uses a sentiment analysis engine to recognize the user's emotions. Specifically, it inputs text data into the sentiment analysis engine and identifies the type of emotion (joy, anger, sadness, etc.).

[1551] Example: If a user says, "I'm very happy today," the system will recognize it as "joy."

[1552] 7. Adjust the conversation

[1553] The server adjusts the content of the conversation based on the emotions it recognizes, specifically by using a generative AI model to generate appropriate utterances.

[1554] Example: If a user says, "I'm so happy today," the system responds, "That's great! Did something good happen?"

[1555] Prompt Sentence Examples

[1556] "The user has frequently mentioned forgetfulness in recent utterances. Please analyze whether they have a tendency toward dementia."

[1557] "User recognized as angry. Generate appropriate utterance to transition to a calmer topic."

[1558] In this way, the present invention realizes monitoring of the user's health condition and emotional care by analyzing the user's speech content and emotions in detail and taking appropriate measures.

[1559] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1560] Step 1:

[1561] Collecting user speech data

[1562] The device collects the user's speech as voice data. Specifically, it uses the microphone of a smartphone or tablet to record the user's speech.

[1563] Input: User utterance (e.g., "The weather is nice today.")

[1564] Output: Audio data (e.g., a recorded audio file)

[1565] What it does: The device's microphone captures the user's speech and saves it as an audio file.

[1566] Step 2:

[1567] Converting audio data to text

[1568] The device sends the collected voice data to the server.

[1569] The server uses voice recognition software to convert the voice data into text data.

[1570] Input: Audio data (e.g., a recorded audio file)

[1571] Output: Text data (e.g. "The weather is nice today")

[1572] What happens: The server calls speech recognition software to convert the audio file into text.

[1573] Step 3:

[1574] Analysis of speech content

[1575] The server uses a natural language processing library to analyze the text data, specifically tagging words by part of speech and analyzing sentence structure.

[1576] Input: Text data (e.g., "The weather is nice today")

[1577] Output: Analysis result (e.g. "weather" is tagged as a noun and "good" as an adjective)

[1578] Specific operation: The server uses a natural language processing library to tag the text data by part of speech and analyze the sentence structure.

[1579] Step 4:

[1580] Anomaly detection

[1581] The server uses a machine learning model to detect abnormalities based on the analysis results. Specifically, the server inputs the user's speech patterns and frequency into the model to determine whether they are prone to dementia.

[1582] Input: Analysis results (e.g., part-of-speech tagged text data)

[1583] Output: Anomaly detection result (e.g., tendency toward dementia)

[1584] Specific operation: The server inputs the analysis results into the machine learning model and determines whether there are any abnormalities.

[1585] Step 5:

[1586] Abnormality notification

[1587] The server will notify hospitals, facilities, families, etc. of any abnormalities it detects. Possible notification methods include email, SMS, or push notifications via a dedicated app.

[1588] Input: Anomaly detection result (e.g., tendency toward dementia)

[1589] Output: Notification message (e.g. "The user has frequently mentioned forgetfulness recently. He may be prone to dementia.")

[1590] Specific operation: The server generates a notification message based on the anomaly detection result and sends it to the relevant parties.

[1591] Step 6:

[1592] Emotion recognition

[1593] The server uses a sentiment analysis engine to recognize the user's emotions. Specifically, it inputs text data into the sentiment analysis engine and identifies the type of emotion (joy, anger, sadness, etc.).

[1594] Input: Text data (e.g., "I'm very happy today")

[1595] Output: Emotion recognition result (e.g., joy)

[1596] Specific operation: The server inputs text data into a sentiment analysis engine to identify the type of sentiment.

[1597] Step 7:

[1598] Adjusting the conversation

[1599] The server adjusts the content of the conversation based on the emotions it recognizes, specifically by using a generative AI model to generate appropriate utterances.

[1600] Input: Emotion recognition result (e.g., joy)

[1601] Output: Tailored conversation (e.g., "That's great! Did something good happen?")

[1602] Specific operation: The server inputs the emotion recognition results into the generated AI model, generates appropriate utterances, and sends them to the device.

[1603] (Application example 2)

[1604] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1605] Dementia and mental health issues among the elderly are serious problems in modern society. It is particularly difficult to monitor daily life and detect abnormalities early for elderly people who live alone or away from their families. There is also a need to detect changes in users' emotions and abnormal speech patterns in real time and respond appropriately. To solve these problems, a system is needed that analyzes the content of users' conversations and emotions, detects abnormalities, and sends appropriate notifications.

[1606] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1607] In this invention, the server includes means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation, means for analyzing the state of the conversation and detecting abnormalities such as dementia trends, means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, means for providing advice on improving health through collaboration with IoT and healthcare systems, means for supporting the realization of a watchful society through collaboration with the local community, means for analyzing the user's emotions in real time and detecting abnormal speech patterns and changes in emotions, means for notifying pre-set contacts of the detected abnormalities, and means for adjusting the content of the conversation according to the user's emotions.This makes it possible to maintain the user's mental health, detect abnormalities early, and take appropriate measures.

[1608] "Voice input" is a technology that captures a user's speech as digital data.

[1609] "Mental stability" refers to maintaining the user's psychological stability.

[1610] "Brain activation" refers to stimulating the user's brain functions and improving cognitive function.

[1611] "Conversational state" refers to the overall state of a user's speech, including their speech content, speech patterns, and speech frequency.

[1612] "Dementia tendency" refers to a state in which a user may develop dementia.

[1613] "Abnormal" refers to a user's speech or behavior that deviates from normal.

[1614] "Notification" refers to sending information to pre-defined contacts when an abnormality is detected.

[1615] "IoT" refers to technology that connects various devices via the Internet.

[1616] "Healthcare system" refers to a system for managing and improving the health status of users.

[1617] "Health Improvement Advice" refers to specific instructions or suggestions for improving a user's health.

[1618] "Local community" refers to the local community in which the user lives.

[1619] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[1620] "Real-time emotion analysis" refers to technology that instantly analyzes emotions from the content of a user's speech.

[1621] "Speech patterns" refer to characteristics such as the rhythm and frequency of a user's speech, and word choice.

[1622] "Emotional changes" refers to the user's emotional state changing over time.

[1623] "Pre-configured Contacts" refers to contacts configured to receive notifications when an anomaly is detected.

[1624] "Conversational content adjustment" refers to changing the topic and tone of a conversation depending on the user's emotional state.

[1625] The following system configuration is used as an embodiment of the present invention.

[1626] System Configuration

[1627] The system consists of a voice input device, a server, a notification device, and a linked device. The voice input device captures the user's speech and converts it into digital data. The server analyzes the captured voice data and analyzes the user's mental state and emotions in real time. The notification device sends a notification to pre-set contacts if an abnormality is detected. The linked device connects with IoT and healthcare systems to provide the user with advice on improving their health.

[1628] Hardware and software used

[1629] Hardware: smartphone, microphone, notification device (e.g. smartwatch)

[1630] Software: Google Cloud Natural Language API, Communication Services API, Python

[1631] Data processing and calculation

[1632] 1. Acquiring voice data: Using a voice input device (smartphone or microphone), the user's speech is acquired as digital data.

[1633] 2. Natural language processing: The server uses the Google Cloud Natural Language API to analyze the acquired voice data and calculate the user's emotion score.

[1634] 3. Anomaly detection: The server determines an anomaly if the emotion score exceeds a certain threshold.

[1635] 4. Send notifications: If an anomaly is detected, a notification is sent to pre-defined contacts using the communication service API.

[1636] 5. Conversation content adjustment: The server adjusts the conversation content according to the user's emotional state.

[1637] Specific examples

[1638] For example, if a user says, "I've been feeling unmotivated lately," the voice input device captures this utterance and sends it to the server. The server uses the Google Cloud Natural Language API to calculate the emotion score for this utterance, resulting in a score of -0.7. Because this score is deemed abnormal, the communication service API is used to send a notification to pre-defined contacts stating, "An abnormal speech pattern has been detected."

[1639] Prompt Sentence Examples

[1640] If the user says, "I just don't feel like doing anything these days," use the following prompt:

[1641] user_input = "I've been feeling unmotivated lately."

[1642] score, magnitude = analyze_text(user_input)

[1643] if score < -0.5:

[1644] send_alert("An unusual speech pattern has been detected.")

[1645] In this way, it is possible to realize a system that analyzes the content of user utterances in real time and takes appropriate action if an abnormality is detected.

[1646] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1647] Step 1:

[1648] The user speaks.

[1649] The user speaks using a smartphone or microphone, and this speech becomes input data for the system.

[1650] Step 2:

[1651] The terminal acquires the audio data.

[1652] The device (smartphone or microphone) captures the user's speech as digital voice data, which serves as input for the next processing step.

[1653] Step 3:

[1654] The terminal transmits the voice data to the server.

[1655] The terminal transmits the acquired voice data to the server, which then uses the data as input for processing.

[1656] Step 4:

[1657] The server converts the voice data into text data.

[1658] The server uses speech recognition technology to convert the voice data into text data, which serves as input for the next processing step.

[1659] Step 5:

[1660] The server parses the text data.

[1661] The server uses the Google Cloud Natural Language API to analyze the text data for sentiment scores and sentiment intensity, which serve as input for the next processing step.

[1662] Step 6:

[1663] The server detects an abnormality.

[1664] The server determines an abnormality if the emotion score of the analysis result exceeds a certain threshold. For example, if the emotion score is less than -0.5, it determines an abnormality. The result of this anomaly detection becomes the input for the next processing step.

[1665] Step 7:

[1666] The server sends a notification.

[1667] If an anomaly is detected, the server uses the communication service API to send a notification to pre-defined contacts, and the content of this notification becomes the input for the next processing step.

[1668] Step 8:

[1669] A notification device receives the notification.

[1670] The notification device (smartphone or smartwatch) receives the notification sent from the server, and this notification is output to notify the user or other relevant parties of the abnormality.

[1671] Step 9:

[1672] The server moderates the conversation.

[1673] The server adjusts the conversation content depending on the user's emotional state. For example, if the user is angry, it switches to a calmer topic. This adjusted conversation content becomes the input for the next processing step.

[1674] Step 10:

[1675] The terminal provides the adjusted conversation content to the user.

[1676] The terminal provides the user with the adjusted conversation content sent from the server, and this provided conversation content becomes the output for the user.

[1677] The above processing steps realize a system that analyzes the content of a user's speech in real time and takes appropriate action if an abnormality is detected.

[1678] Example 3

[1679] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1680] Modern society demands systems that comprehensively manage users' mental and physical health and provide appropriate advice. However, existing systems struggle to effectively collect and analyze users' lifestyle and health data and provide appropriate advice. Furthermore, collaboration with local communities and the collection and analysis of users' emotional data are insufficient, making it difficult to prevent users from becoming isolated or their mental health from deteriorating. To address these issues, systems with more advanced data analysis technology and collaboration capabilities are needed.

[1681] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1682] In this invention, the server includes a means for collecting a user's lifestyle and health data, a means for preprocessing the collected data, and a means for analyzing the preprocessed data using a generative AI model. This enables highly accurate analysis of the user's health status and lifestyle patterns and the provision of appropriate advice for improving their physical condition. Furthermore, by including a means for providing an interface for collecting the user's emotional data and a means for collecting the user's feedback, the server can comprehensively manage the user's mental health status and realize early intervention and appropriate care. Furthermore, by including a means for supporting the realization of a monitoring society through collaboration with the local community, it becomes possible to prevent user isolation and establish a monitoring system throughout the community.

[1683] "Voice input" is a means by which a user provides information to a system through speech.

[1684] "Mental stability" is a state that maintains the user's mental balance and reduces stress and anxiety.

[1685] "Brain activation" is a state that improves the user's brain function and enhances cognitive ability and memory.

[1686] "Conversational state" refers to the content and quality of the voice interaction a user has with a system.

[1687] "Dementia tendency" refers to a state in which the user may be showing early symptoms of dementia.

[1688] An "abnormality" is a state in which a user's health condition or behavior is deemed to be different from normal or problematic.

[1689] "Medical institution" refers to a facility that manages the health of users, such as a hospital or clinic.

[1690] A "facility" is a specialized institution that supports the lives and health of users.

[1691] "Family" refers to the User's immediate family and close relatives.

[1692] "IoT" is a network of physical devices and sensors connected via the internet.

[1693] A "healthcare system" is an information system that supports users in managing their health.

[1694] "Health Improvement Advice" refers to specific suggestions and guidance for improving the user's health.

[1695] "Local community" refers to the residents and stakeholders of the area where the user lives.

[1696] A "watchful society" is a system in which the entire community watches over the safety and health of users.

[1697] "Lifestyle data" is information about the user's daily life.

[1698] "Health Data" means information relating to a User's health status.

[1699] "Preprocessing" is the process of preparing collected data in a form that is easy to analyze.

[1700] A "generative AI model" is a model that uses machine learning algorithms to analyze data and make predictions and classifications.

[1701] "Analysis" is the process of evaluating the user's condition based on collected data.

[1702] An "interface" is the means by which a user interacts with a system.

[1703] "Feedback" refers to the evaluations and opinions that users provide to the system.

[1704] MODE FOR CARRYING OUT THE INVENTION

[1705] The present invention is a system for collecting lifestyle and health data of a user, analyzing the data, and providing appropriate advice for improving physical condition. A specific embodiment of this system will be described below.

[1706] server

[1707] The server collects users' lifestyle and health data, including heart rate, step count, and sleep patterns from IoT devices such as smartwatches and fitness trackers. It also obtains health checkup results and medical history from electronic medical record systems. These data are managed as data frames using the Python Pandas library.

[1708] Next, the server preprocesses the collected data, specifically by imputing missing values, removing outliers, normalizing the data, etc. For example, it uses the Pandas fillna method to impute missing values ​​with the mean value.

[1709] The preprocessed data is then analyzed using generative AI models. Machine learning models are trained using TensorFlow and PyTorch to analyze the user's health and lifestyle patterns. For example, a model is used to predict stress levels using the user's heart rate data as input.

[1710] Based on the analysis results, the server generates appropriate advice to improve health. For example, if the user is feeling stressed, it will suggest relaxation or meditation techniques. If the user is feeling joy, it will suggest activities and habits to maintain that joy.

[1711] The generated advice is sent to the user's device using a messaging service.

[1712] Terminal

[1713] The device receives the advice to improve one's physical condition sent from the server. This corresponds to the user's smartphone or tablet. The device then displays the received advice to the user. Specifically, the advice is displayed on the app's notification screen or on a dedicated dashboard. For example, the app may display a message saying, "Try deep breathing techniques for relaxation."

[1714] The device also provides an interface for collecting user emotional data. For example, the user can record their emotions in a diary app, and the data is sent to the server. Specifically, the app displays a prompt such as "Please choose your mood today."

[1715] user

[1716] Users receive advice on how to improve their physical condition through their device. For example, they can check their smartphone notifications and practice relaxation techniques. Users can also record their emotions and health status in a diary app or similar and send that data to a server. For example, they can record in the app that they felt stressed today.

[1717] If the user continues to display sad emotions over a period of time, the system will notify medical institutions and family members, allowing for early intervention and appropriate care.

[1718] Specific examples

[1719] Examples of prompts to input to a generative AI model include:

[1720] 1. "Analyze user's heart rate data and assess their stress level."

[1721] 2. "Analyze users' sleep patterns and provide advice for improvement."

[1722] 3. "Please suggest appropriate relaxation methods based on the user's recent emotions."

[1723] In this way, a system is realized that comprehensively manages the lifestyle habit data and health data of the user and provides appropriate advice for improving physical condition. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1724] Step 1:

[1725] The server collects users' lifestyle and health data. As input, it receives data such as heart rate, step count, and sleep patterns from smartwatches and fitness trackers, as well as health checkup results and medical history from electronic medical record systems. It stores this data in a database. Specifically, it retrieves data from devices and systems through APIs and converts it into a data frame using Python's Pandas library.

[1726] Step 2:

[1727] The server preprocesses the collected data. As input, it receives the raw data collected in step 1. It imputes missing values ​​in the data, removes outliers, and normalizes the data. Specifically, it uses Pandas' fillna method to impute missing values ​​with the mean value, and detects and removes outliers. It also uses Scikit-learn's StandardScaler to scale the data. As output, it obtains preprocessed, clean data.

[1728] Step 3:

[1729] The server analyzes the preprocessed data using a generative AI model. It receives the preprocessed data from step 2 as input. It uses a generative AI model (e.g., a machine learning model using TensorFlow or PyTorch) to analyze the user's health condition and lifestyle patterns. Specifically, it runs a model that predicts stress levels using heart rate data as input. It obtains the analysis results (e.g., stress level assessment) as output.

[1730] Step 4:

[1731] The server generates appropriate advice for improving physical condition based on the analysis results. It receives the analysis results obtained in step 3 as input. It uses the generative AI model to generate advice according to the user's condition. Specifically, if the stress level is high, it suggests relaxation or meditation techniques, and if the user is feeling joy, it suggests activities and habits to maintain that joy. The generated advice is obtained as output.

[1732] Step 5:

[1733] The server notifies the user's device of the generated advice. As input, it receives the advice generated in step 4. It uses a messaging service to send a push notification to the user's smartphone or tablet. Specifically, it creates and sends a notification message using the messaging service's API. As output, the notification is displayed on the user's device.

[1734] Step 6:

[1735] The device receives health improvement advice sent from the server and displays it to the user. As input, it receives a notification message from the server. Specific operations include displaying the advice using the smartphone's notification function and providing detailed information within the app. As output, the user can confirm the advice.

[1736] Step 7:

[1737] The device provides an interface for collecting user emotion data. As input, it receives input about the user's emotions. As specific operations, it provides a diary application and emotion recording function to allow the user to record their emotions. As output, it sends the recorded emotion data to a server.

[1738] Step 8:

[1739] The user receives advice on how to improve their health through their device and puts it into practice. As input, they receive a notification of the advice from their device. As a specific action, they check the notification and practice the suggested relaxation techniques or meditation methods. As output, the user's health condition improves.

[1740] Step 9:

[1741] The user records their emotions and health status in a diary app or similar and sends the data to a server. The input is information about daily emotions and health status. A specific action is to record "I felt stressed today" in the app. The output is the recorded data sent to the server.

[1742] Step 10:

[1743] The server collects user feedback and uses it to improve the system. As input, it receives feedback from users. Specific operations include providing feedback forms and survey functions so that users can report on the effectiveness of advice and areas for improvement. As output, it analyzes the collected feedback data and reflects it in improving the system.

[1744] (Application example 3)

[1745] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1746] In modern society, maintaining users' mental and physical health is an important issue. However, in environments where users tend to be isolated, it is difficult for them to receive appropriate support and advice. It is also difficult to detect changes in users' emotions and health status early and take appropriate measures. Furthermore, there is a need to strengthen cooperation with local communities and establish a monitoring system.

[1747] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia trends; a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; a means for providing advice on improving health through collaboration with IoT and healthcare systems; a means for supporting the realization of a monitoring society through collaboration with the local community; a means for collecting and analyzing the user's lifestyle data and health data; a means for tracking changes in the user's emotions using an emotion engine and providing necessary support; a means for sharing the user's data with relevant people in the local community; and a means for notifying medical institutions of changes in the user's emotions. This makes it possible to comprehensively support the user's mental and physical health and strengthen collaboration with the local community.

[1748] "Voice input" is a technology that recognizes the words a user speaks as digital data and inputs it into a system.

[1749] "Mental stability" refers to stabilizing the user's mental state and reducing stress and anxiety.

[1750] "Brain activation" refers to enhancing the user's brain function and improving cognitive ability and memory.

[1751] "Conversation status" refers to information such as the content, tone, and frequency of a user's conversations.

[1752] "Dementia tendency" refers to a state in which a user may develop dementia.

[1753] "Abnormal" refers to a state in the user's health or emotional state that is different from normal and is deemed problematic.

[1754] "Hospitals, Facilities, and Families" refers to medical institutions, nursing homes, and family members of users who may receive information about their health and emotional state.

[1755] "IoT" refers to physical devices and sensors connected via the internet.

[1756] "Healthcare system" refers to a system for managing and improving the health status of users.

[1757] "Health Improvement Advice" refers to specific suggestions or instructions to improve a user's health.

[1758] "Local community" refers to the residents and stakeholders of the area where the user lives.

[1759] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[1760] "Lifestyle data" refers to data related to a user's daily life (e.g., diet, exercise, sleep, etc.).

[1761] "Health Data" refers to data relating to a User's health status (e.g., heart rate, blood pressure, body temperature, etc.).

[1762] "Emotion engine" refers to technology for analyzing a user's emotional state and tracking changes.

[1763] "Emotional changes" refers to the user's emotional state changing over time.

[1764] "Necessary support" refers to appropriate support and interventions provided based on the user's health and emotional state.

[1765] "Community stakeholders" refers to local residents and organizations involved in the lives and health of users.

[1766] "Medical institution" refers to a facility that receives information about a user's health condition and provides appropriate medical services.

[1767] The system for implementing this invention is designed to comprehensively support the mental and physical health of users and strengthen ties with the local community. Specific embodiments of the system are described below.

[1768] 1. System Configuration

[1769] The system consists of the following major components:

[1770] Server: The central location for data collection, analysis, advice generation, and notifications.

[1771] Device: The smartphone or IoT device (e.g., smartwatch, fitness tracker) used by the user.

[1772] User: An individual who uses the system.

[1773] 2. Data Collection

[1774] The device collects the user's lifestyle and health data, specifically heart rate, sleep data, exercise data, etc. from smartwatches and fitness trackers, and transmits this data to a server.

[1775] 3. Data analysis

[1776] The server analyzes the collected data using machine learning models and an emotion engine, which evaluates the user's health status and emotional changes and generates appropriate advice.

[1777] 4. Advice Generation

[1778] The server generates health improvement advice based on the analysis results. For example, if the user is feeling stressed, it will suggest relaxation or meditation techniques. If the user is feeling joyful, it will suggest activities and habits to maintain that joy.

[1779] 5. Data Sharing and Notification

[1780] The server shares user data with relevant parties in the local community, creating a community-wide monitoring system. It also notifies medical institutions of changes in the user's emotions, enabling early intervention and appropriate care.

[1781] 6. Hardware and Software Used

[1782] Hardware: smartphones, smartwatches, fitness trackers

[1783] Software: Python, APIs (e.g., healthcare API, community API, sentiment analysis API)

[1784] Specific examples

[1785] User A: Heart rate and sleep data are collected from the smartwatch, and if stress levels are determined to be high, relaxation advice is provided.

[1786] User B: The emotion engine detects prolonged sadness and notifies medical authorities.

[1787] Prompt Sentence Examples

[1788] Create a program that collects and analyzes users' lifestyle and health data to generate advice on how to improve their health. Also, create a monitoring system in collaboration with the local community and add features to track changes in emotions using an emotion engine.

[1789] The above is a specific embodiment for carrying out the present invention. This system makes it possible to comprehensively support the mental and physical health of users and strengthen ties with the local community.

[1790] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1791] Step 1:

[1792] The device collects the user's lifestyle and health data. Specifically, it obtains heart rate, sleep data, exercise data, etc. from a smartwatch or fitness tracker. This data is sent from the device to a server. The input is the user's biometric data, and the output is the data sent to the server.

[1793] Step 2:

[1794] The server saves the received data and stores it in a database. The input is the user's biometric data sent from the terminal, and the output is the data stored in the database. Specific operations include unifying the data format and storing it appropriately in the database.

[1795] Step 3:

[1796] The server analyzes the data stored in the database. It uses machine learning models and an emotion engine to evaluate changes in the user's health status and emotions. The input is the user's biometric data stored in the database, and the output is the analysis results. Specific operations include data preprocessing, feature extraction, input to the model, and obtaining prediction results.

[1797] Step 4:

[1798] The server generates advice to improve physical condition based on the analysis results. For example, if the user is feeling stressed, it will suggest relaxation or meditation techniques. The input is the analysis results, and the output is the generated advice. Specifically, it selects appropriate advice based on the analysis results and generates a message to provide to the user.

[1799] Step 5:

[1800] The server sends the generated advice to the terminal. The input is the generated advice, and the output is the advice sent to the terminal. Specific operations include converting the advice into a format that is easy for the user to understand and sending it to the terminal.

[1801] Step 6:

[1802] The terminal notifies the user of the received advice. The input is the advice sent from the server, and the output is the advice displayed to the user. Specific operations include displaying the advice to the user using the notification function.

[1803] Step 7:

[1804] The server shares the user's data with the community stakeholders. The input is the user's biometric data and the analysis results, and the output is the data sent to the community stakeholders. Specific operations include converting the data into an appropriate format and sending it to the stakeholders.

[1805] Step 8:

[1806] The server notifies the medical institution of changes in the user's emotions. The input is the analysis result by the emotion engine, and the output is a notification sent to the medical institution. Specific operations include detecting changes in emotions and notifying the medical institution at the appropriate time.

[1807] The above are the specific processing steps for carrying out the present invention, which make it possible to comprehensively support the mental and physical health of users and strengthen ties with the local community.

[1808] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1809] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1810] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1811] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1812] [Third embodiment]

[1813] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1814] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1815] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1816] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1817] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1818] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1819] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1820] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1821] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1822] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1823] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1824] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1825] "Example 1"

[1826] The system of the present invention uses speech recognition technology to converse with the user. This conversation contributes to the user's mental stability and brain activity. Specifically, it receives voice input from the user, converts it into text, and generates an appropriate response. This response is then conveyed to the user using speech synthesis technology.

[1827] "Example 2"

[1828] The system of the present invention also analyzes the state of conversation. This analysis uses natural language processing and machine learning technologies. Specifically, it detects abnormalities such as dementia trends from the content, speech patterns, and frequency of the user's speech. Detected abnormalities are notified to hospitals, facilities, family members, etc.

[1829] "Example 3"

[1830] Furthermore, the system of the present invention provides advice on improving health by linking with IoT and healthcare systems. Specifically, it collects and analyzes the user's lifestyle and health data. Based on the results, it generates appropriate advice and delivers it to the user.

[1831] "Example 4"

[1832] Furthermore, the system of the present invention supports the realization of a society where people are monitored in cooperation with the local community. Specifically, the system shares information about the user's living situation and health status with relevant people in the local community. This prevents users from becoming isolated and builds a monitoring system that involves the entire community.

[1833] The processing flow of each embodiment will be described below.

[1834] "Example 1"

[1835] Step 1: Receive speech input from the user. This speech input is part of a conversation between the user and the system.

[1836] Step 2: Convert the received voice input into text using voice recognition technology.

[1837] Step 3: Analyze the converted text and generate an appropriate response, using natural language processing and machine learning techniques.

[1838] Step 4: The generated response is converted into speech and conveyed to the user. This speech conversion is achieved using speech synthesis technology.

[1839] "Example 2"

[1840] Step 1: Analyze the state of the conversation with the user. This analysis uses natural language processing and machine learning technologies.

[1841] Step 2: Detect abnormalities such as dementia trends from the analysis results.

[1842] Step 3: Notify the hospital, facility, family, etc. of the detected abnormality. This notification is made via communication methods such as email, messaging apps, and telephone.

[1843] "Example 3"

[1844] Step 1: Collect user lifestyle and health data from IoT and healthcare systems.

[1845] Step 2: Analyze the collected data. This analysis uses statistical methods and machine learning techniques.

[1846] Step 3: Based on the analysis results, appropriate advice for improving health is generated.

[1847] Step 4: The generated advice is communicated to the user using voice output, text output, or a graphical display.

[1848] "Example 4"

[1849] Step 1: Share your living situation and health status with relevant people in your community via email, messaging apps, phone calls, or other communication methods.

[1850] Step 2: Collaborate with relevant parties in the local community to establish a system for monitoring users. This collaboration will involve local public institutions, volunteer groups, and local residents.

[1851] Example 1

[1852] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1853] In modern society, there is a demand for methods to stabilize users' mental health and stimulate their brains. It is also important to detect abnormalities such as dementia early and take appropriate measures. However, systems to solve these issues have not yet been fully developed. Furthermore, there is a need for advice on improving health and for a society that monitors users through cooperation with the local community.

[1854] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes a means for conducting a conversation via voice input and contributing to the user's mental stability and brain activation, a means for converting the user's voice into text using voice recognition technology, a means for generating an appropriate response using a generative AI model, a means for converting the generated response into voice using text-to-speech conversion technology, a means for analyzing the state of the conversation and detecting abnormalities such as dementia tendencies, a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities, a means for providing advice on improving health through collaboration with IoT and healthcare systems, and a means for supporting the realization of a monitoring society through collaboration with the local community. This enables early detection of abnormalities and appropriate responses while promoting the user's mental stability and brain activation.

[1855] "Voice input" is a technology that captures the voice a user speaks through a microphone as digital data.

[1856] "Mental stability" refers to maintaining or improving the user's psychological stability.

[1857] "Brain activation" refers to stimulating the user's brain functions and improving their cognitive and thinking abilities.

[1858] "Speech recognition technology" is a technology that converts voice data into text data.

[1859] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input.

[1860] "Text-to-speech conversion technology" is a technology that converts text data into voice data.

[1861] "Means for analyzing the state of conversation" refers to technology that analyzes the content and patterns of user speech and detects abnormalities.

[1862] "Dementia tendency" refers to a state in which a user shows early symptoms of dementia.

[1863] "Means for detecting anomalies" refers to technology that discovers anomalies from users' statements and behavior.

[1864] "Means of notifying hospitals, facilities, families, etc." refers to technology that notifies relevant parties of detected abnormalities.

[1865] "IoT" is a technology that allows various devices to connect via the Internet.

[1866] A "healthcare system" is a system for managing and improving the health status of users.

[1867] "Health Improvement Advice" refers to providing advice to improve a user's health.

[1868] "Collaboration with the local community" refers to working with the local community in which the user lives to monitor and provide support.

[1869] A "watchful society" refers to a system in which the entire local community watches over the safety and health of users.

[1870] This invention is a system for promoting mental stability and brain activation for users, and is realized using voice input, voice recognition, generative AI models, and text-to-speech conversion technologies. Specific embodiments of this system are described below.

[1871] Hardware and Software Configuration

[1872] 1. Voice Input

[1873] The user inputs voice through a microphone. The device captures this voice and sends it to the server as digital data. The hardware used is a microphone and a voice capture device.

[1874] 2. Voice Recognition

[1875] The server uses speech recognition technology to convert the received voice data into text data. Specifically, it uses a speech recognition API (for example, a general speech recognition service). This API provides highly accurate speech recognition and accurately converts the user's speech into text.

[1876] 3. Response Generation

[1877] The server uses a generative AI model (e.g., a general generative AI model) to generate appropriate responses to the user's textual utterances. The model generates responses that include content that contributes to the user's mental stability and brain activity.

[1878] 4. Text to speech conversion

[1879] The generated response text is converted into audio data using text-to-speech conversion technology. Specifically, a speech synthesis API (e.g., a general speech synthesis service) is used. This API provides natural speech synthesis and generates audio that is easy for the user to understand.

[1880] 5. Providing voice responses

[1881] The terminal receives the voice data sent from the server and provides a response to the user through the speaker, allowing the user to continue the conversation with the system. The hardware used is a speaker and an audio playback device.

[1882] Specific examples

[1883] Example 1: When a user is stressed

[1884] 1. User: "I'm really stressed out today."

[1885] 2. Device: Captures audio with a microphone and sends it to the server.

[1886] 3. Server: Uses speech recognition API to convert "I'm very stressed today" into text.

[1887] 4. Server: Uses a generative AI model to generate a response such as "That's tough. Try taking a deep breath to relax."

[1888] 5. Server: Convert the response into speech using a speech synthesis API.

[1889] 6. Device: Through the speaker, the user is told, "That's tough. Try taking a deep breath to relax."

[1890] Example 2: When the user wants to stimulate their brain

[1891] 1. User: "I want to learn something new today."

[1892] 2. Device: Captures audio with a microphone and sends it to the server.

[1893] 3. Server: Use speech recognition API to convert "I want to learn something new today" into text.

[1894] 4. Server: Uses a generative AI model to generate a response like, "That's great! Want to talk about some recent advances in science and technology?"

[1895] 5. Server: Convert the response into speech using a speech synthesis API.

[1896] 6. Terminal: Through the speaker, the user is told, "That's great! Shall I tell you about some recent advances in science and technology?"

[1897] Prompt Sentence Examples

[1898] "How do I respond if a user is stressed?"

[1899] "When a user says they want to learn something new, how do I respond?"

[1900] This system combines voice recognition technology with generative AI models to enable natural conversations with users, contributing to mental stability and brain activation.

[1901] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1902] Step 1:

[1903] The user inputs voice through a microphone. The device captures this voice and stores it as digital data. Specifically, the device's microphone picks up the user's voice and stores it as voice data. This data is sent to the server in real time. The input is the user's voice, and the output is digital voice data.

[1904] Step 2:

[1905] The server uses voice recognition technology to convert the received voice data into text data. Specifically, the server sends the voice data to a voice recognition API, which analyzes the voice and converts it into text. The converted text is returned to the server and passed on to the next processing step. The input is digital voice data, and the output is text data.

[1906] Step 3:

[1907] The server uses a generative AI model to generate appropriate responses to the user's textual utterances. Specifically, the server inputs text data into the generative AI model, which then generates a response based on the user's utterances. This response includes content that contributes to the user's mental stability and brain activation. The input is text data, and the output is a response text.

[1908] Step 4:

[1909] The generated response text is converted into audio data using text-to-speech technology. Specifically, the server sends the response text to a speech synthesis API, which analyzes the text and generates natural-sounding audio. This audio data is returned to the server and passed on to the next processing step. The input is the response text, and the output is audio data.

[1910] Step 5:

[1911] The terminal receives the voice data sent from the server and provides a response to the user through the speaker. Specifically, the terminal receives the voice data and plays it through the speaker, allowing the user to continue a conversation with the system. The input is the voice data, and the output is a voice response to the user.

[1912] (Application example 1)

[1913] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1914] In modern society, there is a demand for methods to stabilize users' mental health and stimulate their brains. It is also important to detect abnormalities such as dementia early and take appropriate measures. Furthermore, there is a demand to improve the quality of life by enabling users to easily use food delivery services. However, it is difficult to realize these functions in a single system, and an efficient solution is needed.

[1915] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1916] In this invention, the server includes: a means for conducting a conversation via voice input, contributing to the user's mental stability and brain activation; a means for analyzing the state of the conversation and detecting abnormalities such as dementia; a means for notifying hospitals, facilities, family members, etc. of the detected abnormalities; a means for providing health improvement advice through collaboration with IoT and healthcare systems; a means for supporting the realization of a watchful society through collaboration with the local community; a means for converting voice input into text, generating an appropriate response, and conveying it to the user using speech synthesis technology; a means for placing a food delivery order based on the user's voice input; and a means for generating an appropriate response based on the user's order using a generative AI model. This enables the early detection of abnormalities such as dementia and appropriate responses to be taken while promoting the user's mental stability and brain activation. It also allows users to easily use food delivery services, improving their quality of life.

[1917] "Voice input" refers to a user inputting voice through a device such as a microphone.

[1918] "Conversation" refers to the exchange of information between a user and a system via voice.

[1919] "Mental stability" refers to maintaining or improving the user's psychological stability.

[1920] "Brain activation" means stimulating and activating the user's brain functions.

[1921] "Conversational state" refers to the content and quality of the conversation between the user and the system.

[1922] "Dementia tendency" refers to a state in which the user may develop dementia.

[1923] An "abnormality" is a phenomenon or symptom that deviates from the normal state.

[1924] "Hospitals, facilities, families, etc." refers to medical institutions, nursing homes, families, etc. that are involved in the user's health condition.

[1925] "Notification" is the act of conveying specific information to others.

[1926] "IoT" is a technology that connects various devices to each other via the Internet.

[1927] A "healthcare system" is a system that supports users in managing their health.

[1928] "Health Improvement Advice" refers to advice or suggestions for improving the user's health.

[1929] "Local community" refers to the local community in which the user lives.

[1930] A "watchful society" is a system in which the entire local community watches over the safety and health of users.

[1931] "Speech synthesis technology" refers to the technology that converts text data into speech.

[1932] "Food delivery" refers to a service that delivers meals ordered by users to a specified location.

[1933] A "generative AI model" is a model that uses artificial intelligence to generate responses or results for a specific task.

[1934] An "appropriate response" is an appropriate and helpful reply to the user's input.

[1935] A system for implementing this invention provides a function that contributes to the user's mental stability and brain activation by conducting conversations via voice input. The system realizes interaction with the user using voice recognition technology, voice synthesis technology, and generative AI models.

[1936] Hardware and Software Configuration

[1937] The system uses the following hardware and software:

[1938] Hardware: microphone, speakers, smartphone or computer

[1939] Software: Speech recognition library, text-to-speech library, generative AI model

[1940] Processing flow

[1941] 1. Acquiring voice input:

[1942] The user provides voice input through a microphone, which is converted to text using a speech recognition library.

[1943] 2. Response generation using generative AI models:

[1944] The converted text is then fed into a generative AI model (e.g., OpenAI's API) to generate an appropriate response. An example prompt is "The user wants to order food delivery. Please generate an appropriate response based on the following order details: I would like to order pizza."

[1945] 3. Providing a response via speech synthesis:

[1946] The generated response is converted into audio using a text-to-speech library and provided to the user through the speaker.

[1947] Specific examples

[1948] For example, if a user says "I want to order a pizza," the system will do the following:

[1949] 1. The user's voice input "I would like to order pizza" is picked up by the microphone.

[1950] 2. The speech recognition library converts the speech into text: "I would like to order a pizza."

[1951] 3. The generative AI model generates a response based on the prompt: "The user wants to order food delivery. Please generate an appropriate response based on the following order: I would like to order pizza."

[1952] 4. The generated response, "What type and size of pizza do you want?" is converted into speech by the speech library and provided to the user through the speaker.

[1953] In this way, users can place food delivery orders through voice input, and the system will provide appropriate responses, contributing to the user's mental well-being and brain activity.

[1954] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1955] Step 1:

[1956] The user inputs voice through the microphone. The input is the user's voice, and the output is voice data. The terminal acquires this voice data and passes it to the next step.

[1957] Step 2:

[1958] The device converts voice data into text using a speech recognition library. The input is voice data and the output is text data. Specifically, the speech recognition engine analyzes the voice waveform and generates the corresponding text.

[1959] Step 3:

[1960] The server uses a generative AI model to generate an appropriate response based on the text data. The input is text data, and the output is response text. Specifically, the text data is input to the generative AI model as a prompt sentence, and the respo...

Claims

1. means for conversing with a user through voice input from the user; A means for analyzing a state of conversation with the user and detecting an abnormality in the daily life of the user; A means for notifying a previously set contact of the detected abnormality; A means for analyzing the user's emotions in real time and detecting changes in the user's emotions; A means for adjusting the content of the conversation in response to the emotion of the user; A system including:

2. The abnormality detection means detects at least one of an abnormality in the user's eating habits and an abnormality in the user's conversation. The system of claim 1 .

3. A means for collecting and analyzing lifestyle and health data of the user; means for generating advice for improving the user's physical condition based on the analysis result by the analyzing means and providing the advice to the user; The system of claim 1 , comprising:

Images