System

Abstract

To provide a system.SOLUTION: The system includes means for acquiring voice data including sound quality, tone and volume of a user's voice through interaction with the user by a conversational AI, means for detecting signs of illness by inputting the voice data into a generative AI, and means for notifying at least one of a medical institution, a nursing care facility and a security company of the detected signs of illness.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the super-aging society and labor shortage have become more serious, and health management of elderly people living alone has become a particular issue. However, it is difficult for humans to check the health status of elderly people 24 hours a day. [Means for solving the problem]

[0005] This invention provides a means for conversational AI to acquire vocal cord data from conversations with elderly people, and for a disease diagnosis AI model to detect signs of illness from vocal cord analysis. It also provides a means for notifying medical institutions, the nursing care industry, and security companies of any signs of illness that are detected. This eliminates the need for humans to check the health status of elderly people 24 hours a day, reducing the burden on those monitoring them. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The present invention is a system that combines a conversational AI and a disease diagnosis AI model. The conversational AI acquires vocal cord data through dialogue with the elderly. This dialogue is conducted, for example, through everyday conversations or health-related questions. The acquired vocal cord data is sent to the disease diagnosis AI model for analysis. The disease diagnosis AI model detects signs of illness from the vocal cord data. The detected signs of illness are notified to medical institutions, the nursing care industry, and security companies. This makes it possible to check the health status of the elderly 24 hours a day and detect signs of illness early.

[0029] "Example 2"

[0030] One aspect of the present invention is a system that uses the vast knowledge of a conversational AI to engage in conversations with the elderly. The conversational AI possesses a wide range of knowledge, such as medical knowledge and topics that may be of interest to the elderly. This makes it possible to acquire vocal cord data through natural conversations with the elderly.

[0031] "Example 3"

[0032] One aspect of the present invention is a system in which a disease diagnosis AI model uses generative AI to detect signs of disease from vocal cord analysis. The generative AI learns patterns in vocal cord data and can generate signs of disease from those patterns. This enables the disease diagnosis AI model to more accurately detect signs of disease from vocal cord data.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The conversational AI initiates a dialogue with the elderly person through questions such as, "How are you feeling today?" and "Has anything changed recently?"

[0036] Step 2: The conversational AI acquires the elderly person's vocal data, including the quality, tone, and volume of the elderly person's voice.

[0037] Step 3: The acquired vocal cord data is sent to a disease diagnosis AI model, which then identifies signs of disease from the vocal cord data.

[0038] Step 4: Any signs of illness detected by the disease diagnosis AI model are notified to medical institutions, the nursing care industry, and security companies, making it possible to detect signs of illness early and take appropriate measures.

[0039] "Example 2"

[0040] Step 1: The conversational AI initiates a dialogue with the senior through questions about topics that may be of interest to the senior, such as "What book have you read recently?" or "What is your favorite TV show?"

[0041] Step 2: The conversational AI acquires the elderly person's voice data (vocal cord data), which includes the quality, tone, and volume of the elderly person's voice.

[0042] Step 3: The acquired vocal cord data is sent to a disease diagnosis AI model, which then identifies signs of disease from the vocal cord data.

[0043] "Example 3"

[0044] Step 1: The conversational AI initiates a dialogue with the elderly person and acquires vocal cord data.

[0045] Step 2: The acquired vocal cord data is sent to a disease diagnosis AI model. The disease diagnosis AI model uses generative AI to find signs of disease from the vocal cord data. The generative AI learns patterns in the vocal cord data and generates signs of disease from those patterns.

[0046] Step 3: Any signs of illness detected by generative AI are reported to medical institutions, the nursing care industry, and security companies. This allows for early detection of signs of illness and the appropriate response.

[0047] Example 1

[0048] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] There is a need to check the health status of the elderly 24 hours a day and detect early signs of illness. However, conventional systems do not adequately collect and analyze voice data, which can result in signs of illness being overlooked. In addition, there is an insufficient mechanism for appropriately notifying medical institutions, the nursing care industry, and security companies of the collected data. This creates the problem of ineffective health management for the elderly.

[0050] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0051] In this invention, the server includes a means for the conversational AI to acquire voice data from a conversation with the elderly person, a means for the terminal to transmit the acquired voice data to the server, a means for the server to input the voice data into a disease diagnosis AI model, a means for the disease diagnosis AI model to detect signs of illness from the voice data, and a means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness. This makes it possible to check the health status of the elderly person 24 hours a day and detect signs of illness early.

[0052] "Conversational AI" is an artificial intelligence system that acquires voice data through dialogue with elderly people.

[0053] "Voice data" refers to voice information collected when elderly people speak to conversational AI.

[0054] A "terminal" is a device on which conversational AI is installed and which collects and transmits voice data to a server.

[0055] The "server" is a computer system that receives voice data sent from the terminal and inputs it into the disease diagnosis AI model.

[0056] The "disease diagnosis AI model" is an artificial intelligence model that analyzes voice data and detects signs of disease.

[0057] "Signs of illness" are abnormalities in health or early symptoms of illness detected from voice data.

[0058] A "medical institution" is an organization that receives symptoms of illness and provides appropriate medical treatment.

[0059] The "care industry" is an industry that manages the health of the elderly and provides necessary care services.

[0060] A "security company" is a company that ensures the safety of the elderly and responds quickly when necessary.

[0061] "Generative AI" is an artificial intelligence technology used by disease diagnosis AI models to find signs of disease from voice data.

[0062] MODE FOR CARRYING OUT THE INVENTION

[0063] This invention is a system that checks the health status of elderly people 24 hours a day and detects early signs of illness. This system is composed of a conversational AI, a terminal, a server, and a disease diagnosis AI model.

[0064] Conversational AI and Devices

[0065] The user (elderly person) speaks to a device (smartphone or dedicated device) on which the conversational AI is installed. The conversational AI acquires voice data through everyday conversations with the user and health-related questions. This conversational AI uses a general dialogue engine that uses natural language processing technology, for example.

[0066] Collection and transmission of voice data

[0067] The device collects voice data acquired through conversational AI in real time. The collected voice data is sent to the server using a secure communication protocol (e.g., HTTPS). The data is encrypted before transmission to protect the user's privacy.

[0068] Server and disease diagnosis AI model

[0069] The server receives the voice data sent from the device. The received voice data is input into a disease diagnosis AI model. This disease diagnosis AI model is built using machine learning frameworks such as TENSORFLOW (registered trademark) and PyTorch. The server also performs preprocessing of the voice data (e.g., noise removal and voice normalization).

[0070] Disease symptom detection and notification

[0071] The AI ​​disease diagnosis model analyzes voice data to detect signs of illness. For example, it detects abnormalities such as voice tremors or changes in pitch and determines what kind of illness they are. The analysis results are then generated as a detailed report to assess the user's health status.

[0072] The server then sends the analysis results to medical institutions, the nursing care industry, and security companies via email, SMS, or a dedicated application, including any detected symptoms of illness and recommended measures.

[0073] Examples of specific examples and prompts

[0074] Examples:

[0075] The user speaks to the conversational AI, saying, "I've been coughing at night lately."

[0076] The device records this conversation and transmits the audio data to a server in real time.

[0077] The server inputs the received voice data into a disease diagnosis AI model to analyze cough patterns.

[0078] The server detects from the coughing pattern that it may be an early symptom of a cold or bronchitis.

[0079] The server notifies medical institutions of this information and encourages the user to seek medical attention early.

[0080] Example prompt sentence:

[0081] If an elderly person says to a conversational AI, "I've been coughing at night lately," please explain how the voice data is collected and analyzed by the disease diagnosis AI model.

[0082] This system makes it possible to check the health status of elderly people 24 hours a day and detect early signs of illness.

[0083] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0084] Program processing flow

[0085] Step 1: The user initiates a dialogue with the conversational AI

[0086] Users talk to a device with conversational AI installed, asking everyday conversational or health-related questions such as, "What's the weather like today?" or "I've had a bit of a sore throat lately."

[0087] Input: User's voice

[0088] Output: Acquiring voice data using conversational AI

[0089] Specific operation: When a user speaks into the device, the device's built-in microphone captures the voice, and the conversational AI recognizes the voice.

[0090] Step 2: The device collects audio data

[0091] The device records the user's voice through conversational AI, using the device's built-in microphone or an external microphone, and the recorded voice data is processed in real time.

[0092] Input: User's voice

[0093] Output: Recorded audio data

[0094] What it does: The device stores the audio data digitally and performs noise reduction and voice normalization as needed.

[0095] Step 3: The device sends the audio data to the server

[0096] The device sends the collected voice data to the server using a secure communication protocol (e.g., HTTPS). The data is encrypted before transmission to protect the user's privacy.

[0097] Input: Recorded audio data

[0098] Output: Audio data sent to the server

[0099] Specific operation: The device encrypts the voice data and sends it to a server over the Internet.

[0100] Step 4: The server inputs the voice data into the disease diagnosis AI model

[0101] The server inputs the received voice data into a disease diagnosis AI model, which is built using machine learning frameworks such as TensorFlow and PyTorch. The server also performs data preprocessing (e.g., noise removal and voice normalization).

[0102] Input: Audio data sent to the server

[0103] Output: Data input into the disease diagnosis AI model

[0104] Specific operation: The server preprocesses the voice data and converts it into a format suitable for the disease diagnosis AI model.

[0105] Step 5: The disease diagnosis AI model analyzes the voice data

[0106] The AI ​​disease diagnosis model analyzes voice data to detect signs of illness, for example, detecting abnormalities such as voice tremors or changes in pitch and determining what kind of illness they indicate.

[0107] Input: Preprocessed audio data

[0108] Output: Analysis results for symptoms of disease

[0109] How it works: The disease diagnosis AI model analyzes the voice data and detects abnormal patterns.

[0110] Step 6: The server notifies the results to medical institutions, the nursing care industry, and security companies.

[0111] The server notifies medical institutions, the nursing care industry, and security companies of the analysis results output by the disease diagnosis AI model via email, SMS, and a dedicated application.

[0112] Input: Analysis results for disease symptoms

[0113] Output: Notified analysis results

[0114] Specific behavior: The server converts the parsed results into an appropriate format and sends them to the specified recipient.

[0115] In this way, the system can monitor the health of the elderly 24 hours a day and detect early signs of illness.

[0116] (Application example 1)

[0117] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] It is important to monitor the health status of the elderly on a daily basis and detect early signs of illness, but the current system makes it difficult for the elderly to check their health themselves. Frequent visits to medical institutions and nursing homes are also a burden. Furthermore, there is an inadequate notification system for prompt action when abnormalities are detected.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0120] In this invention, the server includes means for an interactive AI to acquire vocal cord data from a conversation with the elderly person, means for an AI disease diagnosis model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for interacting with the elderly person using a smart device to acquire vocal cord data, means for transmitting the acquired vocal cord data to the AI ​​disease diagnosis model, means for displaying the diagnosis results in real time, and means for notifying when an abnormality is detected. This makes it possible to monitor the health status of the elderly person on a daily basis, detect signs of illness early, and respond promptly.

[0121] "Conversational AI" is an artificial intelligence system that acquires voice data through dialogue with elderly people and uses natural language processing technology to hold conversations.

[0122] "Vocal cord data" refers to the voice signals obtained by recording the voices of elderly people and capturing them as digital data.

[0123] The "disease diagnosis AI model" is a machine learning model that analyzes acquired vocal cord data and detects signs of disease.

[0124] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0125] The "care industry" refers to the entire industry that provides care services to the elderly and disabled.

[0126] A "security company" is a company that provides security services to protect the safety of facilities and individuals.

[0127] A "smart device" is a portable electronic device with internet connectivity and advanced computing capabilities.

[0128] "Diagnosis results" are information obtained as a result of the disease diagnosis AI model analyzing vocal cord data.

[0129] "Real-time" refers to data acquisition and processing occurring immediately.

[0130] "Notification" is the act of conveying information to relevant parties when an abnormality is detected.

[0131] The following describes an embodiment of the present invention. The system includes a conversational AI, a disease diagnosis AI model, a smart device, and a notification function.

[0132] First, conversational AI acquires vocal data through dialogue with the elderly. Conversational AI interacts with the elderly using smart devices such as smartphones and smart glasses to collect voice data. The dialogue content includes everyday conversations and health-related questions, and is conducted using natural language processing technology.

[0133] The acquired vocal cord data is then sent to a disease diagnosis AI model, which uses machine learning algorithms to analyze the vocal cord data and detect signs of disease. This analysis can be performed using generative AI.

[0134] The diagnosis results are displayed in real time. The diagnosis results are instantly displayed on the screen of the smart device, where they can be checked by the elderly person and store staff. If the diagnosis results indicate an abnormality, a notification function is activated, and the abnormality is notified to medical institutions, the nursing care industry, and security companies.

[0135] As a concrete example, when an elderly person undergoes a health check in a physical store, they wear smart glasses and a conversational AI asks, "How is your health these days?" The acquired voice data is sent to an AI model for diagnosing illnesses, and if the result indicates "abnormality," a notification is sent to the store staff.

[0136] The implementation of this system requires the following hardware and software: a smartphone, smart glasses, and a microphone. The software uses Python, the SpeechRecognition library, and the requests library.

[0137] An example of a prompt is as follows:

[0138] "Please create a program to acquire vocal cord data of the elderly, send it to a disease diagnosis AI model, and display the diagnosis results. Please also add a function to notify store staff if an abnormality is detected."

[0139] In this way, a system for implementing the present invention is constructed.

[0140] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0141] Step 1:

[0142] A user starts a smart device (smartphone or smart glasses) and starts a conversational AI application. The user begins a dialogue by following the application's instructions. The input is the user's voice, and the output is voice data. The smart device uses a microphone to record the user's voice and acquires it as voice data.

[0143] Step 2:

[0144] The device sends the acquired voice data to the conversational AI. The conversational AI analyzes the voice data using natural language processing technology and engages in a dialogue with the user. The input is the voice data, and the output is the analysis results and the content of the dialogue. The conversational AI converts the user's voice data into text and generates an appropriate response.

[0145] Step 3:

[0146] The device returns the response from the conversational AI to the user by voice. The input is the conversational AI's response text, and the output is a voice response. The device converts the text into voice and transmits it to the user through the speaker.

[0147] Step 4:

[0148] The device sends the acquired voice data to the disease diagnosis AI model. The input is the voice data and the output is the diagnosis result. The disease diagnosis AI model analyzes the voice data and performs data processing and calculations to detect signs of disease.

[0149] Step 5:

[0150] The server receives the diagnosis results from the disease diagnosis AI model and displays them on the device in real time. The input is the diagnosis result and the output is the display content. The server sends the diagnosis result to the device, which then displays the result on the screen.

[0151] Step 6:

[0152] If the diagnosis result indicates an abnormality, the server activates a notification function. The input is the diagnosis result, and the output is the notification content. The server sends a notification of the abnormality to medical institutions, the nursing care industry, and security companies.

[0153] Step 7:

[0154] The terminal displays the diagnosis results and notification content to the user. The input is the diagnosis results and notification content, and the output is the display content. The terminal displays the diagnosis results and notification content on the screen and notifies the user that an abnormality has been detected.

[0155] Through these steps, the system will be able to monitor the health status of elderly people on a daily basis, detect signs of illness early, and respond promptly.

[0156] Example 2

[0157] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] There is a need to acquire vocal cord data through dialogue with the elderly and use that data to detect early signs of illness. However, conventional systems often lack natural dialogue and appropriate responses, which can lead to elderly people not actively participating in the dialogue. Furthermore, there are challenges in effectively analyzing the acquired vocal cord data and accurately detecting signs of illness.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0160] In this invention, the server includes means for the conversational AI to acquire vocal cord data from a conversation with the elderly, means for analyzing the voice data and converting it into text data, means for inputting a prompt sentence into the generation AI model to generate a response, means for converting the generated response into voice data and communicating it to the user, means for the disease diagnosis AI model to detect signs of illness from the vocal cord analysis, and means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness. This makes it possible to effectively acquire vocal cord data through natural conversation with the elderly and to detect signs of illness early based on that data.

[0161] "Conversational AI" is an artificial intelligence system that can engage in natural dialogue with the elderly, and has the ability to analyze voice input and generate appropriate responses.

[0162] "Vocal cord data" is digital data obtained from the voice of an elderly person, and includes vocal characteristics and voice patterns.

[0163] "Voice data" refers to data that is a digital recording of a user's speech and is acquired through a microphone.

[0164] "Text data" is text information that has been converted by analyzing voice data and converted into a format that is easy for conversational AI to understand.

[0165] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to generate appropriate responses based on input prompts.

[0166] A "prompt sentence" is an instruction sentence input into a generative AI model, and is generated based on the user's speech.

[0167] The "disease diagnosis AI model" is an artificial intelligence system that analyzes vocal cord data to find signs of disease and makes diagnoses based on medical knowledge.

[0168] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0169] The "care industry" refers to the industry that provides care services for the elderly and disabled, and includes care facilities and home care services.

[0170] A "security company" is a company that provides security services, responds to emergencies, and manages safety.

[0171] This invention is a system that acquires vocal cord data through dialogue with elderly people and uses that data to detect early signs of illness. The system includes conversational AI, speech recognition software, generative AI models, speech synthesis software, and disease diagnosis AI models.

[0172] The server launches a conversational AI to engage in natural conversation with the elderly. The conversational AI uses a generative AI model, such as OpenAI's GPT-4 (registered trademark). This AI model contains a vast amount of data on medical knowledge and topics of interest to the elderly.

[0173] The terminal is a device used by the elderly person and is equipped with a microphone and speaker. The terminal receives voice input from the user (elderly person) and sends it to the server. The server analyzes the received voice data and converts it into text data using voice recognition software (e.g., Google® Cloud Speech-to-Text API).

[0174] The server inputs a prompt sentence into the generative AI model based on the converted text data. An example of a prompt sentence is, "The user is asking for advice about knee pain. Please provide some advice on what to do about knee pain."

[0175] The generative AI model generates an appropriate response based on the prompt, such as, "There are several ways to treat knee pain. First, it is important to engage in moderate exercise. Walking and water exercises, which do not put strain on the knee, are particularly effective. You can also use cold or hot compresses to relieve pain. If necessary, we recommend that you consult a doctor and receive appropriate treatment."

[0176] The server sends the generated response to the device, which converts the received text data into voice data using speech synthesis software (e.g., Amazon Polly), and transmits the converted voice data to the user through the speaker.

[0177] The server stores the vocal cord data acquired during the dialogue process and performs vocal cord analysis using a disease diagnosis AI model. The disease diagnosis AI model analyzes the vocal cord data and detects signs of illness. Any signs of illness detected are notified to medical institutions, the nursing care industry, and security companies.

[0178] As a concrete example, a user speaks to the device saying, "My knees have been hurting lately. Is there anything I can do about it?" The device captures the user's voice with a microphone and sends it to the server as digital voice data. The server uses the Google Cloud Speech-to-Text API to convert the voice data into text data and inputs the prompt sentence into the generative AI model. The generative AI model generates an appropriate response, and the server sends that response to the device. The device then converts the response into voice data using Amazon Polly and communicates it to the user through the speaker.

[0179] In this way, vocal cord data can be effectively obtained through natural conversation with the elderly, and signs of illness can be detected early based on that data.

[0180] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0181] Step 1:

[0182] The user speaks into the terminal.

[0183] The user asks the device a question such as, "My knees have been hurting lately. Is there anything I can do about it?" The input is the user's voice, and the output is digital voice data captured by the device's microphone.

[0184] Step 2:

[0185] The terminal transmits the voice data to the server.

[0186] The device transmits the captured digital audio data to a server via an internet connection, with the input being the digital audio data and the output being the audio data transmitted to the server.

[0187] Step 3:

[0188] The server analyzes the voice data and converts it into text data.

[0189] The server converts the received voice data into text data using voice recognition software (for example, Google Cloud Speech-to-Text API). The input is voice data, and the output is text data such as "I've been having trouble with knee pain lately. Is there anything I can do about it?"

[0190] Step 4:

[0191] The server inputs a prompt sentence into the generative AI model.

[0192] The server inputs a prompt sentence into a generative AI model (for example, OpenAI's GPT-4) based on the converted text data. The prompt sentence is, "The user is asking for advice about knee pain. Please provide advice on what to do about knee pain." The input is text data, and the output is the prompt sentence.

[0193] Step 5:

[0194] A generative AI model generates a response.

[0195] The generative AI model generates an appropriate response based on the prompt sentence. For example, it generates a response such as, "There are several measures to treat knee pain. First, it is important to engage in moderate exercise. Walking and water exercises, which do not put strain on the knee, are particularly effective. You can also use cold or hot compresses to relieve pain. If necessary, we recommend that you consult a doctor and receive appropriate treatment." The input is the prompt sentence, and the output is the generated response text.

[0196] Step 6:

[0197] The server sends the generated response to the terminal.

[0198] The server sends the response obtained from the generative AI model to the terminal. The response is sent as text data. The input is the generated response text, and the output is the text data sent to the terminal.

[0199] Step 7:

[0200] The terminal will then vocalize the response to the user.

[0201] The device converts the received text data into voice data using speech synthesis software (e.g., Amazon Polly), and transmits the converted voice data to the user through a speaker. The input is text data, and the output is voice data.

[0202] Step 8:

[0203] The server stores the vocal cord data acquired during the dialogue.

[0204] The server stores the voice and text data acquired during the dialogue, which can then be used for later analysis and system improvement. The input is the voice and text data, and the output is the stored data.

[0205] (Application example 2)

[0206] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0207] When elderly people shop in physical stores, it is difficult for them to obtain product information and health advice. There is also the issue of difficulty in collecting data to monitor the health status of elderly people. To solve these problems, a system that allows elderly people to interact naturally is needed.

[0208] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0209] In this invention, the server includes means for the conversational AI to acquire vocal cord data from a conversation with the elderly person, means for the disease diagnostic AI model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for the elderly person to receive product information and health advice by talking to the conversational AI via smartphone while shopping in a physical store, and means for acquiring the elderly person's vocal cord data through the conversation and monitoring their health condition.This will support the elderly in making shopping in physical stores more comfortable while also enabling health condition monitoring.

[0210] "Conversational AI" is an artificial intelligence system that provides information and collects data through natural dialogue with the elderly.

[0211] "Vocal cord data" is audio information obtained by analyzing the voices of elderly people, and is data used to monitor their health status.

[0212] The "disease diagnosis AI model" is an artificial intelligence model that analyzes vocal cord data to find signs of disease.

[0213] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0214] The "care industry" is an industry that provides care services to the elderly and disabled.

[0215] A "security company" is a company that provides security services to protect the safety of facilities and individuals.

[0216] A "brick and mortar store" is a store that exists in a physical location where customers can visit and purchase products in person.

[0217] A "smartphone" is a mobile phone that can connect to the Internet and use applications.

[0218] "Product information" refers to information about the product's features, price, usage, etc.

[0219] "Health advice" is advice or information about maintaining health and preventing disease.

[0220] "Health monitoring" means continuously observing an individual's health status to check for any abnormalities.

[0221] The system for implementing this invention includes an interactive AI, a disease diagnosis AI model, a smartphone, and a server. The specific configuration and operation of the system are described below.

[0222] System configuration

[0223] 1. Conversational AI: Provides information and collects vocal data through natural conversations with seniors. Conversational AI possesses vast knowledge, including medical knowledge and topics of interest to seniors.

[0224] 2. Disease diagnosis AI model: Conversational AI analyzes collected vocal cord data to detect signs of illness. Generative AI is used to provide more accurate diagnoses.

[0225] 3. Smartphones: These devices are used by seniors while shopping in physical stores and provide an interface for interacting with conversational AI.

[0226] 4. Server: Hosts the conversational AI and disease diagnosis AI models, and processes and stores data.

[0227] System Operation

[0228] 1. Acquiring voice input: The elderly person speaks to the conversational AI through their smartphone. The smartphone's microphone captures the voice and converts it into text using speech recognition software (e.g., SpeechRecognition library).

[0229] 2. Response generation by conversational AI: Based on the converted text, conversational AI generates an appropriate response. This is done using OpenAI's API.

[0230] 3. Voice output: The responses generated by the conversational AI are output as voice using speech synthesis software (e.g., the pyttsx3 library).

[0231] 4. Collection and analysis of vocal cord data: The AI ​​disease diagnosis model analyzes vocal cord data collected during the conversation to detect signs of illness.

[0232] 5. Notification: Any signs of illness detected will be notified to medical institutions, the care industry, and security companies.

[0233] Specific examples

[0234] For example, if an elderly person asks, "What effect does this supplement have?", the conversational AI will generate a response such as, "This supplement is rich in vitamin D and supports bone health," and output it in voice. At the same time, the vocal cord data collected through this dialogue will be analyzed by a disease diagnosis AI model, and if any abnormalities are detected, a medical institution will be notified.

[0235] Prompt Sentence Examples

[0236] User: What does this supplement do?

[0237] AI: This supplement is rich in vitamin D and supports bone health.

[0238] In this way, it is possible to help seniors feel more comfortable shopping in physical stores while also monitoring their health.

[0239] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0240] Step 1:

[0241] The user speaks to the conversational AI through their smartphone. The smartphone's microphone captures the voice and uses speech recognition software (e.g., the SpeechRecognition library) to convert the voice data into text data.

[0242] Input: User's voice

[0243] Output: Text data

[0244] What it does: Your smartphone's microphone captures your voice, and the SpeechRecognition library converts it into text.

[0245] Step 2:

[0246] The server receives the text data and the conversational AI generates an appropriate response, using OpenAI's API.

[0247] Input: Text data

[0248] Output: AI response text

[0249] Specific operation: The server sends text data to the OpenAI API and receives the generated response text.

[0250] Step 3:

[0251] The server sends the generated response text to the smartphone, which then outputs it aloud using speech synthesis software (e.g., the pyttsx3 library).

[0252] Input: AI response text

[0253] Output: Voice response

[0254] Specific operation: The server sends the response text to the smartphone, and the pyttsx3 library converts the text into speech and outputs it.

[0255] Step 4:

[0256] The server sends the vocal cord data collected during the conversation to a disease diagnosis AI model, which analyzes the vocal cord data to detect signs of disease.

[0257] Input: Vocal cord data

[0258] Output: Presence or absence of signs of illness

[0259] Specific operation: The server inputs vocal cord data into a disease diagnosis AI model and performs analysis using generative AI.

[0260] Step 5:

[0261] The server notifies medical institutions, the nursing care industry, and security companies of any signs of illness it detects.

[0262] Input: Presence or absence of signs of illness

[0263] Output: Information message

[0264] What it does: If the server detects any signs of illness, it will send a notification message to pre-defined contacts.

[0265] In this way, the system can help seniors feel more comfortable shopping in brick-and-mortar stores while simultaneously monitoring their health.

[0266] Example 3

[0267] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0268] Conventional disease diagnosis systems have had difficulty accurately detecting symptoms of illness from voice data. Furthermore, there was a lack of means to quickly provide diagnostic results to users, which could delay early medical treatment. Furthermore, there was a problem of insufficient preprocessing and feature extraction of voice data, which reduced the accuracy of diagnosis.

[0269] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0270] In this invention, the server includes means for a user to record voice data, means for a terminal to transmit the voice data to the server, means for the server to preprocess the voice data, means for the server to input data to a generative AI model, means for the generative AI model to generate symptoms of illness, means for the server to input the generated symptoms to a disease diagnostic AI model, means for the disease diagnostic AI model to generate a diagnostic result, means for the server to transmit the diagnostic result to the terminal, and means for the terminal to display the diagnostic result to the user. This makes it possible to accurately detect symptoms of illness from voice data and quickly provide a diagnostic result to the user.

[0271] "User" refers to an individual who utilizes the system to record voice data and receive diagnostic results.

[0272] "Audio data" refers to digital data containing voice information recorded by a user.

[0273] "Terminal" refers to a device that records voice data, transmits it to the server, and displays the diagnosis results. Specifically, this includes smartphones and tablets.

[0274] "Server" refers to a computer system that receives voice data, performs preprocessing, and runs the generative AI model and the disease diagnosis AI model.

[0275] "Preprocessing" refers to the process of removing noise from the audio data and extracting necessary features.

[0276] A "generative AI model" refers to an artificial intelligence model that learns patterns in voice data and generates symptoms of disease from those patterns.

[0277] "Disease symptoms" refer to features or patterns detected in voice data that indicate possible illness.

[0278] A "disease diagnosis AI model" refers to an artificial intelligence model that analyzes the symptoms of disease generated by a generative AI model and generates a final diagnosis result.

[0279] "Diagnosis results" refers to the assessment and diagnostic information regarding the user's health condition generated by the disease diagnosis AI model.

[0280] "Display" refers to the act of the terminal visually presenting the diagnostic results to the user.

[0281] This invention is a system that detects signs of illness using voice data and provides a diagnosis result to the user. The system operates by having the user record voice data and send it to a server via a terminal. The server preprocesses the voice data, generates signs of illness using a generative AI model, and generates a diagnosis result using a disease diagnosis AI model. The diagnosis result is sent to the terminal and displayed to the user.

[0282] Hardware and software used

[0283] Hardware: smartphones, tablets, servers

[0284] Software: TensorFlow, PyTorch, Librosa, audio recording app

[0285] Specific operation of the system

[0286] User operations

[0287] A user uses a device such as a smartphone or tablet to launch a voice recording app. For example, they record a phrase such as "Hello, I'm fine." Once the recording is complete, the device sends the voice data to the server.

[0288] Server Processing

[0289] The server receives the voice data sent from the device. The received voice data is first preprocessed. Preprocessing includes noise removal and Mel-Frequency Cepstral Coefficient (MFCC) extraction using Librosa. The preprocessed data is then input into a generative AI model. This model is built using TensorFlow and PyTorch and learns patterns from the voice data.

[0290] The generative AI model generates signs of illness from input voice data. For example, it analyzes voice tremors and pitch fluctuations to find patterns that indicate possible illness. The generated signs of illness are then input into a disease diagnosis AI model. This model evaluates the likelihood of illness based on past diagnostic data.

[0291] The disease diagnosis AI model analyzes the generated symptoms and generates a final diagnosis. For example, it evaluates the possibility of a cold, flu, or vocal cord polyps. The diagnosis result is sent from the server to the user's device. The result is encrypted and transmitted to protect the user's privacy.

[0292] Terminal display

[0293] The device displays the received diagnosis results to the user, who can then check the results via a smartphone app and consult a medical institution if necessary.

[0294] Specific examples

[0295] For example, if a user feels signs of a cold, they can record their vocal cord data on their smartphone and receive a diagnosis through the system, allowing them to receive appropriate medical treatment early.

[0296] Prompt Sentence Examples

[0297] "The user uses a smartphone to record vocal data and send it to the server. The server uses TensorFlow to run a generative AI model to generate disease symptoms. The generated symptoms are passed to a disease diagnosis AI model, which generates a final diagnosis result. The diagnosis result is then sent to the user's smartphone, where the user can view it."

[0298] This system allows users to easily collect vocal cord data and detect signs of illness at an early stage. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0299] Step 1:

[0300] The user records audio data

[0301] The user uses a voice recording app on their smartphone to record their own voice, for example, by saying a phrase such as "Hello, I'm fine." The recorded voice data is then stored digitally on the device.

[0302] Input: User's voice

[0303] Output: Recorded audio data (digital format)

[0304] Step 2:

[0305] The device sends the audio data to the server.

[0306] The device sends the recorded audio data to a server via the Internet, and the data is encrypted for security purposes.

[0307] Input: Recorded audio data

[0308] Output: Audio data sent to the server

[0309] Step 3:

[0310] The server preprocesses the audio data

[0311] The server preprocesses the received audio data, specifically using the Python library Librosa to remove noise and extract Mel-Frequency Cepstral Coefficients (MFCCs).

[0312] Input: Audio data sent to the server

[0313] Output: Preprocessed speech data (after feature extraction)

[0314] Step 4:

[0315] The server inputs data into the generative AI model

[0316] The preprocessed data is then fed into a generative AI model built using TensorFlow and PyTorch, which learns patterns from the audio data.

[0317] Input: Preprocessed audio data

[0318] Output: Data input into a generative AI model

[0319] Step 5:

[0320] Generative AI model generates disease symptoms

[0321] The generative AI model generates symptoms of illness from input voice data, for example by analyzing voice tremors and pitch fluctuations to find patterns that may indicate illness.

[0322] Input: Data input to a generative AI model

[0323] Output: Generated disease symptoms

[0324] Step 6:

[0325] The server inputs the generated symptoms into a disease diagnosis AI model.

[0326] The generated disease symptoms are then fed into a disease diagnosis AI model, which assesses the likelihood of disease based on past diagnostic data.

[0327] Input: Generated disease symptoms

[0328] Output: Data input into the disease diagnosis AI model

[0329] Step 7:

[0330] Disease diagnosis AI model generates diagnosis results

[0331] The disease diagnosis AI model analyzes the generated symptoms and generates a final diagnosis, such as assessing the likelihood of a cold, flu, or vocal cord polyps.

[0332] Input: Data input into the disease diagnosis AI model

[0333] Output: The generated diagnostic results

[0334] Step 8:

[0335] The server sends the diagnostic results to the device.

[0336] The diagnosis results are sent from the server to the user's device, and are encrypted to protect the user's privacy.

[0337] Input: Generated diagnostic results

[0338] Output: Diagnostic results sent to the terminal

[0339] Step 9:

[0340] The device displays the diagnostic results to the user.

[0341] The device displays the received diagnosis results to the user, who can then check the results via a smartphone app and consult a medical institution if necessary.

[0342] Input: Diagnostic results sent to the terminal

[0343] Output: Diagnostic results displayed to the user

[0344] (Application example 3)

[0345] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0346] Conventional disease diagnosis systems have the problem of low accuracy in detecting signs of illness using vocal cord data, making early detection difficult. Furthermore, there is a lack of means to monitor the user's health status in real time, making it difficult to take effective measures, especially in managing the health of the elderly and employees. Furthermore, notification methods after detecting signs of illness are limited, resulting in delayed responses in situations where prompt action is required.

[0347] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0348] In this invention, the server includes: a conversational AI system for acquiring vocal cord data from a conversation with a user; a disease diagnostic AI model for detecting signs of illness from vocal cord analysis; a system for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness; a smartphone microphone for collecting voice data; a preprocessing system for preprocessing the collected voice data; a generative AI model for generating signals from the preprocessed voice data; a disease diagnostic AI model for inputting the generated signals into the disease diagnostic AI model to detect health risks; and a system for notifying the user of the detected health risks. This enables highly accurate detection of signs of illness from the user's vocal cord data and real-time health monitoring. Furthermore, prompt notification of detected health risks enables early response.

[0349] "Conversational AI" is an artificial intelligence system that collects information and generates responses through natural conversation with users.

[0350] "Vocal cord data" refers to data that is a digital record of the voice signals emitted from the user's vocal cords.

[0351] The "disease diagnosis AI model" is an artificial intelligence model trained to analyze vocal cord data and detect signs of disease.

[0352] A "generative AI model" is an artificial intelligence model that has the ability to generate new data from input data.

[0353] A "smartphone" is a portable multi-function information terminal that can collect voice data and run applications.

[0354] "Voice data" refers to data that is a digital recording of a user's speech.

[0355] "Preprocessing" refers to the process of converting collected speech data into a format suitable for analysis and model input.

[0356] A "signal" is data generated by a generative AI model and input into a disease diagnosis AI model.

[0357] A "health risk" is a potential problem with a user's health status detected by the disease diagnosis AI model.

[0358] "Notification" is the act of informing users and relevant organizations of detected health risks.

[0359] A system for implementing this invention includes an interactive AI, a disease diagnosis AI model, a generative AI model, a smartphone, a means for preprocessing audio data, a signal generation means, a health risk detection means, and a notification means.

[0360] First, the user collects voice data using a smartphone. The smartphone's microphone records the user's vocal data. This voice data is collected in real time and transmitted to the system.

[0361] The server then preprocesses the collected voice data, which includes analyzing the data and extracting features such as Mel-Frequency Cepstrum Coefficients (MFCCs). This preprocessing is done to convert the voice data into a format suitable for the disease diagnosis AI model.

[0362] The preprocessed speech data is input to a generative AI model, which generates a new signal from the preprocessed data. This signal is then input to a disease diagnosis AI model.

[0363] The AI ​​disease diagnosis model analyzes the generated signals to detect signs of disease and health risks. This model has learned patterns in vocal cord data and can detect signs of disease with high accuracy.

[0364] Any detected health risks are communicated to users and relevant authorities through notification methods, which may be via smartphone applications, allowing users to be informed of their health status in real time.

[0365] As a concrete example, consider a scenario in which corporate employees use this system to monitor their daily health. Employees use the app to record their voices every morning, and the system analyzes the vocal data to detect stress, fatigue, and potential health risks. Detected risks are notified to employees and corporate health managers, allowing for early response.

[0366] Example prompt sentence:

[0367] "To monitor employee health, employees use this app to record their voice every morning. The app analyzes the vocal data to detect stress, fatigue, and potential health risks."

[0368] In this way, companies can understand the health status of their employees in real time and take early action.

[0369] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0370] Step 1:

[0371] The user collects voice data using a smartphone. Specifically, the smartphone's microphone is used to record the user's vocal data. The input is the user's voice, and the output is digital voice data.

[0372] Step 2:

[0373] The server preprocesses the collected voice data. Specifically, it analyzes the voice data and extracts features such as Mel-Frequency Cepstrum Coefficients (MFCC). The input is digital voice data, and the output is preprocessed feature data.

[0374] Step 3:

[0375] The server inputs the preprocessed speech data into the generative AI model. Specifically, the preprocessed data is passed to the generative AI model to generate a new signal. The input is the preprocessed feature data, and the output is the generated signal.

[0376] Step 4:

[0377] The server inputs the generated signal into a disease diagnosis AI model. Specifically, the generated signal is passed to the disease diagnosis AI model to detect signs of disease and health risks. The input is the generated signal, and the output is the detected health risk.

[0378] Step 5:

[0379] The server notifies the user and relevant organizations of the detected health risks through notification means. Specifically, it notifies the user of their health status in real time through a smartphone application. The input is the detected health risk, and the output is a notification message.

[0380] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0381] "Example 1"

[0382] One embodiment of the present invention is a system that incorporates an emotion engine. In this system, a conversational AI acquires vocal cord data from a conversation with an elderly person. The acquired vocal cord data is sent to both a disease diagnosis AI model and an emotion engine. The disease diagnosis AI model detects signs of illness from vocal cord analysis, and the emotion engine recognizes the elderly person's emotions from the vocal cord data. For example, if the elderly person expresses negative emotions such as anger or sadness, the emotion engine feeds that information back to the conversational AI. The conversational AI then adjusts the conversation with the elderly person based on that feedback.

[0383] "Example 2"

[0384] Another embodiment of the present invention is a system that notifies medical institutions, the nursing care industry, and security companies when emotions recognized by the emotion engine exceed a certain threshold. For example, if the emotion engine determines that an elderly person is experiencing strong stress or deep sadness, that information is notified to medical institutions, the nursing care industry, and security companies. This enables early response to the elderly person's psychological health condition.

[0385] "Example 3"

[0386] Another embodiment of the present invention is a system in which an emotion engine and a disease diagnosis AI model work in conjunction with each other. In this system, emotions recognized by the emotion engine are combined with signs of illness detected by the disease diagnosis AI model, enabling more accurate analysis of health conditions. For example, if the emotion engine determines that an elderly person is feeling anger or sadness, and the disease diagnosis AI model simultaneously detects signs of heart disease, this information is notified to medical institutions, the nursing care industry, and security companies. This enables early response to both the psychological and physical health conditions of the elderly.

[0387] The processing flow of each embodiment will be described below.

[0388] "Example 1"

[0389] Step 1: The conversational AI initiates a dialogue with the elderly person and acquires vocal cord data.

[0390] Step 2: The acquired vocal cord data is sent to both the disease diagnosis AI model and the emotion engine.

[0391] Step 3: The disease diagnosis AI model finds signs of disease from vocal cord analysis, and the emotion engine recognizes the elderly's emotions from vocal cord data.

[0392] Step 4: The emotion engine feeds the recognized emotions back to the conversational AI.

[0393] Step 5: The conversational AI uses the feedback to adjust the dialogue with the elderly.

[0394] "Example 2"

[0395] Step 1: The emotion engine recognizes the emotions of the elderly.

[0396] Step 2: If the emotion recognized by the emotion engine exceeds a certain threshold, the information is notified to medical institutions, the nursing care industry, and security companies.

[0397] Step 3: Medical institutions, the nursing care industry, and security companies will be notified and will take early action to address the psychological health of the elderly.

[0398] "Example 3"

[0399] Step 1: The conversational AI acquires vocal data from a conversation with the elderly person.

[0400] Step 2: The acquired vocal cord data is sent to both the disease diagnosis AI model and the emotion engine.

[0401] Step 3: The disease diagnosis AI model finds signs of disease from vocal cord analysis, and the emotion engine recognizes the elderly's emotions from vocal cord data.

[0402] Step 4: The emotions recognized by the emotion engine are combined with the disease symptoms detected by the disease diagnosis AI model to perform a more accurate analysis of the health status.

[0403] Step 5: If the analysis results exceed a certain threshold, the information is notified to medical institutions, the nursing care industry, and security companies.

[0404] Step 6: Medical institutions, caregiving industries, and security companies are notified and take early action to address both the psychological and physical health of the elderly.

[0405] Example 1

[0406] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0407] There is a need for 24-hour monitoring of the health status of the elderly and early detection of signs of illness. However, existing systems do not adequately collect and analyze voice data, and do not adjust dialogue in response to changes in emotional state. This makes it difficult to accurately grasp the health status of the elderly and provide appropriate treatment.

[0408] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0409] In this invention, the server includes a means for a conversational AI to acquire vocal cord data from a conversation with the elderly, a means for a disease diagnosis AI model to detect signs of illness from vocal cord analysis, a means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, a means for an emotion engine to recognize the emotions of the elderly from the vocal cord data, and a means for the conversational AI to adjust the conversation with the elderly based on feedback from the emotion engine. This makes it possible to monitor the health status of the elderly 24 hours a day and detect signs of illness early.

[0410] "Conversational AI" is an artificial intelligence system that collects voice data through dialogue with elderly people and uses natural language processing technology to hold conversations.

[0411] "Vocal cord data" refers to the voice signals generated by the elderly person's vocalizations, and is data that can be analyzed to evaluate their health and emotional state.

[0412] The "disease diagnosis AI model" is a machine learning model that analyzes voice data and detects signs of disease.

[0413] The "Emotion Engine" is an artificial intelligence system that recognizes and analyzes the emotional state of elderly people from voice data.

[0414] "Feedback" refers to the process by which the emotion engine returns information about the emotional state analyzed to the conversational AI.

[0415] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0416] The "caregiving industry" refers to the entire industry that provides life support and care services for the elderly and disabled.

[0417] A "security company" is a company that provides security services to protect the safety of individuals and facilities.

[0418] "Generative AI" is an artificial intelligence system that has the ability to generate new information and data from given data.

[0419] MODE FOR CARRYING OUT THE INVENTION

[0420] This invention is a system that combines conversational AI and a disease diagnosis AI model, and aims to monitor the health status of elderly people 24 hours a day and detect early signs of illness. A specific embodiment of this system is described below.

[0421] server

[0422] The server plays a central role in running the conversational AI, disease diagnosis AI models, and emotion engines. The server uses the following hardware and software:

[0423] Hardware: powerful processor, sufficient memory, storage devices, and network interfaces

[0424] Software: Conversational AI software, disease diagnosis AI models, emotion engines, database management systems

[0425] The server receives the voice data sent from the device and sends it to the disease diagnosis AI model and emotion engine for analysis. The analysis results are then notified to medical institutions, the nursing care industry, and security companies.

[0426] Terminal

[0427] The terminal is a device for collecting voice data through dialogue with the elderly. The terminal uses the following hardware and software:

[0428] Hardware: Microphone, Speaker, Network Interface

[0429] Software: Conversational AI applications, speech recognition engine

[0430] The device collects voice data in real time through conversations with the elderly person and sends it to the server. For example, if the conversational AI asks, "How was your day today?" and the elderly person replies, "I went for a walk today," that voice data is collected.

[0431] User

[0432] The user uses conversational AI to interact with the elderly. The user performs the following operations:

[0433] Questions are input into the conversational AI, which then asks the elderly people questions via voice.

[0434] Check the collected audio data and understand the content.

[0435] The analysis results sent from the server are checked and, if necessary, medical institutions, the nursing care industry, and security companies are contacted.

[0436] Examples and prompts

[0437] For example, a conversational AI might ask an elderly person, "How was your day today?" If the elderly person replies, "I went for a walk today," the device collects the voice data. The collected data is sent to a server, where a disease diagnosis AI model analyzes the data. If the analysis reveals an abnormality in the vocal cords, the server sends a notification to a medical institution.

[0438] Example prompt sentence:

[0439] "Collect vocal data through conversations with seniors and analyze it with an AI model for disease diagnosis. For example, ask questions like: 'How was your day?'"

[0440] By inputting this prompt sentence into the generative AI model, the conversational AI collects vocal cord data through natural conversation with the elderly, which is then analyzed by the disease diagnosis AI model.

[0441] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0442] Program processing flow

[0443] Step 1: Launching Conversational AI

[0444] The server launches the conversational AI and begins a conversation session with the elderly person.

[0445] Input: Conversational AI startup command

[0446] Output: An instance of the conversational AI

[0447] Specific operation: The server creates an instance of the conversational AI and starts the speech recognition engine.

[0448] Step 2: Collecting audio data

[0449] The device collects voice data through conversations with the elderly.

[0450] Input: Elderly person's voice

[0451] Output: Collected audio data

[0452] Specific operation: The device collects voice data in real time through the microphone and transmits it to the server.

[0453] Step 3: Sending audio data

[0454] The device sends the collected voice data to the server.

[0455] Input: Collected audio data

[0456] Output: Audio data sent to the server

[0457] Specific operation: The device compresses the collected audio data and sends it to the server using a secure communication protocol.

[0458] Step 4: Analyzing the audio data

[0459] The server sends the received voice data to a disease diagnosis AI model for analysis.

[0460] Input: Audio data sent to the server

[0461] Output: Disease diagnosis result

[0462] How it works: The server preprocesses the audio data and inputs it into a disease diagnosis AI model, which then detects signs of illness from the audio data.

[0463] Step 5: Emotion analysis using the emotion engine

[0464] The server also sends the voice data to the emotion engine for emotion analysis.

[0465] Input: Audio data sent to the server

[0466] Output: Emotion analysis results

[0467] Specific operation: The server inputs the voice data into the emotion engine and analyzes the elderly person's emotional state, for example, detecting emotions such as anger or sadness.

[0468] Step 6: Notification of results

[0469] The server notifies medical institutions, the nursing care industry, and security companies of the analysis results.

[0470] Input: Disease diagnosis results, emotion analysis results

[0471] Output: Notification message

[0472] Specific operation: The server generates a notification message based on the analysis results and sends it to the specified contact.

[0473] Step 7: Adjust the conversation

[0474] The server adjusts the dialogue content of the conversational AI based on feedback from the emotion engine.

[0475] Input: Sentiment analysis results

[0476] Output: Adjusted dialogue

[0477] Specific operation: The server receives feedback from the emotion engine and adjusts the dialogue content of the conversational AI. For example, if an elderly person shows negative emotions, the dialogue content will be adjusted to be gentler.

[0478] (Application example 1)

[0479] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0480] It is important to monitor the health status of elderly people in real time and detect early signs of illness. However, conventional systems have the problem of potentially overlooking signs of illness and making it difficult to properly grasp the emotional state of elderly people. Furthermore, there has been a lack of efficient means for nursing home staff to simultaneously monitor the health status and emotions of elderly people.

[0481] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for an interactive AI to acquire vocal cord data from a conversation with the elderly person, means for an illness diagnostic AI model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for an emotion engine to recognize the elderly person's emotions from the vocal cord data, means for the interactive AI to adjust the conversation with the elderly person based on the recognized emotions, and means for monitoring the elderly person's health condition and emotions in real time via an application installed in the smart glasses. This makes it possible to simultaneously monitor the elderly person's health condition and emotions, detect signs of illness early, and take appropriate measures.

[0482] "Conversational AI" is an artificial intelligence system that acquires voice data through dialogue with elderly people and uses natural language processing technology to hold conversations.

[0483] "Vocal cord data" refers to the voice signals generated by the elderly person's vocalizations, and is data that can be analyzed to evaluate their health condition and emotions.

[0484] The "disease diagnosis AI model" is a machine learning model that analyzes voice data and detects signs of disease.

[0485] The "emotion engine" is a system that recognizes the emotions of elderly people from voice data and adjusts dialogue based on that information.

[0486] "Smart glasses" are wearable devices that have built-in displays, microphones, and sensors and can run applications.

[0487] "Real-time monitoring" is the process of instantly acquiring and analyzing data and immediately reflecting the results.

[0488] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[0489] The "care industry" is an industry that provides life support and care services for the elderly and disabled.

[0490] A "security company" is a company that provides security services to protect the safety of facilities and individuals.

[0491] "Notification" is the process by which the system communicates detected information to specific recipients.

[0492] The following system configuration will be described as an embodiment of the present invention.

[0493] System Configuration

[0494] The system includes a conversational AI, a disease diagnosis AI model, an emotion engine, smart glasses, and a notification system. These components work together via a server and a device (smart glasses) to monitor the health and emotions of elderly people in real time.

[0495] Hardware and software used

[0496] Hardware:

[0497] Smart glasses (e.g., Google Glass (registered trademark))

[0498] microphone

[0499] software:

[0500] speech_recognition library: Used to convert speech data into text.

[0501] emotion_recognition library: Used to recognize emotions from text.

[0502] disease_diagnosis library: Used to diagnose disease symptoms from text.

[0503] notification_system library: Used to notify results.

[0504] Data processing and calculation

[0505] The server captures the conversation with the elderly person using the microphone built into the smart glasses. The captured voice data is converted into text using the speech_recognition library. Next, the text data is used to recognize emotions using the emotion_recognition library and to diagnose signs of illness using the disease_diagnosis library. Finally, the server notifies the care staff of the recognized emotions and signs of illness using the notification_system library.

[0506] Specific examples

[0507] For example, if an elderly person says, "I've been having chest pains lately," the disease diagnosis AI model will detect signs of heart disease, and the emotion engine will recognize anxiety. This will notify the caregiver, "You are showing signs of heart disease. You seem to be feeling anxious."

[0508] Prompt Sentence Examples

[0509] If an elderly person says, "I've been having chest pains lately," the disease diagnosis AI model will detect signs of heart disease, and the emotion engine will recognize anxiety. This will notify the caregiver, "You are showing signs of heart disease. You seem to be feeling anxious."

[0510] In this way, the system can simultaneously monitor the health and emotions of elderly people, detect early signs of illness, and take appropriate measures.

[0511] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0512] Step 1:

[0513] The server uses the microphone built into the smart glasses to capture the conversational voice of the elderly. The input is the elderly's voice data, and the output is a voice data file. This voice data is collected in real time by the conversational AI.

[0514] Step 2:

[0515] The server converts the acquired audio data into text using the speech_recognition library. The input is an audio data file, and the output is text data. This conversion process involves analyzing the audio signal and converting it into the corresponding string.

[0516] Step 3:

[0517] The server recognizes emotions in the converted text data using the emotion_recognition library. The input is text data, and the output is recognized emotional information. In this process, the content of the text is analyzed, and the emotion engine identifies the emotional state of the elderly person.

[0518] Step 4:

[0519] The server then uses the disease_diagnosis library to diagnose disease symptoms from the same text data. The input is text data, and the output is disease symptom information. In this process, the content of the text is analyzed and the disease diagnosis AI model detects disease symptoms.

[0520] Step 5:

[0521] The server uses the notification_system library to notify care staff of the recognized emotion information and symptom information. The input is emotion information and symptom information, and the output is a notification message. This notification is performed in real time via the smart glasses display or other notification means.

[0522] Step 6:

[0523] The terminal (smart glasses) displays notification messages to care staff. The input is the notification message, and the output is the display on the screen. This allows care staff to instantly grasp the health and emotional state of the elderly person.

[0524] In this way, the server and device work together to monitor the health and emotions of elderly people in real time, making it possible to take appropriate action early on.

[0525] Example 2

[0526] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0527] There is a need to acquire voice data through conversations with elderly people and use that data to detect early signs of illness and changes in their emotions. However, conventional systems do not adequately acquire and analyze voice data, making it difficult to properly grasp the health and psychological state of elderly people. Another challenge is recognizing changes in emotions in real time and providing the necessary notifications promptly.

[0528] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0529] In this invention, the server includes: a conversational AI acquiring voice data from a conversation with the elderly person; a voice recognition engine converting the voice data into text; a conversational AI model generating a response from the text data; a voice synthesis engine converting the generated text response into voice; an emotion recognition engine analyzing the elderly person's emotions; a notification triggering means when the emotion recognized by the emotion recognition engine exceeds a certain threshold; and a notification system sending notifications to medical institutions, the nursing care industry, and security companies. This makes it possible to grasp the elderly person's health and psychological state in real time and take necessary measures promptly.

[0530] "Conversational AI" is an artificial intelligence system designed to engage in natural dialogue with the elderly.

[0531] "Voice data" is a digital record of the voice spoken by the elderly person to the conversational AI.

[0532] A "voice recognition engine" is software or hardware for converting voice data into text data.

[0533] "Text data" is character information converted by a voice recognition engine.

[0534] An "interactive AI model" is a machine learning model that generates appropriate responses based on text data.

[0535] A "speech synthesis engine" is software or hardware for converting text data into speech.

[0536] An "emotion recognition engine" is software or hardware that analyzes and recognizes emotions from elderly people's voice data.

[0537] The "threshold" is a reference value set for the emotion recognition engine to determine changes in emotion.

[0538] The "notification system" is a system that sends notifications to medical institutions, the nursing care industry, and security companies when the emotion recognition engine exceeds a threshold.

[0539] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0540] The "care industry" is an industry that provides care services to the elderly and disabled.

[0541] A "security company" is a company that provides security services to provide safety and security.

[0542] MODE FOR CARRYING OUT THE INVENTION

[0543] This invention is a system that acquires voice data through conversations with elderly people and uses that data to detect early signs of illness and changes in emotions. The system includes a conversational AI, a voice recognition engine, a conversational AI model, a voice synthesis engine, an emotion recognition engine, and a notification system.

[0544] server

[0545] The server hosts the conversational AI and provides a platform for dialogue with the elderly. The conversational AI is pre-populated with data on medical knowledge and topics of interest to the elderly. The server loads the AI ​​model using machine learning frameworks such as TensorFlow and PyTorch.

[0546] The server receives the voice data sent from the device and converts it to text using the Google Cloud Speech-to-Text API. The converted text is then fed into a generative AI model (e.g., GPT-4) to generate an appropriate response. The generated text response is then converted to speech using Amazon Polly or the Google Cloud Text-to-Speech API.

[0547] Furthermore, the emotion recognition engine analyzes the elderly person's emotions in real time during the conversation. If the emotion recognized by the emotion recognition engine exceeds a certain threshold, the server notifies the information to medical institutions, the nursing care industry, and security companies. Notifications are sent via email and SMS API.

[0548] Terminal

[0549] The device provides an interface for the elderly to interact with the conversational AI. The device captures voice data using a microphone, sends the voice data to the server using an HTTP POST request, receives the response voice data from the server, and plays the voice data using a speaker.

[0550] User

[0551] The user (elderly person) starts the conversation by speaking into the device. For example, they might ask, "My knee has been hurting recently. What should I do?" The user receives the conversational AI's response, which is played back on the device.

[0552] Specific examples

[0553] Example 1: Interaction with conversational AI

[0554] The user (elderly person) uses a device to ask the conversational AI, "My knee has been hurting recently. What should I do?" The conversational AI uses its medical knowledge to respond, "For knee pain, it's important to rest first. Also, if the pain persists, consult a doctor."

[0555] Example 2: Notifications by Emotion Engine

[0556] If the emotion engine recognizes that the user (elderly person) is experiencing strong stress during a conversation, the server notifies the medical institution of this information. The medical institution then checks the elderly person's psychological health status and takes any necessary measures.

[0557] Prompt Sentence Examples

[0558] "Generate example responses for when an elderly person asks a conversational AI a medical question. For example, a question about knee pain."

[0559] "Please explain how the notification system works when the emotion engine recognizes that an elderly person is under a lot of stress."

[0560] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0561] Step 1:

[0562] Acquiring voice input

[0563] The device receives voice input from the user (elderly person). Specifically, the device uses a microphone to capture voice data. The input is the user's voice, and the output is digital voice data.

[0564] Step 2:

[0565] Sending audio data

[0566] The device sends the acquired voice data to the server. Specifically, the device sends the voice data to the server using an HTTP POST request. The input is the digital voice data, and the output is a transmission completion notification to the server.

[0567] Step 3:

[0568] Receiving audio data

[0569] The server receives the audio data sent from the terminal. Specifically, the server receives the audio data via an HTTP request. The input is the audio data sent from the terminal, and the output is the received audio data.

[0570] Step 4:

[0571] Converting audio data to text

[0572] The server uses a speech recognition engine to convert the received voice data into text. Specifically, the server calls the Google Cloud Speech-to-Text API to convert the voice data into text. The input is voice data, and the output is text data.

[0573] Step 5:

[0574] Response generation using conversational AI

[0575] The server inputs the converted text into a conversational AI model to generate an appropriate response. Specifically, the server inputs a prompt sentence into a generative AI model (e.g., GPT-4) to generate a response. The input is text data, and the output is the generated text response.

[0576] Step 6:

[0577] Response transcription

[0578] The server uses a speech synthesis engine to convert the generated text response into speech. Specifically, the server converts text to speech using Amazon Polly or the Google Cloud Text-to-Speech API. The input is the generated text response, and the output is the audio data.

[0579] Step 7:

[0580] Sending audio data

[0581] The server sends the generated audio data to the terminal. Specifically, the server returns the audio data to the terminal as an HTTP response. The input is the audio data, and the output is a transmission completion notification to the terminal.

[0582] Step 8:

[0583] Receiving a response voice

[0584] The terminal receives the response audio data sent from the server. Specifically, the terminal receives the HTTP response and decodes the audio data. The input is the audio data sent from the server, and the output is the decoded audio data.

[0585] Step 9:

[0586] Playing a response voice

[0587] The terminal plays the received response voice data. Specifically, the terminal plays the voice data using a speaker. The input is the decoded voice data, and the output is the played voice.

[0588] Step 10:

[0589] Emotion recognition

[0590] The emotion recognition engine analyzes the elderly person's emotions in real time during the conversation. Specifically, the emotion recognition engine analyzes the voice data and applies emotion recognition algorithms. The input is the voice data, and the output is the emotion analysis result.

[0591] Step 11:

[0592] Threshold determination

[0593] The emotion recognition engine determines whether the recognized emotion exceeds a preset threshold. Specifically, the emotion recognition engine quantifies the level of stress or sadness and compares it with the threshold. The input is the emotion analysis result, and the output is the threshold judgment result.

[0594] Step 12:

[0595] Notification triggers

[0596] If the emotion recognition engine determines that the threshold has been exceeded, it triggers a notification to the server. Specifically, the emotion recognition engine sends an HTTP request to the server to request a notification. The input is the threshold judgment result, and the output is a notification request.

[0597] Step 13:

[0598] Sending notifications

[0599] The server receives notification requests from the emotion recognition engine and sends notifications to medical institutions, the nursing care industry, and security companies. Specifically, the server sends notifications using email or SMS APIs. The input is the notification request, and the output is the sent notification.

[0600] (Application example 2)

[0601] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0602] There is a need for real-time monitoring of the physical and psychological well-being of elderly people, early detection of abnormalities, and appropriate response. However, existing systems lack the functionality to recognize the emotional state of elderly people through natural dialogue and to promptly notify them when necessary, making it difficult to respond early.

[0603] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0604] In this invention, the server includes a means for an interactive AI to acquire voice data from a conversation with the elderly, a means for an AI disease diagnosis model to detect signs of illness from voice analysis, a means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, and a means for an emotion engine to recognize the emotional state of the elderly and notify them when a certain threshold is exceeded. This makes it possible to monitor the physical and psychological health of the elderly in real time, detect abnormalities early, and take appropriate action.

[0605] "Conversational artificial intelligence" is an artificial intelligence technology that allows natural dialogue with elderly people and acquires voice data.

[0606] "Voice data" refers to voice information acquired through conversation with the elderly.

[0607] The "disease diagnosis artificial intelligence model" is an artificial intelligence model that analyzes voice data and detects signs of disease.

[0608] "Speech analysis" is the process of analyzing captured speech data to detect specific patterns and anomalies.

[0609] "Disease symptoms" are abnormal health conditions or early symptoms of disease that can be detected through voice analysis.

[0610] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0611] The "care industry" is an industry that provides life support and care services for the elderly and disabled.

[0612] A "security company" is a company that provides security services and responds to emergencies.

[0613] "Notification" is an action to inform interested parties of specific information.

[0614] The "Emotion Engine" is a technology for recognizing and analyzing the emotional state of the elderly.

[0615] "Emotional state" refers to the type and intensity of emotions experienced by the elderly.

[0616] The "threshold" is a reference value for issuing a notification when the emotional state exceeds a certain level.

[0617] The system for implementing this invention includes a conversational artificial intelligence, a disease diagnosis artificial intelligence model, an emotion engine, and a notification system. The details of each component and their operation are described below.

[0618] Conversational Artificial Intelligence

[0619] Conversational AI is designed to have natural conversations with the elderly. It uses speech recognition technology to convert the elderly's speech into text data, and then analyzes the text data to generate appropriate responses. The hardware used includes smartphones, smart glasses, and head-mounted displays. The software used is Python and the speech_recognition library.

[0620] Disease diagnosis artificial intelligence model

[0621] The AI ​​disease diagnosis model is used to analyze voice data acquired by conversational AI and find signs of illness. This model detects specific patterns and abnormalities in the voice data and identifies early symptoms of illness. The software used is the emotion_recognition library.

[0622] Emotion Engine

[0623] The emotion engine is used to recognize and analyze the emotional state of elderly people in real time. The emotion engine calculates an emotion score from voice data and dialogue content, and notifies users when the score exceeds a certain threshold. The emotion engine is implemented using Python and includes an emotion recognition algorithm.

[0624] Notification System

[0625] The notification system notifies medical institutions, the nursing care industry, and security companies when the disease diagnosis AI model or emotion engine detects an abnormality. Notifications are sent via email, SMS, and a dedicated application. The software used is the notification_service module.

[0626] Specific examples

[0627] For example, if an elderly person says, "I'm not feeling well today," the conversational AI converts that statement into text data, which the disease diagnosis AI model analyzes to detect signs of illness. At the same time, the emotion engine analyzes the elderly person's emotional state, and if they are experiencing high levels of stress or sadness, the notification system will notify medical institutions, the nursing care industry, and security companies.

[0628] Prompt Sentence Examples

[0629] Develop an application that interacts with elderly people and monitors their emotional state using an emotion engine. Include a function to notify medical institutions, the nursing care industry, and security companies if the emotion exceeds a certain threshold.

[0630] In this way, it is possible to monitor the physical and psychological well-being of elderly people in real time, detect abnormalities early, and take appropriate measures.

[0631] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0632] Step 1:

[0633] The user (elderly person) speaks into the device (smartphone, smart glasses, head-mounted display).

[0634] Input: Elderly person's voice

[0635] Output: Audio data

[0636] Specific operation: The device's microphone captures the elderly person's voice and saves it as audio data.

[0637] Step 2:

[0638] The terminal uses voice recognition technology to convert the voice data into text data.

[0639] Input: Audio data

[0640] Output: Text data

[0641] Specific operation: The speech_recognition library on the device analyzes the audio data and generates corresponding text data.

[0642] Step 3:

[0643] The server receives the text data and the conversational artificial intelligence generates an appropriate response.

[0644] Input: Text data

[0645] Output: Response text

[0646] Specific operation: Conversational artificial intelligence on the server analyzes the text data and generates appropriate responses for the elderly.

[0647] Step 4:

[0648] The response text generated by the server is converted into voice data using voice synthesis technology and sent to the terminal.

[0649] Input: Response text

[0650] Output: Response voice data

[0651] Specific operation: The speech synthesis engine on the server converts the response text into voice data and sends it to the terminal.

[0652] Step 5:

[0653] The terminal plays back the response voice data and provides a response to the elderly person.

[0654] Input: Response voice data

[0655] Output: Played audio

[0656] Specific operation: The device's speaker plays the response voice data and lets the elderly person listen.

[0657] Step 6:

[0658] The server sends the text data to an artificial intelligence disease diagnosis model, which analyzes the symptoms of the disease.

[0659] Input: Text data

[0660] Output: Signs of illness

[0661] Specific operation: A disease diagnosis artificial intelligence model on the server analyzes text data and detects signs of disease.

[0662] Step 7:

[0663] The server sends the text data to an emotion engine to analyze the emotional state.

[0664] Input: Text data

[0665] Output: Sentiment score

[0666] Specific operation: The emotion engine on the server analyzes the text data and calculates an emotion score.

[0667] Step 8:

[0668] The server compares the emotion score with a threshold and notifies you if necessary.

[0669] Input: Sentiment score

[0670] Output: Notification message

[0671] Specific operation: The server compares the emotion score with a threshold, and if the threshold is exceeded, it uses the notification_service module to send a notification to medical institutions, the nursing care industry, and security companies.

[0672] In this way, the system can monitor the physical and psychological well-being of elderly people in real time, detect abnormalities early, and take appropriate action.

[0673] Example 3

[0674] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0675] There is a need to detect the health condition of elderly people early and provide appropriate treatment promptly. However, conventional systems have difficulty accurately detecting signs of illness from voice data and are unable to simultaneously evaluate psychological health conditions. Therefore, a system that can comprehensively evaluate physical and psychological health conditions and provide appropriate treatment is needed.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0677] In this invention, the server includes a means for an interactive AI to acquire voice data from a conversation with the elderly, a means for a disease diagnostic AI model to find signs of illness from voice analysis, a means for a generative AI to learn patterns in the voice data and generate signs of illness from those patterns, a means for an emotion recognition engine to recognize emotions from the voice data, a means for combining the recognized emotions and signs of illness to generate analysis results, and a means for notifying medical institutions, the nursing care industry, and security companies of the generated analysis results. This enables a comprehensive evaluation of the physical and psychological health of the elderly and enables early intervention.

[0678] "Conversational artificial intelligence" is artificial intelligence that acquires voice data through dialogue with elderly people and engages in dialogue.

[0679] "Voice data" is digital data that records the voice uttered by the user.

[0680] The "disease diagnosis artificial intelligence model" is an artificial intelligence model that analyzes voice data and detects signs of disease.

[0681] "Generative AI" is an AI that learns patterns in voice data and generates symptoms of disease from those patterns.

[0682] An "emotion recognition engine" is an engine for recognizing a user's emotions from voice data.

[0683] The "analysis results" are the result of combining data generated by the disease diagnosis artificial intelligence model and the emotion recognition engine.

[0684] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0685] The "care industry" is an industry that provides care services to the elderly and disabled.

[0686] A "security company" is a company that provides services to protect the safety of facilities and individuals.

[0687] The present invention is a system for comprehensively evaluating the health status of elderly people and enabling early intervention. A specific embodiment of this system will be described below.

[0688] First, when the user speaks into the device, the device uses its built-in microphone to collect voice data, which is a digital recording of the voice coming from the user's vocal cords. The device then transmits the collected voice data to a server.

[0689] The server preprocesses the received audio data, specifically using an audio processing library (e.g., Librosa) to remove noise and normalize the audio. This preprocessing improves the quality of the audio data and ensures accurate subsequent analysis.

[0690] The server then inputs the preprocessed speech data into a generative AI (e.g., GPT-4), which learns patterns associated with disease symptoms from past data and generates new patterns. These patterns are then input into a disease diagnosis AI model.

[0691] A disease diagnosis AI model (e.g., a model using TensorFlow) detects signs of illness from voice data based on patterns generated by generative AI. The model analyzes the features of the voice data and identifies signs associated with specific diseases.

[0692] Furthermore, the server uses an emotion recognition engine (e.g., EmotionAPI) to recognize the user's emotions from the voice data. The emotion recognition engine analyzes characteristics of the voice data, such as the tone and rhythm, to identify the emotions the user is feeling.

[0693] The server combines the symptoms of illness detected by the disease diagnosis AI model with the emotions recognized by the emotion recognition engine to generate a comprehensive analysis result, which includes information about the user's physical and psychological health status.

[0694] Finally, the server notifies medical institutions, the nursing care industry, and security companies of the results of the analysis, allowing relevant parties to quickly understand the user's health condition and take appropriate action.

[0695] Examples:

[0696] When an elderly person says to the device, "I've been having a lot of chest pain lately," the device records the voice and sends it to the server. The server preprocesses the voice data, and the generative AI learns the patterns. The disease diagnosis AI model detects signs of heart disease, and the emotion recognition engine recognizes that the elderly person is feeling sad. This information is then notified to a medical institution, stating, "The user has signs of heart disease and is feeling sad."

[0697] Example prompt sentence:

[0698] "Please explain the process by which the generative AI learns patterns using the vocal cord data of an elderly person as input, and the disease diagnosis AI model detects signs of heart disease. Please also include how the emotion recognition engine recognizes the emotions of the elderly person and notifies the medical institution of the results."

[0699] In this way, how the server, terminal, and user cooperate to process data and detect signs of illness will be specifically described. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0700] Step 1:

[0701] Audio data collection

[0702] When a user speaks into the terminal, the terminal collects voice data using its built-in microphone.

[0703] Input: User's voice

[0704] Output: Recorded audio data

[0705] What it does: When a user says, "Hello, I'm not feeling too well today," the device records the audio and saves it as digital audio data.

[0706] Step 2:

[0707] Audio data preprocessing

[0708] The server receives the audio data sent from the device and performs noise reduction and audio normalization using an audio processing library (e.g., Librosa).

[0709] Input: Recorded audio data

[0710] Output: Preprocessed audio data

[0711] Specific behavior: The server removes background noise from the recorded audio data and keeps the volume constant.

[0712] Step 3:

[0713] Pattern learning using generative artificial intelligence

[0714] The server then feeds the preprocessed audio data into a generative artificial intelligence (e.g., GPT-4) to learn patterns associated with disease symptoms.

[0715] Input: Preprocessed audio data

[0716] Output: Learned patterns

[0717] Specific operation: The server instructs the generative AI to "learn patterns to detect signs of heart disease from this voice data."

[0718] Step 4:

[0719] Disease diagnosis using artificial intelligence model

[0720] The server inputs the patterns generated by the generative AI into a disease diagnosis AI model (e.g., a model using TensorFlow) to detect signs of disease.

[0721] Input: Learned pattern

[0722] Output: Symptoms of illness

[0723] Specific operation: The server instructs the disease diagnosis AI model to "detect signs of heart disease from this pattern," and the model responds with "Signs of heart disease have been found."

[0724] Step 5:

[0725] Emotion recognition using an emotion recognition engine

[0726] The server uses an emotion recognition engine (e.g., EmotionAPI) to recognize the user's emotion from the voice data.

[0727] Input: Audio data

[0728] Output: Recognized emotion

[0729] Specific operation: The server instructs the emotion recognition engine to "recognize the user's emotion from this voice data," and the emotion recognition engine responds with "The user is feeling sad."

[0730] Step 6:

[0731] Generate analysis results

[0732] The server combines the symptoms of disease detected by the disease diagnosis AI model with the emotions recognized by the emotion recognition engine to generate comprehensive analysis results.

[0733] Input: Symptoms of illness, perceived emotions

[0734] Output: Comprehensive analysis results

[0735] Specific behavior: The server combines the symptoms of illness and emotions to generate an analysis result such as "The user has symptoms of heart disease and feels sad."

[0736] Step 7:

[0737] Notification of results

[0738] The server notifies medical institutions, the nursing care industry, and security companies of the analysis results generated.

[0739] Input: Comprehensive analysis results

[0740] Output: Notified information

[0741] Specific behavior: The server notifies the medical institution that "The user has symptoms of heart disease and is feeling sad."

[0742] (Application example 3)

[0743] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0744] It is important to monitor the health status of the elderly in real time and detect signs of illness early. However, conventional systems have low accuracy in detecting signs of illness from vocal cord data and do not take emotional state into account, making it difficult to grasp a comprehensive health condition. In addition, there is a lack of a mechanism to immediately display detected information and notify appropriate authorities if necessary.

[0745] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for an interactive AI to acquire vocal cord data from a conversation with the elderly person, means for a disease diagnostic AI model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for an emotion engine to recognize emotions from the vocal cord data, means for integrating and analyzing the emotions recognized by the emotion engine and the signs of illness detected by the disease diagnostic AI model, means for displaying the integrated analysis results, and means for notifying security companies and medical institutions as necessary. This makes it possible to comprehensively grasp the health condition of the elderly person and take appropriate measures early on.

[0746] "Conversational AI" is an artificial intelligence system that collects voice data through dialogue with elderly people and communicates with them using natural language processing technology.

[0747] "Vocal cord data" refers to data that is a digital recording of the sound signals emitted from the human vocal cords.

[0748] The "disease diagnosis AI model" is an artificial intelligence model designed to analyze vocal cord data and detect signs of specific diseases.

[0749] An "emotion engine" is an artificial intelligence system for recognizing a speaker's emotional state from voice data.

[0750] "Generative AI" is an artificial intelligence system that has the ability to generate new data from existing data.

[0751] The "means of integration and analysis" is a means for making a comprehensive assessment by combining the emotions recognized by the emotion engine with the signs of disease detected by the disease diagnosis AI model.

[0752] "Display means" refers to a device or interface for visually presenting the analysis results to the user.

[0753] "Means of notification" refers to a system for quickly transmitting detected important information to the appropriate agencies and personnel.

[0754] The system for implementing this invention includes an interactive AI, a disease diagnosis AI model, an emotion engine, a generative AI, a display means, and a notification means. Specific embodiments of each means will be described below.

[0755] Conversational AI

[0756] The server uses conversational AI to converse with the elderly. The conversational AI uses natural language processing technology to understand what the elderly say and generate appropriate responses. This allows vocal cord data to be obtained from the elderly.

[0757] Acquiring vocal cord data

[0758] The device (e.g., smart glasses or smartphone) transmits the vocal cord data collected through conversational AI to a server, where it is digitally recorded and used for subsequent analysis.

[0759] Disease diagnosis AI model

[0760] The server analyzes the acquired vocal cord data using a disease diagnosis AI model designed to detect signs of specific diseases from the vocal cord data. This model uses generative AI to learn patterns in the vocal cord data and make more accurate diagnoses.

[0761] Emotion Engine

[0762] The server recognizes emotions from the vocal data using an emotion engine, which analyzes the characteristics of the voice data and identifies the speaker's emotional state (e.g., anger, sadness, joy, etc.).

[0763] A means of integration and analysis

[0764] The server integrates and analyzes the emotions recognized by the emotion engine and the signs of illness detected by the disease diagnosis AI model. This integrated analysis makes it possible to assess the overall health status of the elderly person.

[0765] Display method

[0766] The device visually presents the integrated analysis results to users, for example by displaying the results on the display of smart glasses, allowing security guards or caregivers to check the information in real time.

[0767] Notification method

[0768] The server notifies security companies and medical institutions as necessary to quickly communicate important information detected (such as signs of heart disease or strong feelings of anger).

[0769] Specific examples

[0770] For example, if a security guard is patrolling a shopping mall and detects that someone is showing signs of anger while also showing signs of heart disease, the information will be instantly displayed on the guard's smart glasses, allowing the guard to quickly respond and notify medical authorities if necessary.

[0771] Prompt Sentence Examples

[0772] "Develop a system in which security guards on patrol in shopping malls can analyze the vocal data of people around them in real time through smart glasses to detect emotions and signs of illness. Include a function that uses an emotion engine and an AI disease diagnosis model to instantly display detected information and notify security companies or medical institutions as necessary."

[0773] The above is a specific embodiment for carrying out the present invention.

[0774] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0775] Step 1:

[0776] The server uses conversational AI to converse with the elderly. The conversational AI uses natural language processing technology to understand what the elderly says and generate appropriate responses. It receives the elderly's voice data as input and generates dialogue content as output. This dialogue content is recorded as voice data.

[0777] Step 2:

[0778] The device transmits the vocal cord data collected through the conversational AI to a server. It receives the voice data obtained from the conversational AI as input and generates digital vocal cord data to send to the server as output.

[0779] Step 3:

[0780] The server analyzes the acquired vocal cord data using a disease diagnosis AI model. It receives the vocal cord data as input and generates a diagnosis result indicating signs of disease as output. The disease diagnosis AI model uses generative AI to learn patterns in the vocal cord data and make more accurate diagnoses.

[0781] Step 4:

[0782] The server recognizes emotions from the vocal data using an emotion engine. It receives the vocal data as input and generates an output indicating an emotional state (e.g., anger, sadness, joy, etc.). The emotion engine analyzes the features of the voice data to identify the speaker's emotional state.

[0783] Step 5:

[0784] The server integrates and analyzes the emotions recognized by the emotion engine and the signs of illness detected by the disease diagnosis AI model. It receives emotional states and signs of illness as input and generates integrated analysis results as output. This integrated analysis makes it possible to evaluate the overall health status of elderly people.

[0785] Step 6:

[0786] The terminal visually presents the integrated analysis results to the user. It receives the integrated analysis results as input and generates information to be displayed on a display as output. For example, by displaying the results on the display of smart glasses, security guards or caregivers can check the information in real time.

[0787] Step 7:

[0788] The server notifies security companies or medical institutions as needed. It receives the integrated analysis results as input and generates notification messages as output. Notifications are used to quickly communicate important detected information (e.g., signs of heart disease or strong feelings of anger).

[0789] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0790] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0791] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0792] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0793] [Second embodiment]

[0794] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0795] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0796] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0797] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0798] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0799] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0800] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0801] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0802] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0803] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0804] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0805] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0806] "Example 1"

[0807] The present invention is a system that combines a conversational AI and a disease diagnosis AI model. The conversational AI acquires vocal cord data through dialogue with the elderly. This dialogue is conducted, for example, through everyday conversations or health-related questions. The acquired vocal cord data is sent to the disease diagnosis AI model for analysis. The disease diagnosis AI model detects signs of illness from the vocal cord data. The detected signs of illness are notified to medical institutions, the nursing care industry, and security companies. This makes it possible to check the health status of the elderly 24 hours a day and detect signs of illness early.

[0808] "Example 2"

[0809] One aspect of the present invention is a system that uses the vast knowledge of a conversational AI to engage in conversations with the elderly. The conversational AI possesses a wide range of knowledge, such as medical knowledge and topics that may be of interest to the elderly. This makes it possible to acquire vocal cord data through natural conversations with the elderly.

[0810] "Example 3"

[0811] One aspect of the present invention is a system in which a disease diagnosis AI model uses generative AI to detect signs of disease from vocal cord analysis. The generative AI learns patterns in vocal cord data and can generate signs of disease from those patterns. This enables the disease diagnosis AI model to more accurately detect signs of disease from vocal cord data.

[0812] The processing flow of each embodiment will be described below.

[0813] "Example 1"

[0814] Step 1: The conversational AI initiates a dialogue with the elderly person through questions such as, "How are you feeling today?" and "Has anything changed recently?"

[0815] Step 2: The conversational AI acquires the elderly person's vocal data, including the quality, tone, and volume of the elderly person's voice.

[0816] Step 3: The acquired vocal cord data is sent to a disease diagnosis AI model, which then identifies signs of disease from the vocal cord data.

[0817] Step 4: Any signs of illness detected by the disease diagnosis AI model are notified to medical institutions, the nursing care industry, and security companies, making it possible to detect signs of illness early and take appropriate measures.

[0818] "Example 2"

[0819] Step 1: The conversational AI initiates a dialogue with the senior through questions about topics that may be of interest to the senior, such as "What book have you read recently?" or "What is your favorite TV show?"

[0820] Step 2: The conversational AI acquires the elderly person's vocal data, including the quality, tone, and volume of the elderly person's voice.

[0821] Step 3: The acquired vocal cord data is sent to a disease diagnosis AI model, which then identifies signs of disease from the vocal cord data.

[0822] "Example 3"

[0823] Step 1: The conversational AI initiates a dialogue with the elderly person and acquires vocal cord data.

[0824] Step 2: The acquired vocal cord data is sent to a disease diagnosis AI model. The disease diagnosis AI model uses generative AI to find signs of disease from the vocal cord data. The generative AI learns patterns in the vocal cord data and generates signs of disease from those patterns.

[0825] Step 3: Any signs of illness detected by generative AI are reported to medical institutions, the nursing care industry, and security companies. This allows for early detection of signs of illness and the appropriate response.

[0826] Example 1

[0827] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0828] There is a need to check the health status of the elderly 24 hours a day and detect early signs of illness. However, conventional systems do not adequately collect and analyze voice data, which can result in signs of illness being overlooked. In addition, there is an insufficient mechanism for appropriately notifying medical institutions, the nursing care industry, and security companies of the collected data. This creates the problem of ineffective health management for the elderly.

[0829] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0830] In this invention, the server includes a means for the conversational AI to acquire voice data from a conversation with the elderly person, a means for the terminal to transmit the acquired voice data to the server, a means for the server to input the voice data into a disease diagnosis AI model, a means for the disease diagnosis AI model to detect signs of illness from the voice data, and a means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness. This makes it possible to check the health status of the elderly person 24 hours a day and detect signs of illness early.

[0831] "Conversational AI" is an artificial intelligence system that acquires voice data through dialogue with elderly people.

[0832] "Voice data" refers to voice information collected when elderly people speak to conversational AI.

[0833] A "terminal" is a device on which conversational AI is installed and which collects and transmits voice data to a server.

[0834] The "server" is a computer system that receives voice data sent from the terminal and inputs it into the disease diagnosis AI model.

[0835] The "disease diagnosis AI model" is an artificial intelligence model that analyzes voice data and detects signs of disease.

[0836] "Signs of illness" are abnormalities in health or early symptoms of illness detected from voice data.

[0837] A "medical institution" is an organization that receives symptoms of illness and provides appropriate medical treatment.

[0838] The "care industry" is an industry that manages the health of the elderly and provides necessary care services.

[0839] A "security company" is a company that ensures the safety of the elderly and responds quickly when necessary.

[0840] "Generative AI" is an artificial intelligence technology used by disease diagnosis AI models to find signs of disease from voice data.

[0841] MODE FOR CARRYING OUT THE INVENTION

[0842] This invention is a system that checks the health status of elderly people 24 hours a day and detects early signs of illness. This system is composed of a conversational AI, a terminal, a server, and a disease diagnosis AI model.

[0843] Conversational AI and Devices

[0844] The user (elderly person) speaks to a device (smartphone or dedicated device) on which the conversational AI is installed. The conversational AI acquires voice data through everyday conversations with the user and health-related questions. This conversational AI uses a general dialogue engine that uses natural language processing technology, for example.

[0845] Collection and transmission of voice data

[0846] The device collects voice data acquired through conversational AI in real time. The collected voice data is sent to the server using a secure communication protocol (e.g., HTTPS). The data is encrypted before transmission to protect the user's privacy.

[0847] Server and disease diagnosis AI model

[0848] The server receives the voice data sent from the device. The received voice data is input into a disease diagnosis AI model. This disease diagnosis AI model is built using machine learning frameworks such as TensorFlow and PyTorch. The server also performs preprocessing of the voice data (e.g., noise removal and voice normalization).

[0849] Disease symptom detection and notification

[0850] The AI ​​disease diagnosis model analyzes voice data to detect signs of illness. For example, it detects abnormalities such as voice tremors or changes in pitch and determines what kind of illness they are. The analysis results are then generated as a detailed report to assess the user's health status.

[0851] The server then sends the analysis results to medical institutions, the nursing care industry, and security companies via email, SMS, or a dedicated application, including any detected symptoms of illness and recommended measures.

[0852] Examples of specific examples and prompts

[0853] Examples:

[0854] The user speaks to the conversational AI, saying, "I've been coughing at night lately."

[0855] The device records this conversation and transmits the audio data to a server in real time.

[0856] The server inputs the received voice data into a disease diagnosis AI model to analyze cough patterns.

[0857] The server detects from the coughing pattern that it may be an early symptom of a cold or bronchitis.

[0858] The server notifies medical institutions of this information and encourages the user to seek medical attention early.

[0859] Example prompt sentence:

[0860] If an elderly person says to a conversational AI, "I've been coughing at night lately," please explain how the voice data is collected and analyzed by the disease diagnosis AI model.

[0861] This system makes it possible to check the health status of elderly people 24 hours a day and detect early signs of illness.

[0862] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0863] Program processing flow

[0864] Step 1: The user initiates a dialogue with the conversational AI

[0865] Users talk to a device with conversational AI installed, asking everyday conversational or health-related questions such as, "What's the weather like today?" or "I've had a bit of a sore throat lately."

[0866] Input: User's voice

[0867] Output: Acquiring voice data using conversational AI

[0868] Specific operation: When a user speaks into the device, the device's built-in microphone captures the voice, and the conversational AI recognizes the voice.

[0869] Step 2: The device collects audio data

[0870] The device records the user's voice through conversational AI, using the device's built-in microphone or an external microphone, and the recorded voice data is processed in real time.

[0871] Input: User's voice

[0872] Output: Recorded audio data

[0873] What it does: The device stores the audio data digitally and performs noise reduction and voice normalization as needed.

[0874] Step 3: The device sends the audio data to the server

[0875] The device sends the collected voice data to the server using a secure communication protocol (e.g., HTTPS). The data is encrypted before transmission to protect the user's privacy.

[0876] Input: Recorded audio data

[0877] Output: Audio data sent to the server

[0878] Specific operation: The device encrypts the voice data and sends it to a server over the Internet.

[0879] Step 4: The server inputs the voice data into the disease diagnosis AI model

[0880] The server inputs the received voice data into a disease diagnosis AI model, which is built using machine learning frameworks such as TensorFlow and PyTorch. The server also performs data preprocessing (e.g., noise removal and voice normalization).

[0881] Input: Audio data sent to the server

[0882] Output: Data input into the disease diagnosis AI model

[0883] Specific operation: The server preprocesses the voice data and converts it into a format suitable for the disease diagnosis AI model.

[0884] Step 5: The disease diagnosis AI model analyzes the voice data

[0885] The AI ​​disease diagnosis model analyzes voice data to detect signs of illness, for example, detecting abnormalities such as voice tremors or changes in pitch and determining what kind of illness they indicate.

[0886] Input: Preprocessed audio data

[0887] Output: Analysis results for symptoms of disease

[0888] How it works: The disease diagnosis AI model analyzes the voice data and detects abnormal patterns.

[0889] Step 6: The server notifies the results to medical institutions, the nursing care industry, and security companies.

[0890] The server notifies medical institutions, the nursing care industry, and security companies of the analysis results output by the disease diagnosis AI model via email, SMS, and a dedicated application.

[0891] Input: Analysis results for disease symptoms

[0892] Output: Notified analysis results

[0893] Specific behavior: The server converts the parsed results into an appropriate format and sends them to the specified recipient.

[0894] In this way, the system can monitor the health of the elderly 24 hours a day and detect early signs of illness.

[0895] (Application example 1)

[0896] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0897] It is important to monitor the health status of the elderly on a daily basis and detect early signs of illness, but the current system makes it difficult for the elderly to check their health themselves. Frequent visits to medical institutions and nursing homes are also a burden. Furthermore, there is an inadequate notification system for prompt action when abnormalities are detected.

[0898] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0899] In this invention, the server includes means for an interactive AI to acquire vocal cord data from a conversation with the elderly person, means for an AI disease diagnosis model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for interacting with the elderly person using a smart device to acquire vocal cord data, means for transmitting the acquired vocal cord data to the AI ​​disease diagnosis model, means for displaying the diagnosis results in real time, and means for notifying when an abnormality is detected. This makes it possible to monitor the health status of the elderly person on a daily basis, detect signs of illness early, and respond promptly.

[0900] "Conversational AI" is an artificial intelligence system that acquires voice data through dialogue with elderly people and uses natural language processing technology to hold conversations.

[0901] "Vocal cord data" refers to the voice signals obtained by recording the voices of elderly people and capturing them as digital data.

[0902] The "disease diagnosis AI model" is a machine learning model that analyzes acquired vocal cord data and detects signs of disease.

[0903] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0904] The "care industry" refers to the entire industry that provides care services to the elderly and disabled.

[0905] A "security company" is a company that provides security services to protect the safety of facilities and individuals.

[0906] A "smart device" is a portable electronic device with internet connectivity and advanced computing capabilities.

[0907] "Diagnosis results" are information obtained as a result of the disease diagnosis AI model analyzing vocal cord data.

[0908] "Real-time" refers to data acquisition and processing occurring immediately.

[0909] "Notification" is the act of conveying information to relevant parties when an abnormality is detected.

[0910] The following describes an embodiment of the present invention. The system includes a conversational AI, a disease diagnosis AI model, a smart device, and a notification function.

[0911] First, conversational AI acquires vocal data through dialogue with the elderly. Conversational AI interacts with the elderly using smart devices such as smartphones and smart glasses to collect voice data. The dialogue content includes everyday conversations and health-related questions, and is conducted using natural language processing technology.

[0912] The acquired vocal cord data is then sent to a disease diagnosis AI model, which uses machine learning algorithms to analyze the vocal cord data and detect signs of disease. This analysis can be performed using generative AI.

[0913] The diagnosis results are displayed in real time. The diagnosis results are instantly displayed on the screen of the smart device, where they can be checked by the elderly person and store staff. If the diagnosis results indicate an abnormality, a notification function is activated, and the abnormality is notified to medical institutions, the nursing care industry, and security companies.

[0914] As a concrete example, when an elderly person undergoes a health check in a physical store, they wear smart glasses and a conversational AI asks, "How is your health these days?" The acquired voice data is sent to an AI model for diagnosing illnesses, and if the result indicates "abnormality," a notification is sent to the store staff.

[0915] The implementation of this system requires the following hardware and software: a smartphone, smart glasses, and a microphone. The software uses Python, the SpeechRecognition library, and the requests library.

[0916] An example of a prompt is as follows:

[0917] "Please create a program to acquire vocal cord data of the elderly, send it to a disease diagnosis AI model, and display the diagnosis results. Please also add a function to notify store staff if an abnormality is detected."

[0918] In this way, a system for implementing the present invention is constructed.

[0919] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0920] Step 1:

[0921] A user starts a smart device (smartphone or smart glasses) and starts a conversational AI application. The user begins a dialogue by following the application's instructions. The input is the user's voice, and the output is voice data. The smart device uses a microphone to record the user's voice and acquires it as voice data.

[0922] Step 2:

[0923] The device sends the acquired voice data to the conversational AI. The conversational AI analyzes the voice data using natural language processing technology and engages in a dialogue with the user. The input is the voice data, and the output is the analysis results and the content of the dialogue. The conversational AI converts the user's voice data into text and generates an appropriate response.

[0924] Step 3:

[0925] The device returns the response from the conversational AI to the user by voice. The input is the conversational AI's response text, and the output is a voice response. The device converts the text into voice and transmits it to the user through the speaker.

[0926] Step 4:

[0927] The device sends the acquired voice data to the disease diagnosis AI model. The input is the voice data and the output is the diagnosis result. The disease diagnosis AI model analyzes the voice data and performs data processing and calculations to detect signs of disease.

[0928] Step 5:

[0929] The server receives the diagnosis results from the disease diagnosis AI model and displays them on the device in real time. The input is the diagnosis result and the output is the display content. The server sends the diagnosis result to the device, which then displays the result on the screen.

[0930] Step 6:

[0931] If the diagnosis result indicates an abnormality, the server activates a notification function. The input is the diagnosis result, and the output is the notification content. The server sends a notification of the abnormality to medical institutions, the nursing care industry, and security companies.

[0932] Step 7:

[0933] The terminal displays the diagnosis results and notification content to the user. The input is the diagnosis results and notification content, and the output is the display content. The terminal displays the diagnosis results and notification content on the screen and notifies the user that an abnormality has been detected.

[0934] Through these steps, the system will be able to monitor the health status of elderly people on a daily basis, detect signs of illness early, and respond promptly.

[0935] Example 2

[0936] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0937] There is a need to acquire vocal cord data through dialogue with the elderly and use that data to detect early signs of illness. However, conventional systems often lack natural dialogue and appropriate responses, which can lead to elderly people not actively participating in the dialogue. Furthermore, there are challenges in effectively analyzing the acquired vocal cord data and accurately detecting signs of illness.

[0938] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0939] In this invention, the server includes means for the conversational AI to acquire vocal cord data from a conversation with the elderly, means for analyzing the voice data and converting it into text data, means for inputting a prompt sentence into the generation AI model to generate a response, means for converting the generated response into voice data and communicating it to the user, means for the disease diagnosis AI model to detect signs of illness from the vocal cord analysis, and means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness. This makes it possible to effectively acquire vocal cord data through natural conversation with the elderly and to detect signs of illness early based on that data.

[0940] "Conversational AI" is an artificial intelligence system that can engage in natural dialogue with the elderly, and has the ability to analyze voice input and generate appropriate responses.

[0941] "Vocal cord data" is digital data obtained from the voice of an elderly person, and includes vocal characteristics and voice patterns.

[0942] "Voice data" refers to data that is a digital recording of a user's speech and is acquired through a microphone.

[0943] "Text data" is text information that has been converted by analyzing voice data and converted into a format that is easy for conversational AI to understand.

[0944] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to generate appropriate responses based on input prompts.

[0945] A "prompt sentence" is an instruction sentence input into a generative AI model, and is generated based on the user's speech.

[0946] The "disease diagnosis AI model" is an artificial intelligence system that analyzes vocal cord data to find signs of disease and makes diagnoses based on medical knowledge.

[0947] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0948] The "care industry" refers to the industry that provides care services for the elderly and disabled, and includes care facilities and home care services.

[0949] A "security company" is a company that provides security services, responds to emergencies, and manages safety.

[0950] This invention is a system that acquires vocal cord data through dialogue with elderly people and uses that data to detect early signs of illness. The system includes conversational AI, speech recognition software, generative AI models, speech synthesis software, and disease diagnosis AI models.

[0951] The server launches a conversational AI to engage in natural conversations with the elderly. The conversational AI uses a generative AI model, such as OpenAI's GPT-4. This AI model contains a vast amount of data on medical knowledge and topics of interest to the elderly.

[0952] The terminal is a device used by the elderly person and is equipped with a microphone and speaker. The terminal receives voice input from the user (elderly person) and sends it to the server. The server analyzes the received voice data and converts it into text data using voice recognition software (e.g., Google Cloud Speech-to-Text API).

[0953] The server inputs a prompt sentence into the generative AI model based on the converted text data. An example of a prompt sentence is, "The user is asking for advice about knee pain. Please provide some advice on what to do about knee pain."

[0954] The generative AI model generates an appropriate response based on the prompt, such as, "There are several ways to treat knee pain. First, it is important to engage in moderate exercise. Walking and water exercises, which do not put strain on the knee, are particularly effective. You can also use cold or hot compresses to relieve pain. If necessary, we recommend that you consult a doctor and receive appropriate treatment."

[0955] The server sends the generated response to the device, which converts the received text data into voice data using speech synthesis software (e.g., Amazon Polly), and transmits the converted voice data to the user through the speaker.

[0956] The server stores the vocal cord data acquired during the dialogue process and performs vocal cord analysis using a disease diagnosis AI model. The disease diagnosis AI model analyzes the vocal cord data and detects signs of illness. Any signs of illness detected are notified to medical institutions, the nursing care industry, and security companies.

[0957] As a concrete example, a user speaks to the device saying, "My knees have been hurting lately. Is there anything I can do about it?" The device captures the user's voice with a microphone and sends it to the server as digital voice data. The server uses the Google Cloud Speech-to-Text API to convert the voice data into text data and inputs the prompt sentence into the generative AI model. The generative AI model generates an appropriate response, and the server sends that response to the device. The device then converts the response into voice data using Amazon Polly and communicates it to the user through the speaker.

[0958] In this way, vocal cord data can be effectively obtained through natural conversation with the elderly, and signs of illness can be detected early based on that data.

[0959] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0960] Step 1:

[0961] The user speaks into the terminal.

[0962] The user asks the device a question such as, "My knees have been hurting lately. Is there anything I can do about it?" The input is the user's voice, and the output is digital voice data captured by the device's microphone.

[0963] Step 2:

[0964] The terminal transmits the voice data to the server.

[0965] The device transmits the captured digital audio data to a server via an internet connection, with the input being the digital audio data and the output being the audio data transmitted to the server.

[0966] Step 3:

[0967] The server analyzes the voice data and converts it into text data.

[0968] The server converts the received voice data into text data using voice recognition software (for example, Google Cloud Speech-to-Text API). The input is voice data, and the output is text data such as "I've been having trouble with knee pain lately. Is there anything I can do about it?"

[0969] Step 4:

[0970] The server inputs a prompt sentence into the generative AI model.

[0971] The server inputs a prompt sentence into a generative AI model (for example, OpenAI's GPT-4) based on the converted text data. The prompt sentence is, "The user is asking for advice about knee pain. Please provide advice on what to do about knee pain." The input is text data, and the output is the prompt sentence.

[0972] Step 5:

[0973] A generative AI model generates a response.

[0974] The generative AI model generates an appropriate response based on the prompt sentence. For example, it generates a response such as, "There are several measures to treat knee pain. First, it is important to engage in moderate exercise. Walking and water exercises, which do not put strain on the knee, are particularly effective. You can also use cold or hot compresses to relieve pain. If necessary, we recommend that you consult a doctor and receive appropriate treatment." The input is the prompt sentence, and the output is the generated response text.

[0975] Step 6:

[0976] The server sends the generated response to the terminal.

[0977] The server sends the response obtained from the generative AI model to the terminal. The response is sent as text data. The input is the generated response text, and the output is the text data sent to the terminal.

[0978] Step 7:

[0979] The terminal will then vocalize the response to the user.

[0980] The device converts the received text data into voice data using speech synthesis software (e.g., Amazon Polly), and transmits the converted voice data to the user through a speaker. The input is text data, and the output is voice data.

[0981] Step 8:

[0982] The server stores the vocal cord data acquired during the dialogue.

[0983] The server stores the voice and text data acquired during the dialogue, which can then be used for later analysis and system improvement. The input is the voice and text data, and the output is the stored data.

[0984] (Application example 2)

[0985] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0986] When elderly people shop in physical stores, it is difficult for them to obtain product information and health advice. There is also the issue of difficulty in collecting data to monitor the health status of elderly people. To solve these problems, a system that allows elderly people to interact naturally is needed.

[0987] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0988] In this invention, the server includes means for the conversational AI to acquire vocal cord data from a conversation with the elderly person, means for the disease diagnostic AI model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for the elderly person to receive product information and health advice by talking to the conversational AI via smartphone while shopping in a physical store, and means for acquiring the elderly person's vocal cord data through the conversation and monitoring their health condition.This will support the elderly in making shopping in physical stores more comfortable while also enabling health condition monitoring.

[0989] "Conversational AI" is an artificial intelligence system that provides information and collects data through natural dialogue with the elderly.

[0990] "Vocal cord data" is audio information obtained by analyzing the voices of elderly people, and is data used to monitor their health status.

[0991] The "disease diagnosis AI model" is an artificial intelligence model that analyzes vocal cord data to find signs of disease.

[0992] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0993] The "care industry" is an industry that provides care services to the elderly and disabled.

[0994] A "security company" is a company that provides security services to protect the safety of facilities and individuals.

[0995] A "brick and mortar store" is a store that exists in a physical location where customers can visit and purchase products in person.

[0996] A "smartphone" is a mobile phone that can connect to the Internet and use applications.

[0997] "Product information" refers to information about the product's features, price, usage, etc.

[0998] "Health advice" is advice or information about maintaining health and preventing disease.

[0999] "Health monitoring" means continuously observing an individual's health status to check for any abnormalities.

[1000] The system for implementing this invention includes an interactive AI, a disease diagnosis AI model, a smartphone, and a server. The specific configuration and operation of the system are described below.

[1001] System configuration

[1002] 1. Conversational AI: Provides information and collects vocal data through natural conversations with seniors. Conversational AI possesses vast knowledge, including medical knowledge and topics of interest to seniors.

[1003] 2. Disease diagnosis AI model: Conversational AI analyzes collected vocal data to detect signs of illness. Generative AI is used to provide more accurate diagnoses.

[1004] 3. Smartphones: These devices are used by seniors while shopping in physical stores and provide an interface for interacting with conversational AI.

[1005] 4. Server: Hosts the conversational AI and disease diagnosis AI models, and processes and stores data.

[1006] System Operation

[1007] 1. Acquiring voice input: The elderly person speaks to the conversational AI through their smartphone. The smartphone's microphone captures the voice and converts it into text using speech recognition software (e.g., SpeechRecognition library).

[1008] 2. Response generation by conversational AI: Based on the converted text, conversational AI generates an appropriate response. This is done using OpenAI's API.

[1009] 3. Voice output: The responses generated by the conversational AI are output as voice using speech synthesis software (e.g., the pyttsx3 library).

[1010] 4. Collection and analysis of vocal cord data: The AI ​​disease diagnosis model analyzes vocal cord data collected during the conversation to detect signs of illness.

[1011] 5. Notification: Any signs of illness detected will be notified to medical institutions, the care industry, and security companies.

[1012] Specific examples

[1013] For example, if an elderly person asks, "What effect does this supplement have?", the conversational AI will generate a response such as, "This supplement is rich in vitamin D and supports bone health," and output it in voice. At the same time, the vocal cord data collected through this dialogue will be analyzed by a disease diagnosis AI model, and if any abnormalities are detected, a medical institution will be notified.

[1014] Prompt Sentence Examples

[1015] User: What does this supplement do?

[1016] AI: This supplement is rich in vitamin D and supports bone health.

[1017] In this way, it is possible to help seniors feel more comfortable shopping in physical stores while also monitoring their health.

[1018] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1019] Step 1:

[1020] The user speaks to the conversational AI through their smartphone. The smartphone's microphone captures the voice and uses speech recognition software (e.g., the SpeechRecognition library) to convert the voice data into text data.

[1021] Input: User's voice

[1022] Output: Text data

[1023] What it does: Your smartphone's microphone captures your voice, and the SpeechRecognition library converts it into text.

[1024] Step 2:

[1025] The server receives the text data and the conversational AI generates an appropriate response, using OpenAI's API.

[1026] Input: Text data

[1027] Output: AI response text

[1028] Specific operation: The server sends text data to the OpenAI API and receives the generated response text.

[1029] Step 3:

[1030] The server sends the generated response text to the smartphone, which then outputs it aloud using speech synthesis software (e.g., the pyttsx3 library).

[1031] Input: AI response text

[1032] Output: Voice response

[1033] Specific operation: The server sends the response text to the smartphone, and the pyttsx3 library converts the text into speech and outputs it.

[1034] Step 4:

[1035] The server sends the vocal cord data collected during the conversation to a disease diagnosis AI model, which analyzes the vocal cord data to detect signs of disease.

[1036] Input: Vocal cord data

[1037] Output: Presence or absence of signs of illness

[1038] Specific operation: The server inputs vocal cord data into a disease diagnosis AI model and performs analysis using generative AI.

[1039] Step 5:

[1040] The server notifies medical institutions, the nursing care industry, and security companies of any signs of illness it detects.

[1041] Input: Presence or absence of signs of illness

[1042] Output: Information message

[1043] What it does: If the server detects any signs of illness, it will send a notification message to pre-defined contacts.

[1044] In this way, the system can help seniors feel more comfortable shopping in brick-and-mortar stores while simultaneously monitoring their health.

[1045] Example 3

[1046] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1047] Conventional disease diagnosis systems have had difficulty accurately detecting symptoms of illness from voice data. Furthermore, there was a lack of means to quickly provide diagnostic results to users, which could delay early medical treatment. Furthermore, there was a problem of insufficient preprocessing and feature extraction of voice data, which reduced the accuracy of diagnosis.

[1048] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1049] In this invention, the server includes means for a user to record voice data, means for a terminal to transmit the voice data to the server, means for the server to preprocess the voice data, means for the server to input data to a generative AI model, means for the generative AI model to generate symptoms of illness, means for the server to input the generated symptoms to a disease diagnostic AI model, means for the disease diagnostic AI model to generate a diagnostic result, means for the server to transmit the diagnostic result to the terminal, and means for the terminal to display the diagnostic result to the user. This makes it possible to accurately detect symptoms of illness from voice data and quickly provide a diagnostic result to the user.

[1050] "User" refers to an individual who utilizes the system to record voice data and receive diagnostic results.

[1051] "Audio data" refers to digital data containing voice information recorded by a user.

[1052] "Terminal" refers to a device that records voice data, transmits it to the server, and displays the diagnosis results. Specifically, this includes smartphones and tablets.

[1053] "Server" refers to a computer system that receives voice data, performs preprocessing, and runs the generative AI model and the disease diagnosis AI model.

[1054] "Preprocessing" refers to the process of removing noise from the audio data and extracting necessary features.

[1055] A "generative AI model" refers to an artificial intelligence model that learns patterns in voice data and generates symptoms of disease from those patterns.

[1056] "Disease symptoms" refer to features or patterns detected in voice data that indicate possible illness.

[1057] A "disease diagnosis AI model" refers to an artificial intelligence model that analyzes the symptoms of disease generated by a generative AI model and generates a final diagnosis result.

[1058] "Diagnosis results" refers to the assessment and diagnostic information regarding the user's health condition generated by the disease diagnosis AI model.

[1059] "Display" refers to the act of the terminal visually presenting the diagnostic results to the user.

[1060] This invention is a system that detects signs of illness using voice data and provides a diagnosis result to the user. The system operates by having the user record voice data and send it to a server via a terminal. The server preprocesses the voice data, generates signs of illness using a generative AI model, and generates a diagnosis result using a disease diagnosis AI model. The diagnosis result is sent to the terminal and displayed to the user.

[1061] Hardware and software used

[1062] Hardware: smartphones, tablets, servers

[1063] Software: TensorFlow, PyTorch, Librosa, audio recording app

[1064] Specific operation of the system

[1065] User operations

[1066] A user uses a device such as a smartphone or tablet to launch a voice recording app. For example, they record a phrase such as "Hello, I'm fine." Once the recording is complete, the device sends the voice data to the server.

[1067] Server Processing

[1068] The server receives the voice data sent from the device. The received voice data is first preprocessed. Preprocessing includes noise removal and Mel-Frequency Cepstral Coefficient (MFCC) extraction using Librosa. The preprocessed data is then input into a generative AI model. This model is built using TensorFlow and PyTorch and learns patterns from the voice data.

[1069] The generative AI model generates signs of illness from input voice data. For example, it analyzes voice tremors and pitch fluctuations to find patterns that indicate possible illness. The generated signs of illness are then input into a disease diagnosis AI model. This model evaluates the likelihood of illness based on past diagnostic data.

[1070] The disease diagnosis AI model analyzes the generated symptoms and generates a final diagnosis. For example, it evaluates the possibility of a cold, flu, or vocal cord polyps. The diagnosis result is sent from the server to the user's device. The result is encrypted and transmitted to protect the user's privacy.

[1071] Terminal display

[1072] The device displays the received diagnosis results to the user, who can then check the results via a smartphone app and consult a medical institution if necessary.

[1073] Specific examples

[1074] For example, if a user feels signs of a cold, they can record their vocal cord data on their smartphone and receive a diagnosis through the system, allowing them to receive appropriate medical treatment early.

[1075] Prompt Sentence Examples

[1076] "The user uses a smartphone to record vocal data and send it to the server. The server uses TensorFlow to run a generative AI model to generate disease symptoms. The generated symptoms are passed to a disease diagnosis AI model, which generates a final diagnosis result. The diagnosis result is then sent to the user's smartphone, where the user can view it."

[1077] This system allows users to easily collect vocal cord data and detect signs of illness at an early stage. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1078] Step 1:

[1079] The user records audio data

[1080] The user uses a voice recording app on their smartphone to record their own voice, for example, by saying a phrase such as "Hello, I'm fine." The recorded voice data is then stored digitally on the device.

[1081] Input: User's voice

[1082] Output: Recorded audio data (digital format)

[1083] Step 2:

[1084] The device sends the audio data to the server.

[1085] The device sends the recorded audio data to a server via the Internet, and the data is encrypted for security purposes.

[1086] Input: Recorded audio data

[1087] Output: Audio data sent to the server

[1088] Step 3:

[1089] The server preprocesses the audio data

[1090] The server preprocesses the received audio data, specifically using the Python library Librosa to remove noise and extract Mel-Frequency Cepstral Coefficients (MFCCs).

[1091] Input: Audio data sent to the server

[1092] Output: Preprocessed speech data (after feature extraction)

[1093] Step 4:

[1094] The server inputs data into the generative AI model

[1095] The preprocessed data is then fed into a generative AI model built using TensorFlow and PyTorch, which learns patterns from the audio data.

[1096] Input: Preprocessed audio data

[1097] Output: Data input into a generative AI model

[1098] Step 5:

[1099] Generative AI model generates disease symptoms

[1100] The generative AI model generates symptoms of illness from input voice data, for example by analyzing voice tremors and pitch fluctuations to find patterns that may indicate illness.

[1101] Input: Data input to a generative AI model

[1102] Output: Generated disease symptoms

[1103] Step 6:

[1104] The server inputs the generated symptoms into a disease diagnosis AI model.

[1105] The generated disease symptoms are then fed into a disease diagnosis AI model, which assesses the likelihood of disease based on past diagnostic data.

[1106] Input: Generated disease symptoms

[1107] Output: Data input into the disease diagnosis AI model

[1108] Step 7:

[1109] Disease diagnosis AI model generates diagnosis results

[1110] The disease diagnosis AI model analyzes the generated symptoms and generates a final diagnosis, such as assessing the likelihood of a cold, flu, or vocal cord polyps.

[1111] Input: Data input into the disease diagnosis AI model

[1112] Output: The generated diagnostic results

[1113] Step 8:

[1114] The server sends the diagnostic results to the device.

[1115] The diagnosis results are sent from the server to the user's device, and are encrypted to protect the user's privacy.

[1116] Input: Generated diagnostic results

[1117] Output: Diagnostic results sent to the terminal

[1118] Step 9:

[1119] The device displays the diagnostic results to the user.

[1120] The device displays the received diagnosis results to the user, who can then check the results via a smartphone app and consult a medical institution if necessary.

[1121] Input: Diagnostic results sent to the terminal

[1122] Output: Diagnostic results displayed to the user

[1123] (Application example 3)

[1124] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1125] Conventional disease diagnosis systems have the problem of low accuracy in detecting signs of illness using vocal cord data, making early detection difficult. Furthermore, there is a lack of means to monitor the user's health status in real time, making it difficult to take effective measures, especially in managing the health of the elderly and employees. Furthermore, notification methods after detecting signs of illness are limited, resulting in delayed responses in situations where prompt action is required.

[1126] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1127] In this invention, the server includes: a conversational AI system for acquiring vocal cord data from a conversation with a user; a disease diagnostic AI model for detecting signs of illness from vocal cord analysis; a system for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness; a smartphone microphone for collecting voice data; a preprocessing system for preprocessing the collected voice data; a generative AI model for generating signals from the preprocessed voice data; a disease diagnostic AI model for inputting the generated signals into the disease diagnostic AI model to detect health risks; and a system for notifying the user of the detected health risks. This enables highly accurate detection of signs of illness from the user's vocal cord data and real-time health monitoring. Furthermore, prompt notification of detected health risks enables early response.

[1128] "Conversational AI" is an artificial intelligence system that collects information and generates responses through natural conversation with users.

[1129] "Vocal cord data" refers to data that is a digital record of the voice signals emitted from the user's vocal cords.

[1130] The "disease diagnosis AI model" is an artificial intelligence model trained to analyze vocal cord data and detect signs of disease.

[1131] A "generative AI model" is an artificial intelligence model that has the ability to generate new data from input data.

[1132] A "smartphone" is a portable multi-function information terminal that can collect voice data and run applications.

[1133] "Voice data" refers to data that is a digital recording of a user's speech.

[1134] "Preprocessing" refers to the process of converting collected speech data into a format suitable for analysis and model input.

[1135] A "signal" is data generated by a generative AI model and input into a disease diagnosis AI model.

[1136] A "health risk" is a potential problem with a user's health status detected by the disease diagnosis AI model.

[1137] "Notification" is the act of informing users and relevant organizations of detected health risks.

[1138] A system for implementing this invention includes an interactive AI, a disease diagnosis AI model, a generative AI model, a smartphone, a means for preprocessing audio data, a signal generation means, a health risk detection means, and a notification means.

[1139] First, the user collects voice data using a smartphone. The smartphone's microphone records the user's vocal data. This voice data is collected in real time and transmitted to the system.

[1140] The server then preprocesses the collected voice data, which includes analyzing the data and extracting features such as Mel-Frequency Cepstrum Coefficients (MFCCs). This preprocessing is done to convert the voice data into a format suitable for the disease diagnosis AI model.

[1141] The preprocessed speech data is input to a generative AI model, which generates a new signal from the preprocessed data. This signal is then input to a disease diagnosis AI model.

[1142] The AI ​​disease diagnosis model analyzes the generated signals to detect signs of disease and health risks. This model has learned patterns in vocal cord data and can detect signs of disease with high accuracy.

[1143] Any detected health risks are communicated to users and relevant authorities through notification methods, which may be via smartphone applications, allowing users to be informed of their health status in real time.

[1144] As a concrete example, consider a scenario in which corporate employees use this system to monitor their daily health. Employees use the app to record their voices every morning, and the system analyzes the vocal data to detect stress, fatigue, and potential health risks. Detected risks are notified to employees and corporate health managers, allowing for early response.

[1145] Example prompt sentence:

[1146] "To monitor employee health, employees use this app to record their voice every morning. The app analyzes the vocal data to detect stress, fatigue, and potential health risks."

[1147] In this way, companies can understand the health status of their employees in real time and take early action.

[1148] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1149] Step 1:

[1150] The user collects voice data using a smartphone. Specifically, the smartphone's microphone is used to record the user's vocal data. The input is the user's voice, and the output is digital voice data.

[1151] Step 2:

[1152] The server preprocesses the collected voice data. Specifically, it analyzes the voice data and extracts features such as Mel-Frequency Cepstrum Coefficients (MFCC). The input is digital voice data, and the output is preprocessed feature data.

[1153] Step 3:

[1154] The server inputs the preprocessed speech data into the generative AI model. Specifically, the preprocessed data is passed to the generative AI model to generate a new signal. The input is the preprocessed feature data, and the output is the generated signal.

[1155] Step 4:

[1156] The server inputs the generated signal into a disease diagnosis AI model. Specifically, the generated signal is passed to the disease diagnosis AI model to detect signs of disease and health risks. The input is the generated signal, and the output is the detected health risk.

[1157] Step 5:

[1158] The server notifies the user and relevant organizations of the detected health risks through notification means. Specifically, it notifies the user of their health status in real time through a smartphone application. The input is the detected health risk, and the output is a notification message.

[1159] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1160] "Example 1"

[1161] One embodiment of the present invention is a system that incorporates an emotion engine. In this system, a conversational AI acquires vocal cord data from a conversation with an elderly person. The acquired vocal cord data is sent to both a disease diagnosis AI model and an emotion engine. The disease diagnosis AI model detects signs of illness from vocal cord analysis, and the emotion engine recognizes the elderly person's emotions from the vocal cord data. For example, if the elderly person expresses negative emotions such as anger or sadness, the emotion engine feeds that information back to the conversational AI. The conversational AI then adjusts the conversation with the elderly person based on that feedback.

[1162] "Example 2"

[1163] Another embodiment of the present invention is a system that notifies medical institutions, the nursing care industry, and security companies when emotions recognized by the emotion engine exceed a certain threshold. For example, if the emotion engine determines that an elderly person is experiencing strong stress or deep sadness, that information is notified to medical institutions, the nursing care industry, and security companies. This enables early response to the elderly person's psychological health condition.

[1164] "Example 3"

[1165] Another embodiment of the present invention is a system in which an emotion engine and a disease diagnosis AI model work in conjunction with each other. In this system, emotions recognized by the emotion engine are combined with signs of illness detected by the disease diagnosis AI model, enabling more accurate analysis of health conditions. For example, if the emotion engine determines that an elderly person is feeling anger or sadness, and the disease diagnosis AI model simultaneously detects signs of heart disease, this information is notified to medical institutions, the nursing care industry, and security companies. This enables early response to both the psychological and physical health conditions of the elderly.

[1166] The processing flow of each embodiment will be described below.

[1167] "Example 1"

[1168] Step 1: The conversational AI initiates a dialogue with the elderly person and acquires vocal cord data.

[1169] Step 2: The acquired vocal cord data is sent to both the disease diagnosis AI model and the emotion engine.

[1170] Step 3: The disease diagnosis AI model finds signs of disease from vocal cord analysis, and the emotion engine recognizes the elderly's emotions from vocal cord data.

[1171] Step 4: The emotion engine feeds the recognized emotions back to the conversational AI.

[1172] Step 5: The conversational AI uses the feedback to adjust the dialogue with the elderly.

[1173] "Example 2"

[1174] Step 1: The emotion engine recognizes the emotions of the elderly.

[1175] Step 2: If the emotion recognized by the emotion engine exceeds a certain threshold, the information is notified to medical institutions, the nursing care industry, and security companies.

[1176] Step 3: Medical institutions, the nursing care industry, and security companies will be notified and will take early action to address the psychological health of the elderly.

[1177] "Example 3"

[1178] Step 1: The conversational AI acquires vocal data from a conversation with the elderly person.

[1179] Step 2: The acquired vocal cord data is sent to both the disease diagnosis AI model and the emotion engine.

[1180] Step 3: The disease diagnosis AI model finds signs of disease from vocal cord analysis, and the emotion engine recognizes the elderly's emotions from vocal cord data.

[1181] Step 4: The emotions recognized by the emotion engine are combined with the disease symptoms detected by the disease diagnosis AI model to perform a more accurate analysis of the health status.

[1182] Step 5: If the analysis results exceed a certain threshold, the information is notified to medical institutions, the nursing care industry, and security companies.

[1183] Step 6: Medical institutions, caregiving industries, and security companies are notified and take early action to address both the psychological and physical health of the elderly.

[1184] Example 1

[1185] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1186] There is a need for 24-hour monitoring of the health status of the elderly and early detection of signs of illness. However, existing systems do not adequately collect and analyze voice data, and do not adjust dialogue in response to changes in emotional state. This makes it difficult to accurately grasp the health status of the elderly and provide appropriate treatment.

[1187] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1188] In this invention, the server includes a means for a conversational AI to acquire vocal cord data from a conversation with the elderly, a means for a disease diagnosis AI model to detect signs of illness from vocal cord analysis, a means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, a means for an emotion engine to recognize the emotions of the elderly from the vocal cord data, and a means for the conversational AI to adjust the conversation with the elderly based on feedback from the emotion engine. This makes it possible to monitor the health status of the elderly 24 hours a day and detect signs of illness early.

[1189] "Conversational AI" is an artificial intelligence system that collects voice data through dialogue with elderly people and uses natural language processing technology to hold conversations.

[1190] "Vocal cord data" refers to the voice signals generated by the elderly person's vocalizations, and is data that can be analyzed to evaluate their health and emotional state.

[1191] The "disease diagnosis AI model" is a machine learning model that analyzes voice data and detects signs of disease.

[1192] The "Emotion Engine" is an artificial intelligence system that recognizes and analyzes the emotional state of elderly people from voice data.

[1193] "Feedback" refers to the process by which the emotion engine returns information about the emotional state analyzed to the conversational AI.

[1194] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[1195] The "caregiving industry" refers to the entire industry that provides life support and care services for the elderly and disabled.

[1196] A "security company" is a company that provides security services to protect the safety of individuals and facilities.

[1197] "Generative AI" is an artificial intelligence system that has the ability to generate new information and data from given data.

[1198] MODE FOR CARRYING OUT THE INVENTION

[1199] This invention is a system that combines conversational AI and a disease diagnosis AI model, and aims to monitor the health status of elderly people 24 hours a day and detect early signs of illness. A specific embodiment of this system is described below.

[1200] server

[1201] The server plays a central role in running the conversational AI, disease diagnosis AI models, and emotion engines. The server uses the following hardware and software:

[1202] Hardware: powerful processor, sufficient memory, storage devices, and network interfaces

[1203] Software: Conversational AI software, disease diagnosis AI models, emotion engines, database management systems

[1204] The server receives the voice data sent from the device and sends it to the disease diagnosis AI model and emotion engine for analysis. The analysis results are then notified to medical institutions, the nursing care industry, and security companies.

[1205] Terminal

[1206] The terminal is a device for collecting voice data through dialogue with the elderly. The terminal uses the following hardware and software:

[1207] Hardware: Microphone, Speaker, Network Interface

[1208] Software: Conversational AI applications, speech recognition engine

[1209] The device collects voice data in real time through conversations with the elderly person and sends it to the server. For example, if the conversational AI asks, "How was your day today?" and the elderly person replies, "I went for a walk today," that voice data is collected.

[1210] User

[1211] The user uses conversational AI to interact with the elderly. The user performs the following operations:

[1212] Questions are input into the conversational AI, which then asks the elderly people questions via voice.

[1213] Check the collected audio data and understand the content.

[1214] The analysis results sent from the server are checked and, if necessary, medical institutions, the nursing care industry, and security companies are contacted.

[1215] Examples and prompts

[1216] For example, a conversational AI might ask an elderly person, "How was your day today?" If the elderly person replies, "I went for a walk today," the device collects the voice data. The collected data is sent to a server, where a disease diagnosis AI model analyzes the data. If the analysis reveals an abnormality in the vocal cords, the server sends a notification to a medical institution.

[1217] Example prompt sentence:

[1218] "Collect vocal data through conversations with seniors and analyze it with an AI model for disease diagnosis. For example, ask questions like: 'How was your day?'"

[1219] By inputting this prompt sentence into the generative AI model, the conversational AI collects vocal cord data through natural conversation with the elderly, which is then analyzed by the disease diagnosis AI model.

[1220] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1221] Program processing flow

[1222] Step 1: Launching Conversational AI

[1223] The server launches the conversational AI and begins a conversation session with the elderly person.

[1224] Input: Conversational AI startup command

[1225] Output: An instance of the conversational AI

[1226] Specific operation: The server creates an instance of the conversational AI and starts the speech recognition engine.

[1227] Step 2: Collecting audio data

[1228] The device collects voice data through conversations with the elderly.

[1229] Input: Elderly person's voice

[1230] Output: Collected audio data

[1231] Specific operation: The device collects voice data in real time through the microphone and transmits it to the server.

[1232] Step 3: Sending audio data

[1233] The device sends the collected voice data to the server.

[1234] Input: Collected audio data

[1235] Output: Audio data sent to the server

[1236] Specific operation: The device compresses the collected audio data and sends it to the server using a secure communication protocol.

[1237] Step 4: Analyzing the audio data

[1238] The server sends the received voice data to a disease diagnosis AI model for analysis.

[1239] Input: Audio data sent to the server

[1240] Output: Disease diagnosis result

[1241] How it works: The server preprocesses the audio data and inputs it into a disease diagnosis AI model, which then detects signs of illness from the audio data.

[1242] Step 5: Emotion analysis using the emotion engine

[1243] The server also sends the voice data to the emotion engine for emotion analysis.

[1244] Input: Audio data sent to the server

[1245] Output: Emotion analysis results

[1246] Specific operation: The server inputs the voice data into the emotion engine and analyzes the elderly person's emotional state, for example, detecting emotions such as anger or sadness.

[1247] Step 6: Notification of results

[1248] The server notifies medical institutions, the nursing care industry, and security companies of the analysis results.

[1249] Input: Disease diagnosis results, emotion analysis results

[1250] Output: Notification message

[1251] Specific operation: The server generates a notification message based on the analysis results and sends it to the specified contact.

[1252] Step 7: Adjust the conversation

[1253] The server adjusts the dialogue content of the conversational AI based on feedback from the emotion engine.

[1254] Input: Sentiment analysis results

[1255] Output: Adjusted dialogue

[1256] Specific operation: The server receives feedback from the emotion engine and adjusts the dialogue content of the conversational AI. For example, if an elderly person shows negative emotions, the dialogue content will be adjusted to be gentler.

[1257] (Application example 1)

[1258] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1259] It is important to monitor the health status of elderly people in real time and detect early signs of illness. However, conventional systems have the problem of potentially overlooking signs of illness and making it difficult to properly grasp the emotional state of elderly people. Furthermore, there has been a lack of efficient means for nursing home staff to simultaneously monitor the health status and emotions of elderly people.

[1260] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for an interactive AI to acquire vocal cord data from a conversation with the elderly person, means for an illness diagnostic AI model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for an emotion engine to recognize the elderly person's emotions from the vocal cord data, means for the interactive AI to adjust the conversation with the elderly person based on the recognized emotions, and means for monitoring the elderly person's health condition and emotions in real time via an application installed in the smart glasses. This makes it possible to simultaneously monitor the elderly person's health condition and emotions, detect signs of illness early, and take appropriate measures.

[1261] "Conversational AI" is an artificial intelligence system that acquires voice data through dialogue with elderly people and uses natural language processing technology to hold conversations.

[1262] "Vocal cord data" refers to the voice signals generated by the elderly person's vocalizations, and is data that can be analyzed to evaluate their health condition and emotions.

[1263] The "disease diagnosis AI model" is a machine learning model that analyzes voice data and detects signs of disease.

[1264] The "emotion engine" is a system that recognizes the emotions of elderly people from voice data and adjusts dialogue based on that information.

[1265] "Smart glasses" are wearable devices that have built-in displays, microphones, and sensors and can run applications.

[1266] "Real-time monitoring" is the process of instantly acquiring and analyzing data and immediately reflecting the results.

[1267] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[1268] The "care industry" is an industry that provides life support and care services for the elderly and disabled.

[1269] A "security company" is a company that provides security services to protect the safety of facilities and individuals.

[1270] "Notification" is the process by which the system communicates detected information to specific recipients.

[1271] The following system configuration will be described as an embodiment of the present invention.

[1272] System Configuration

[1273] The system includes a conversational AI, a disease diagnosis AI model, an emotion engine, smart glasses, and a notification system. These components work together via a server and a device (smart glasses) to monitor the health and emotions of elderly people in real time.

[1274] Hardware and software used

[1275] Hardware:

[1276] Smart glasses (e.g. Google Glass)

[1277] microphone

[1278] software:

[1279] speech_recognition library: Used to convert speech data into text.

[1280] emotion_recognition library: Used to recognize emotions from text.

[1281] disease_diagnosis library: Used to diagnose disease symptoms from text.

[1282] notification_system library: Used to notify results.

[1283] Data processing and calculation

[1284] The server captures the conversation with the elderly person using the microphone built into the smart glasses. The captured voice data is converted into text using the speech_recognition library. Next, the text data is used to recognize emotions using the emotion_recognition library and to diagnose signs of illness using the disease_diagnosis library. Finally, the server notifies the care staff of the recognized emotions and signs of illness using the notification_system library.

[1285] Specific examples

[1286] For example, if an elderly person says, "I've been having chest pains lately," the disease diagnosis AI model will detect signs of heart disease, and the emotion engine will recognize anxiety. This will notify the caregiver, "You are showing signs of heart disease. You seem to be feeling anxious."

[1287] Prompt Sentence Examples

[1288] If an elderly person says, "I've been having chest pains lately," the disease diagnosis AI model will detect signs of heart disease, and the emotion engine will recognize anxiety. This will notify the caregiver, "You are showing signs of heart disease. You seem to be feeling anxious."

[1289] In this way, the system can simultaneously monitor the health and emotions of elderly people, detect early signs of illness, and take appropriate measures.

[1290] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1291] Step 1:

[1292] The server uses the microphone built into the smart glasses to capture the conversational voice of the elderly. The input is the elderly's voice data, and the output is a voice data file. This voice data is collected in real time by the conversational AI.

[1293] Step 2:

[1294] The server converts the acquired audio data into text using the speech_recognition library. The input is an audio data file, and the output is text data. This conversion process involves analyzing the audio signal and converting it into the corresponding string.

[1295] Step 3:

[1296] The server recognizes emotions in the converted text data using the emotion_recognition library. The input is text data, and the output is recognized emotional information. In this process, the content of the text is analyzed, and the emotion engine identifies the emotional state of the elderly person.

[1297] Step 4:

[1298] The server then uses the disease_diagnosis library to diagnose disease symptoms from the same text data. The input is text data, and the output is disease symptom information. In this process, the content of the text is analyzed and the disease diagnosis AI model detects disease symptoms.

[1299] Step 5:

[1300] The server uses the notification_system library to notify care staff of the recognized emotion information and symptom information. The input is emotion information and symptom information, and the output is a notification message. This notification is performed in real time via the smart glasses display or other notification means.

[1301] Step 6:

[1302] The terminal (smart glasses) displays notification messages to care staff. The input is the notification message, and the output is the display on the screen. This allows care staff to instantly grasp the health and emotional state of the elderly person.

[1303] In this way, the server and device work together to monitor the health and emotions of elderly people in real time, making it possible to take appropriate action early on.

[1304] Example 2

[1305] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1306] There is a need to acquire voice data through conversations with elderly people and use that data to detect early signs of illness and changes in their emotions. However, conventional systems do not adequately acquire and analyze voice data, making it difficult to properly grasp the health and psychological state of elderly people. Another challenge is recognizing changes in emotions in real time and providing the necessary notifications promptly.

[1307] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1308] In this invention, the server includes: a conversational AI acquiring voice data from a conversation with the elderly person; a voice recognition engine converting the voice data into text; a conversational AI model generating a response from the text data; a voice synthesis engine converting the generated text response into voice; an emotion recognition engine analyzing the elderly person's emotions; a notification triggering means when the emotion recognized by the emotion recognition engine exceeds a certain threshold; and a notification system sending notifications to medical institutions, the nursing care industry, and security companies. This makes it possible to grasp the elderly person's health and psychological state in real time and take necessary measures promptly.

[1309] "Conversational AI" is an artificial intelligence system designed to engage in natural dialogue with the elderly.

[1310] "Voice data" is a digital record of the voice spoken by the elderly person to the conversational AI.

[1311] A "voice recognition engine" is software or hardware for converting voice data into text data.

[1312] "Text data" is character information converted by a voice recognition engine.

[1313] An "interactive AI model" is a machine learning model that generates appropriate responses based on text data.

[1314] A "speech synthesis engine" is software or hardware for converting text data into speech.

[1315] An "emotion recognition engine" is software or hardware that analyzes and recognizes emotions from elderly people's voice data.

[1316] The "threshold" is a reference value set for the emotion recognition engine to determine changes in emotion.

[1317] The "notification system" is a system that sends notifications to medical institutions, the nursing care industry, and security companies when the emotion recognition engine exceeds a threshold.

[1318] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[1319] The "care industry" is an industry that provides care services to the elderly and disabled.

[1320] A "security company" is a company that provides security services to provide safety and security.

[1321] MODE FOR CARRYING OUT THE INVENTION

[1322] This invention is a system that acquires voice data through conversations with elderly people and uses that data to detect early signs of illness and changes in emotions. The system includes a conversational AI, a voice recognition engine, a conversational AI model, a speech synthesis engine, an emotion recognition engine, and a notification system.

[1323] server

[1324] The server hosts the conversational AI and provides a platform for dialogue with the elderly. The conversational AI is pre-populated with data on medical knowledge and topics of interest to the elderly. The server loads the AI ​​model using machine learning frameworks such as TensorFlow and PyTorch.

[1325] The server receives the voice data sent from the device and converts it to text using the Google Cloud Speech-to-Text API. The converted text is then fed into a generative AI model (e.g., GPT-4) to generate an appropriate response. The generated text response is then converted to speech using Amazon Polly or the Google Cloud Text-to-Speech API.

[1326] Furthermore, the emotion recognition engine analyzes the elderly person's emotions in real time during the conversation. If the emotion recognized by the emotion recognition engine exceeds a certain threshold, the server notifies the information to medical institutions, the nursing care industry, and security companies. Notifications are sent via email or SMS API.

[1327] Terminal

[1328] The device provides an interface for the elderly to interact with the conversational AI. The device captures voice data using a microphone, sends the voice data to the server using an HTTP POST request, receives the response voice data from the server, and plays the voice data using a speaker.

[1329] User

[1330] The user (elderly person) starts the conversation by speaking into the device. For example, they might ask, "My knee has been hurting recently. What should I do?" The user receives the conversational AI's response, which is played back on the device.

[1331] Specific examples

[1332] Example 1: Interaction with conversational AI

[1333] The user (elderly person) uses a device to ask the conversational AI, "My knee has been hurting recently. What should I do?" The conversational AI uses its medical knowledge to respond, "For knee pain, it's important to rest first. Also, if the pain persists, consult a doctor."

[1334] Example 2: Notifications by Emotion Engine

[1335] If the emotion engine recognizes that the user (elderly person) is experiencing strong stress during a conversation, the server notifies the medical institution of this information. The medical institution then checks the elderly person's psychological health status and takes any necessary measures.

[1336] Prompt Sentence Examples

[1337] "Generate example responses for when an elderly person asks a conversational AI a medical question. For example, a question about knee pain."

[1338] "Please explain how the notification system works when the emotion engine recognizes that an elderly person is under a lot of stress."

[1339] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1340] Step 1:

[1341] Acquiring voice input

[1342] The device receives voice input from the user (elderly person). Specifically, the device uses a microphone to capture voice data. The input is the user's voice, and the output is digital voice data.

[1343] Step 2:

[1344] Sending audio data

[1345] The device sends the acquired voice data to the server. Specifically, the device sends the voice data to the server using an HTTP POST request. The input is the digital voice data, and the output is a transmission completion notification to the server.

[1346] Step 3:

[1347] Receiving audio data

[1348] The server receives the audio data sent from the terminal. Specifically, the server receives the audio data via an HTTP request. The input is the audio data sent from the terminal, and the output is the received audio data.

[1349] Step 4:

[1350] Converting audio data to text

[1351] The server uses a speech recognition engine to convert the received voice data into text. Specifically, the server calls the Google Cloud Speech-to-Text API to convert the voice data into text. The input is voice data, and the output is text data.

[1352] Step 5:

[1353] Response generation using conversational AI

[1354] The server inputs the converted text into a conversational AI model to generate an appropriate response. Specifically, the server inputs a prompt sentence into a generative AI model (e.g., GPT-4) to generate a response. The input is text data, and the output is the generated text response.

[1355] Step 6:

[1356] Response transcription

[1357] The server uses a speech synthesis engine to convert the generated text response into speech. Specifically, the server converts text to speech using Amazon Polly or the Google Cloud Text-to-Speech API. The input is the generated text response, and the output is the audio data.

[1358] Step 7:

[1359] Sending audio data

[1360] The server sends the generated audio data to the terminal. Specifically, the server returns the audio data to the terminal as an HTTP response. The input is the audio data, and the output is a transmission completion notification to the terminal.

[1361] Step 8:

[1362] Receiving a response voice

[1363] The terminal receives the response audio data sent from the server. Specifically, the terminal receives the HTTP response and decodes the audio data. The input is the audio data sent from the server, and the output is the decoded audio data.

[1364] Step 9:

[1365] Playing a response voice

[1366] The terminal plays the received response voice data. Specifically, the terminal plays the voice data using a speaker. The input is the decoded voice data, and the output is the played voice.

[1367] Step 10:

[1368] Emotion recognition

[1369] The emotion recognition engine analyzes the elderly person's emotions in real time during the conversation. Specifically, the emotion recognition engine analyzes the voice data and applies emotion recognition algorithms. The input is the voice data, and the output is the emotion analysis result.

[1370] Step 11:

[1371] Threshold determination

[1372] The emotion recognition engine determines whether the recognized emotion exceeds a preset threshold. Specifically, the emotion recognition engine quantifies the level of stress or sadness and compares it with the threshold. The input is the emotion analysis result, and the output is the threshold judgment result.

[1373] Step 12:

[1374] Notification triggers

[1375] If the emotion recognition engine determines that the threshold has been exceeded, it triggers a notification to the server. Specifically, the emotion recognition engine sends an HTTP request to the server to request a notification. The input is the threshold judgment result, and the output is a notification request.

[1376] Step 13:

[1377] Sending notifications

[1378] The server receives notification requests from the emotion recognition engine and sends notifications to medical institutions, the nursing care industry, and security companies. Specifically, the server sends notifications using email or SMS APIs. The input is the notification request, and the output is the sent notification.

[1379] (Application example 2)

[1380] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1381] There is a need for real-time monitoring of the physical and psychological well-being of elderly people, early detection of abnormalities, and appropriate response. However, existing systems lack the functionality to recognize the emotional state of elderly people through natural dialogue and to promptly notify them when necessary, making it difficult to respond early.

[1382] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1383] In this invention, the server includes a means for an interactive AI to acquire voice data from a conversation with the elderly, a means for an AI disease diagnosis model to detect signs of illness from voice analysis, a means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, and a means for an emotion engine to recognize the emotional state of the elderly and notify them when a certain threshold is exceeded. This makes it possible to monitor the physical and psychological health of the elderly in real time, detect abnormalities early, and take appropriate action.

[1384] "Conversational artificial intelligence" is an artificial intelligence technology that allows natural dialogue with elderly people and acquires voice data.

[1385] "Voice data" refers to voice information acquired through conversation with the elderly.

[1386] The "disease diagnosis artificial intelligence model" is an artificial intelligence model that analyzes voice data and detects signs of disease.

[1387] "Speech analysis" is the process of analyzing captured speech data to detect specific patterns and anomalies.

[1388] "Disease symptoms" are abnormal health conditions or early symptoms of disease that can be detected through voice analysis.

[1389] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[1390] The "care industry" is an industry that provides life support and care services for the elderly and disabled.

[1391] A "security company" is a company that provides security services and responds to emergencies.

[1392] "Notification" is an action to inform interested parties of specific information.

[1393] The "Emotion Engine" is a technology for recognizing and analyzing the emotional state of the elderly.

[1394] "Emotional state" refers to the type and intensity of emotions experienced by the elderly.

[1395] The "threshold" is a reference value for issuing a notification when the emotional state exceeds a certain level.

[1396] The system for implementing this invention includes a conversational artificial intelligence, a disease diagnosis artificial intelligence model, an emotion engine, and a notification system. The details of each component and their operation are described below.

[1397] Conversational Artificial Intelligence

[1398] Conversational AI is designed to have natural conversations with the elderly. It uses speech recognition technology to convert the elderly's speech into text data, and then analyzes the text data to generate appropriate responses. The hardware used includes smartphones, smart glasses, and head-mounted displays. The software used is Python and the speech_recognition library.

[1399] Disease diagnosis artificial intelligence model

[1400] The AI ​​disease diagnosis model is used to analyze voice data acquired by conversational AI and find signs of illness. This model detects specific patterns and abnormalities in the voice data and identifies early symptoms of illness. The software used is the emotion_recognition library.

[1401] Emotion Engine

[1402] The emotion engine is used to recognize and analyze the emotional state of elderly people in real time. The emotion engine calculates an emotion score from voice data and dialogue content, and notifies users when the score exceeds a certain threshold. The emotion engine is implemented using Python and includes an emotion recognition algorithm.

[1403] Notification System

[1404] The notification system notifies medical institutions, the nursing care industry, and security companies when the disease diagnosis AI model or emotion engine detects an abnormality. Notifications are sent via email, SMS, and a dedicated application. The software used includes the notification_service module.

[1405] Specific examples

[1406] For example, if an elderly person says, "I'm not feeling well today," the conversational AI converts that statement into text data, which the disease diagnosis AI model analyzes to detect signs of illness. At the same time, the emotion engine analyzes the elderly person's emotional state, and if they are experiencing high levels of stress or sadness, the notification system will notify medical institutions, the nursing care industry, and security companies.

[1407] Prompt Sentence Examples

[1408] Develop an application that interacts with elderly people and monitors their emotional state using an emotion engine. Include a function to notify medical institutions, the nursing care industry, and security companies if the emotion exceeds a certain threshold.

[1409] In this way, it is possible to monitor the physical and psychological well-being of elderly people in real time, detect abnormalities early, and take appropriate measures.

[1410] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1411] Step 1:

[1412] The user (elderly person) speaks into the device (smartphone, smart glasses, head-mounted display).

[1413] Input: Elderly person's voice

[1414] Output: Audio data

[1415] Specific operation: The device's microphone captures the elderly person's voice and saves it as audio data.

[1416] Step 2:

[1417] The terminal uses voice recognition technology to convert the voice data into text data.

[1418] Input: Audio data

[1419] Output: Text data

[1420] Specific operation: The speech_recognition library on the device analyzes the audio data and generates corresponding text data.

[1421] Step 3:

[1422] The server receives the text data and the conversational artificial intelligence generates an appropriate response.

[1423] Input: Text data

[1424] Output: Response text

[1425] Specific operation: Conversational artificial intelligence on the server analyzes the text data and generates appropriate responses for the elderly.

[1426] Step 4:

[1427] The response text generated by the server is converted into voice data using voice synthesis technology and sent to the terminal.

[1428] Input: Response text

[1429] Output: Response voice data

[1430] Specific operation: The speech synthesis engine on the server converts the response text into voice data and sends it to the terminal.

[1431] Step 5:

[1432] The terminal plays back the response voice data and provides a response to the elderly person.

[1433] Input: Response voice data

[1434] Output: Played audio

[1435] Specific operation: The device's speaker plays the response voice data and lets the elderly person listen.

[1436] Step 6:

[1437] The server sends the text data to an artificial intelligence disease diagnosis model, which analyzes the symptoms of the disease.

[1438] Input: Text data

[1439] Output: Signs of illness

[1440] Specific operation: A disease diagnosis artificial intelligence model on the server analyzes text data and detects signs of disease.

[1441] Step 7:

[1442] The server sends the text data to an emotion engine to analyze the emotional state.

[1443] Input: Text data

[1444] Output: Sentiment score

[1445] Specific operation: The emotion engine on the server analyzes the text data and calculates an emotion score.

[1446] Step 8:

[1447] The server compares the emotion score with a threshold and notifies you if necessary.

[1448] Input: Sentiment score

[1449] Output: Notification message

[1450] Specific operation: The server compares the emotion score with a threshold, and if the threshold is exceeded, it uses the notification_service module to send a notification to medical institutions, the nursing care industry, and security companies.

[1451] In this way, the system can monitor the physical and psychological well-being of elderly people in real time, detect abnormalities early, and take appropriate action.

[1452] Example 3

[1453] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1454] There is a need to detect the health condition of elderly people early and provide appropriate treatment promptly. However, conventional systems have difficulty accurately detecting signs of illness from voice data and are unable to simultaneously evaluate psychological health conditions. Therefore, a system that can comprehensively evaluate physical and psychological health conditions and provide appropriate treatment is needed.

[1455] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1456] In this invention, the server includes a means for an interactive AI to acquire voice data from a conversation with the elderly, a means for a disease diagnostic AI model to find signs of illness from voice analysis, a means for a generative AI to learn patterns in the voice data and generate signs of illness from those patterns, a means for an emotion recognition engine to recognize emotions from the voice data, a means for combining the recognized emotions and signs of illness to generate analysis results, and a means for notifying medical institutions, the nursing care industry, and security companies of the generated analysis results. This enables a comprehensive evaluation of the physical and psychological health of the elderly and enables early intervention.

[1457] "Conversational artificial intelligence" is artificial intelligence that acquires voice data through dialogue with elderly people and engages in dialogue.

[1458] "Voice data" is digital data that records the voice uttered by the user.

[1459] The "disease diagnosis artificial intelligence model" is an artificial intelligence model that analyzes voice data and detects signs of disease.

[1460] "Generative AI" is an AI that learns patterns in voice data and generates symptoms of disease from those patterns.

[1461] An "emotion recognition engine" is an engine for recognizing a user's emotions from voice data.

[1462] The "analysis results" are the result of combining data generated by the disease diagnosis artificial intelligence model and the emotion recognition engine.

[1463] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[1464] The "care industry" is an industry that provides care services to the elderly and disabled.

[1465] A "security company" is a company that provides services to protect the safety of facilities and individuals.

[1466] This invention is a system that comprehensively evaluates the health status of elderly people and enables early intervention. A specific embodiment of this system will be described below.

[1467] First, when the user speaks into the device, the device uses its built-in microphone to collect voice data, which is a digital recording of the voice coming from the user's vocal cords. The device then transmits the collected voice data to a server.

[1468] The server preprocesses the received audio data, specifically using an audio processing library (e.g., Librosa) to remove noise and normalize the audio. This preprocessing improves the quality of the audio data and ensures accurate subsequent analysis.

[1469] The server then inputs the preprocessed speech data into a generative AI (e.g., GPT-4), which learns patterns associated with disease symptoms from past data and generates new patterns. These patterns are then input into a disease diagnosis AI model.

[1470] A disease diagnosis AI model (e.g., a model using TensorFlow) detects signs of illness from voice data based on patterns generated by generative AI. The model analyzes the features of the voice data and identifies signs associated with specific diseases.

[1471] Furthermore, the server uses an emotion recognition engine (e.g., EmotionAPI) to recognize the user's emotions from the voice data. The emotion recognition engine analyzes characteristics of the voice data, such as the tone and rhythm, to identify the emotions the user is feeling.

[1472] The server combines the symptoms of illness detected by the disease diagnosis AI model with the emotions recognized by the emotion recognition engine to generate a comprehensive analysis result, which includes information about the user's physical and psychological health status.

[1473] Finally, the server notifies medical institutions, the nursing care industry, and security companies of the results of the analysis, allowing relevant parties to quickly understand the user's health condition and take appropriate action.

[1474] Examples:

[1475] When an elderly person says to the device, "I've been having a lot of chest pain lately," the device records the voice and sends it to the server. The server preprocesses the voice data, and the generative AI learns the patterns. The disease diagnosis AI model detects signs of heart disease, and the emotion recognition engine recognizes that the elderly person is feeling sad. This information is then notified to a medical institution, stating, "The user has signs of heart disease and is feeling sad."

[1476] Example prompt sentence:

[1477] "Please explain the process by which the generative AI learns patterns using the vocal cord data of an elderly person as input, and the disease diagnosis AI model detects signs of heart disease. Please also include how the emotion recognition engine recognizes the emotions of the elderly person and notifies the medical institution of the results."

[1478] In this way, how the server, terminal, and user cooperate to process data and detect signs of illness will be specifically described. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1479] Step 1:

[1480] Audio data collection

[1481] When a user speaks into the terminal, the terminal collects voice data using its built-in microphone.

[1482] Input: User's voice

[1483] Output: Recorded audio data

[1484] What it does: When a user says, "Hello, I'm not feeling too well today," the device records the audio and saves it as digital audio data.

[1485] Step 2:

[1486] Audio data preprocessing

[1487] The server receives the audio data sent from the device and performs noise reduction and audio normalization using an audio processing library (e.g., Librosa).

[1488] Input: Recorded audio data

[1489] Output: Preprocessed audio data

[1490] Specific behavior: The server removes background noise from the recorded audio data and keeps the volume constant.

[1491] Step 3:

[1492] Pattern learning using generative artificial intelligence

[1493] The server then feeds the preprocessed audio data into a generative artificial intelligence (e.g., GPT-4) to learn patterns associated with disease symptoms.

[1494] Input: Preprocessed audio data

[1495] Output: Learned patterns

[1496] Specific operation: The server instructs the generative AI to "learn patterns to detect signs of heart disease from this voice data."

[1497] Step 4:

[1498] Disease diagnosis using artificial intelligence model

[1499] The server inputs the patterns generated by the generative AI into a disease diagnosis AI model (e.g., a model using TensorFlow) to detect signs of disease.

[1500] Input: Learned pattern

[1501] Output: Symptoms of illness

[1502] Specific operation: The server instructs the disease diagnosis AI model to "detect signs of heart disease from this pattern," and the model responds with "Signs of heart disease have been found."

[1503] Step 5:

[1504] Emotion recognition using an emotion recognition engine

[1505] The server uses an emotion recognition engine (e.g., EmotionAPI) to recognize the user's emotion from the voice data.

[1506] Input: Audio data

[1507] Output: Recognized emotion

[1508] Specific operation: The server instructs the emotion recognition engine to "recognize the user's emotion from this voice data," and the emotion recognition engine responds with "The user is feeling sad."

[1509] Step 6:

[1510] Generate analysis results

[1511] The server combines the symptoms of disease detected by the disease diagnosis AI model with the emotions recognized by the emotion recognition engine to generate comprehensive analysis results.

[1512] Input: Symptoms of illness, perceived emotions

[1513] Output: Comprehensive analysis results

[1514] Specific behavior: The server combines the symptoms of illness and emotions to generate an analysis result such as "The user has symptoms of heart disease and feels sad."

[1515] Step 7:

[1516] Notification of results

[1517] The server notifies medical institutions, the nursing care industry, and security companies of the analysis results generated.

[1518] Input: Comprehensive analysis results

[1519] Output: Notified information

[1520] Specific behavior: The server notifies the medical institution that "The user has symptoms of heart disease and is feeling sad."

[1521] (Application example 3)

[1522] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1523] It is important to monitor the health status of the elderly in real time and detect signs of illness early. However, conventional systems have low accuracy in detecting signs of illness from vocal cord data and do not take emotional state into account, making it difficult to grasp a comprehensive health condition. In addition, there is a lack of a mechanism to immediately display detected information and notify appropriate authorities if necessary.

[1524] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for an interactive AI to acquire vocal cord data from a conversation with the elderly person, means for a disease diagnostic AI model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for an emotion engine to recognize emotions from the vocal cord data, means for integrating and analyzing the emotions recognized by the emotion engine and the signs of illness detected by the disease diagnostic AI model, means for displaying the integrated analysis results, and means for notifying security companies and medical institutions as necessary. This makes it possible to comprehensively grasp the health condition of the elderly person and take appropriate measures early on.

[1525] "Conversational AI" is an artificial intelligence system that collects voice data through dialogue with elderly people and communicates with them using natural language processing technology.

[1526] "Vocal cord data" refers to data that is a digital recording of the sound signals emitted from the human vocal cords.

[1527] The "disease diagnosis AI model" is an artificial intelligence model designed to analyze vocal cord data and detect signs of specific diseases.

[1528] An "emotion engine" is an artificial intelligence system for recognizing a speaker's emotional state from voice data.

[1529] "Generative AI" is an artificial intelligence system that has the ability to generate new data from existing data.

[1530] The "means of integration and analysis" is a means for making a comprehensive assessment by combining the emotions recognized by the emotion engine with the signs of disease detected by the disease diagnosis AI model.

[1531] "Display means" refers to a device or interface for visually presenting the analysis results to the user.

[1532] "Means of notification" refers to a system for quickly transmitting detected important information to the appropriate agencies and personnel.

[1533] The system for implementing this invention includes an interactive AI, a disease diagnosis AI model, an emotion engine, a generative AI, a display means, and a notification means. Specific embodiments of each means will be described below.

[1534] Conversational AI

[1535] The server uses conversational AI to converse with the elderly. The conversational AI uses natural language processing technology to understand what the elderly say and generate appropriate responses. This allows vocal cord data to be obtained from the elderly.

[1536] Acquiring vocal cord data

[1537] The device (e.g., smart glasses or smartphone) transmits the vocal cord data collected through conversational AI to a server, where it is digitally recorded and used for subsequent analysis.

[1538] Disease diagnosis AI model

[1539] The server analyzes the acquired vocal cord data using a disease diagnosis AI model designed to detect signs of specific diseases from the vocal cord data. This model uses generative AI to learn patterns in the vocal cord data and make more accurate diagnoses.

[1540] Emotion Engine

[1541] The server recognizes emotions from the vocal data using an emotion engine, which analyzes the characteristics of the voice data and identifies the speaker's emotional state (e.g., anger, sadness, joy, etc.).

[1542] A means of integration and analysis

[1543] The server integrates and analyzes the emotions recognized by the emotion engine and the signs of illness detected by the disease diagnosis AI model. This integrated analysis makes it possible to assess the overall health status of the elderly person.

[1544] Display method

[1545] The device visually presents the integrated analysis results to users, for example by displaying the results on the display of smart glasses, allowing security guards or caregivers to check the information in real time.

[1546] Notification method

[1547] The server notifies security companies and medical institutions as necessary to quickly communicate important information detected (such as signs of heart disease or strong feelings of anger).

[1548] Specific examples

[1549] For example, if a security guard is patrolling a shopping mall and detects that someone is showing signs of anger while also showing signs of heart disease, the information will be instantly displayed on the guard's smart glasses, allowing the guard to quickly respond and notify medical authorities if necessary.

[1550] Prompt Sentence Examples

[1551] "Develop a system in which security guards on patrol in shopping malls can analyze the vocal data of people around them in real time through smart glasses to detect emotions and signs of illness. Include a function that uses an emotion engine and an AI disease diagnosis model to instantly display detected information and notify security companies or medical institutions as necessary."

[1552] The above is a specific embodiment for carrying out the present invention.

[1553] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1554] Step 1:

[1555] The server uses conversational AI to engage in dialogue with the elderly. The conversational AI uses natural language processing technology to understand what the elderly says and generate appropriate responses. It receives the elderly's voice data as input and generates dialogue content as output. This dialogue content is recorded as voice data.

[1556] Step 2:

[1557] The device transmits the vocal cord data collected through the conversational AI to a server. It receives the voice data obtained from the conversational AI as input and generates digital vocal cord data to send to the server as output.

[1558] Step 3:

[1559] The server analyzes the acquired vocal cord data using a disease diagnosis AI model. It receives the vocal cord data as input and generates a diagnosis result indicating signs of disease as output. The disease diagnosis AI model uses generative AI to learn patterns in the vocal cord data and make more accurate diagnoses.

[1560] Step 4:

[1561] The server recognizes emotions from the vocal data using an emotion engine. It receives the vocal data as input and generates an output indicating an emotional state (e.g., anger, sadness, joy, etc.). The emotion engine analyzes the features of the voice data to identify the speaker's emotional state.

[1562] Step 5:

[1563] The server integrates and analyzes the emotions recognized by the emotion engine and the signs of illness detected by the disease diagnosis AI model. It receives emotional states and signs of illness as input and generates integrated analysis results as output. This integrated analysis makes it possible to evaluate the overall health status of elderly people.

[1564] Step 6:

[1565] The terminal visually presents the integrated analysis results to the user. It receives the integrated analysis results as input and generates information to be displayed on a display as output. For example, by displaying the results on the display of smart glasses, security guards or caregivers can check the information in real time.

[1566] Step 7:

[1567] The server notifies security companies or medical institutions as needed. It receives the integrated analysis results as input and generates notification messages as output. Notifications are used to quickly communicate important detected information (e.g., signs of heart disease or strong feelings of anger).

[1568] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1569] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1570] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1571] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1572] [Third embodiment]

[1573] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1574] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1575] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1576] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1577] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1578] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1579] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1580] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1581] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1582] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1583] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1584] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1585] "Example 1"

[1586] The present invention is a system that combines a conversational AI and a disease diagnosis AI model. The conversational AI acquires vocal cord data through dialogue with the elderly. This dialogue is conducted, for example, through everyday conversations or health-related questions. The acquired vocal cord data is sent to the disease diagnosis AI model for analysis. The disease diagnosis AI model detects signs of illness from the vocal cord data. The detected signs of illness are notified to medical institutions, the nursing care industry, and security companies. This makes it possible to check the health status of the elderly 24 hours a day and detect signs of illness early.

[1587] "Example 2"

[1588] One aspect of the present invention is a system that uses the vast knowledge of a conversational AI to engage in conversations with the elderly. The conversational AI possesses a wide range of knowledge, such as medical knowledge and topics that may be of interest to the elderly. This makes it possible to acquire vocal cord data through natural conversations with the elderly.

[1589] "Example 3"

[1590] One aspect of the present invention is a system in which a disease diagnosis AI model uses generative AI to detect signs of disease from vocal cord analysis. The generative AI learns patterns in vocal cord data and can generate signs of disease from those patterns. This enables the disease diagnosis AI model to more accurately detect signs of disease from vocal cord data.

[1591] The processing flow of each embodiment will be described below.

[1592] "Example 1"

[1593] Step 1: The conversational AI initiates a dialogue with the elderly person through questions such as, "How are you feeling today?" and "Has anything changed recently?"

[1594] Step 2: The conversational AI acquires the elderly person's vocal data, including the quality, tone, and volume of the elderly person's voice.

[1595] Step 3: The acquired vocal cord data is sent to a disease diagnosis AI model, which then identifies signs of disease from the vocal cord data.

[1596] Step 4: Any signs of illness detected by the disease diagnosis AI model are notified to medical institutions, the nursing care industry, and security companies, making it possible to detect signs of illness early and take appropriate measures.

[1597] "Example 2"

[1598] Step 1: The conversational AI initiates a dialogue with the senior through questions about topics that may be of interest to the senior, such as "What book have you read recently?" or "What is your favorite TV show?"

[1599] Step 2: The conversational AI acquires the elderly person's vocal data, including the quality, tone, and volume of the elderly person's voice.

[1600] Step 3: The acquired vocal cord data is sent to a disease diagnosis AI model, which then identifies signs of disease from the vocal cord data.

[1601] "Example 3"

[1602] Step 1: The conversational AI initiates a dialogue with the elderly person and acquires vocal cord data.

[1603] Step 2: The acquired vocal cord data is sent to a disease diagnosis AI model. The disease diagnosis AI model uses generative AI to find signs of disease from the vocal cord data. The generative AI learns patterns in the vocal cord data and generates signs of disease from those patterns.

[1604] Step 3: Any signs of illness detected by generative AI are reported to medical institutions, the nursing care industry, and security companies. This allows for early detection of signs of illness and the appropriate response.

[1605] Example 1

[1606] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1607] There is a need to check the health status of the elderly 24 hours a day and detect early signs of illness. However, conventional systems do not adequately collect and analyze voice data, which can result in signs of illness being overlooked. In addition, there is an insufficient mechanism for appropriately notifying medical institutions, the nursing care industry, and security companies of the collected data. This creates the problem of ineffective health management for the elderly.

[1608] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1609] In this invention, the server includes a means for the conversational AI to acquire voice data from a conversation with the elderly person, a means for the terminal to transmit the acquired voice data to the server, a means for the server to input the voice data into a disease diagnosis AI model, a means for the disease diagnosis AI model to detect signs of illness from the voice data, and a means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness. This makes it possible to check the health status of the elderly person 24 hours a day and detect signs of illness early.

[1610] "Conversational AI" is an artificial intelligence system that acquires voice data through dialogue with elderly people.

[1611] "Voice data" refers to voice information collected when elderly people speak to conversational AI.

[1612] A "terminal" is a device on which conversational AI is installed and which collects and transmits voice data to a server.

[1613] The "server" is a computer system that receives voice data sent from the terminal and inputs it into the disease diagnosis AI model.

[1614] The "disease diagnosis AI model" is an artificial intelligence model that analyzes voice data and detects signs of disease.

[1615] "Signs of illness" are abnormalities in health or early symptoms of illness detected from voice data.

[1616] A "medical institution" is an organization that receives symptoms of illness and provides appropriate medical treatment.

[1617] The "care industry" is an industry that manages the health of the elderly and provides necessary care services.

[1618] A "security company" is a company that ensures the safety of the elderly and responds quickly when necessary.

[1619] "Generative AI" is an artificial intelligence technology used by disease diagnosis AI models to find signs of disease from voice data.

[1620] MODE FOR CARRYING OUT THE INVENTION

[1621] This invention is a system that checks the health status of elderly people 24 hours a day and detects early signs of illness. This system is composed of a conversational AI, a terminal, a server, and a disease diagnosis AI model.

[1622] Conversational AI and Devices

[1623] The user (elderly person) speaks to a device (smartphone or dedicated device) on which the conversational AI is installed. The conversational AI acquires voice data through everyday conversations with the user and health-related questions. This conversational AI uses a general dialogue engine that uses natural language processing technology, for example.

[1624] Collection and transmission of voice data

[1625] The device collects voice data acquired through conversational AI in real time. The collected voice data is sent to the server using a secure communication protocol (e.g., HTTPS). The data is encrypted before transmission to protect the user's privacy.

[1626] Server and disease diagnosis AI model

[1627] The server receives the voice data sent from the device. The received voice data is input into a disease diagnosis AI model. This disease diagnosis AI model is built using machine learning frameworks such as TensorFlow and PyTorch. The server also performs preprocessing of the voice data (e.g., noise removal and voice normalization).

[1628] Disease symptom detection and notification

[1629] The AI ​​disease diagnosis model analyzes voice data to detect signs of illness. For example, it detects abnormalities such as voice tremors or changes in pitch and determines what kind of illness they are. The analysis results are then generated as a detailed report to assess the user's health status.

[1630] The server then sends the analysis results to medical institutions, the nursing care industry, and security companies via email, SMS, or a dedicated application, including any detected symptoms of illness and recommended measures.

[1631] Examples of concrete examples and prompts

[1632] Examples:

[1633] The user speaks to the conversational AI, saying, "I've been coughing at night lately."

[1634] The device records this conversation and transmits the audio data to a server in real time.

[1635] The server inputs the received voice data into a disease diagnosis AI model to analyze cough patterns.

[1636] The server detects from the coughing pattern that it may be an early symptom of a cold or bronchitis.

[1637] The server notifies medical institutions of this information and encourages the user to seek medical attention early.

[1638] Example prompt sentence:

[1639] If an elderly person says to a conversational AI, "I've been coughing at night lately," please explain how the voice data is collected and analyzed by the disease diagnosis AI model.

[1640] This system makes it possible to check the health status of elderly people 24 hours a day and detect early signs of illness.

[1641] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1642] Program processing flow

[1643] Step 1: The user initiates a dialogue with the conversational AI

[1644] Users talk to a device with conversational AI installed, asking everyday conversational or health-related questions such as, "What's the weather like today?" or "I've had a bit of a sore throat lately."

[1645] Input: User's voice

[1646] Output: Acquiring voice data using conversational AI

[1647] Specific operation: When a user speaks into the device, the device's built-in microphone captures the voice, and the conversational AI recognizes the voice.

[1648] Step 2: The device collects audio data

[1649] The device records the user's voice through conversational AI, using the device's built-in microphone or an external microphone, and the recorded voice data is processed in real time.

[1650] Input: User's voice

[1651] Output: Recorded audio data

[1652] What it does: The device stores the audio data digitally and performs noise reduction and voice normalization as needed.

[1653] Step 3: The device sends the audio data to the server

[1654] The device sends the collected voice data to the server using a secure communication protocol (e.g., HTTPS). The data is encrypted before transmission to protect the user's privacy.

[1655] Input: Recorded audio data

[1656] Output: Audio data sent to the server

[1657] Specific operation: The device encrypts the voice data and sends it to a server over the Internet.

[1658] Step 4: The server inputs the voice data into the disease diagnosis AI model

[1659] The server inputs the received voice data into a disease diagnosis AI model, which is built using machine learning frameworks such as TensorFlow and PyTorch. The server also performs data preprocessing (e.g., noise removal and voice normalization).

[1660] Input: Audio data sent to the server

[1661] Output: Data input into the disease diagnosis AI model

[1662] Specific operation: The server preprocesses the voice data and converts it into a format suitable for the disease diagnosis AI model.

[1663] Step 5: The disease diagnosis AI model analyzes the voice data

[1664] The AI ​​disease diagnosis model analyzes voice data to detect signs of illness, for example, detecting abnormalities such as voice tremors or changes in pitch and determining what kind of illness they indicate.

[1665] Input: Preprocessed audio data

[1666] Output: Analysis results for symptoms of disease

[1667] How it works: The disease diagnosis AI model analyzes the voice data and detects abnormal patterns.

[1668] Step 6: The server notifies the results to medical institutions, the nursing care industry, and security companies.

[1669] The server notifies medical institutions, the nursing care industry, and security companies of the analysis results output by the disease diagnosis AI model via email, SMS, and a dedicated application.

[1670] Input: Analysis results for disease symptoms

[1671] Output: Notified analysis results

[1672] Specific behavior: The server converts the parsed results into an appropriate format and sends them to the specified recipient.

[1673] In this way, the system can monitor the health status of elderly people 24 hours a day and detect early signs of illness.

[1674] (Application example 1)

[1675] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1676] It is important to monitor the health status of the elderly on a daily basis and detect early signs of illness, but the current system makes it difficult for the elderly to check their health themselves. Frequent visits to medical institutions and nursing homes are also a burden. Furthermore, there is an inadequate notification system for prompt action when abnormalities are detected.

[1677] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1678] In this invention, the server includes means for an interactive AI to acquire vocal cord data from a conversation with the elderly person, means for an AI disease diagnosis model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for interacting with the elderly person using a smart device to acquire vocal cord data, means for transmitting the acquired vocal cord data to the AI ​​disease diagnosis model, means for displaying the diagnosis results in real time, and means for notifying when an abnormality is detected. This makes it possible to monitor the health status of the elderly person on a daily basis, detect signs of illness early, and respond promptly.

[1679] "Conversational AI" is an artificial intelligence system that acquires voice data through dialogue with elderly people and uses natural language processing technology to hold conversations.

[1680] "Vocal cord data" refers to the voice signals obtained by recording the voices of elderly people and capturing them as digital data.

[1681] The "disease diagnosis AI model" is a machine learning model that analyzes acquired vocal cord data and detects signs of disease.

[1682] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[1683] The "care industry" refers to the entire industry that provides care services to the elderly and disabled.

[1684] A "security company" is a company that provides security services to protect the safety of facilities and individuals.

[1685] A "smart device" is a portable electronic device with internet connectivity and advanced computing capabilities.

[1686] "Diagnosis results" are information obtained as a result of the disease diagnosis AI model analyzing vocal cord data.

[1687] "Real-time" refers to data acquisition and processing occurring immediately.

[1688] "Notification" is the act of conveying information to relevant parties when an abnormality is detected.

[1689] The following describes an embodiment of the present invention. The system includes a conversational AI, a disease diagnosis AI model, a smart device, and a notification function.

[1690] First, conversational AI acquires vocal data through dialogue with the elderly. Conversational AI interacts with the elderly using smart devices such as smartphones and smart glasses to collect voice data. The dialogue content includes everyday conversations and health-related questions, and is conducted using natural language processing technology.

[1691] The acquired vocal cord data is then sent to a disease diagnosis AI model, which uses machine learning algorithms to analyze the vocal cord data and detect signs of disease. This analysis can be performed using generative AI.

[1692] The diagnosis results are displayed in real time. The diagnosis results are instantly displayed on the screen of the smart device, where they can be checked by the elderly person and store staff. If the diagnosis results indicate an abnormality, a notification function is activated, and the abnormality is notified to medical institutions, the nursing care industry, and security companies.

[1693] As a concrete example, when an elderly person undergoes a health check in a physical store, they wear smart glasses and a conversational AI asks, "How is your health these days?" The acquired voice data is sent to an AI model for diagnosing illnesses, and if the result indicates "abnormality," a notification is sent to the store staff.

[1694] The implementation of this system requires the following hardware and software: a smartphone, smart glasses, and a microphone. The software uses Python, the SpeechRecognition library, and the requests library.

[1695] An example of a prompt is as follows:

[1696] "Please create a program to acquire vocal cord data of the elderly, send it to a disease diagnosis AI model, and display the diagnosis results. Please also add a function to notify store staff if an abnormality is detected."

[1697] In this way, a system for implementing the present invention is constructed.

[1698] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1699] Step 1:

[1700] A user starts a smart device (smartphone or smart glasses) and starts a conversational AI application. The user begins a dialogue by following the application's instructions. The input is the user's voice, and the output is voice data. The smart device uses a microphone to record the user's voice and acquires it as voice data.

[1701] Step 2:

[1702] The device sends the acquired voice data to the conversational AI. The conversational AI analyzes the voice data using natural language processing technology and engages in a dialogue with the user. The input is the voice data, and the output is the analysis results and the content of the dialogue. The conversational AI converts the user's voice data into text and generates an appropriate response.

[1703] Step 3:

[1704] The device returns the response from the conversational AI to the user by voice. The input is the conversational AI's response text, and the output is a voice response. The device converts the text into voice and transmits it to the user through the speaker.

[1705] Step 4:

[1706] The device sends the acquired voice data to the disease diagnosis AI model. The input is the voice data and the output is the diagnosis result. The disease diagnosis AI model analyzes the voice data and performs data processing and calculations to detect signs of disease.

[1707] Step 5:

[1708] The server receives the diagnosis results from the disease diagnosis AI model and displays them on the device in real time. The input is the diagnosis result and the output is the display content. The server sends the diagnosis result to the device, which then displays the result on the screen.

[1709] Step 6:

[1710] If the diagnosis result indicates an abnormality, the server activates a notification function. The input is the diagnosis result, and the output is the notification content. The server sends a notification of the abnormality to medical institutions, the nursing care industry, and security companies.

[1711] Step 7:

[1712] The terminal displays the diagnosis results and notification content to the user. The input is the diagnosis results and notification content, and the output is the display content. The terminal displays the diagnosis results and notification content on the screen and notifies the user that an abnormality has been detected.

[1713] Through these steps, the system will be able to monitor the health status of elderly people on a daily basis, detect signs of illness early, and respond promptly.

[1714] Example 2

[1715] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1716] There is a need to acquire vocal cord data through dialogue with the elderly and use that data to detect early signs of illness. However, conventional systems often lack natural dialogue and appropriate responses, which can lead to elderly people not actively participating in the dialogue. Furthermore, there are challenges in effectively analyzing the acquired vocal cord data and accurately detecting signs of illness.

[1717] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1718] In this invention, the server includes means for the conversational AI to acquire vocal cord data from a conversation with the elderly, means for analyzing the voice data and converting it into text data, means for inputting a prompt sentence into the generation AI model to generate a response, means for converting the generated response into voice data and communicating it to the user, means for the disease diagnosis AI model to detect signs of illness from the vocal cord analysis, and means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness. This makes it possible to effectively acquire vocal cord data through natural conversation with the elderly and to detect signs of illness early based on that data.

[1719] "Conversational AI" is an artificial intelligence system that can engage in natural dialogue with the elderly, and has the ability to analyze voice input and generate appropriate responses.

[1720] "Vocal cord data" is digital data obtained from the voice of an elderly person, and includes vocal characteristics and voice patterns.

[1721] "Voice data" refers to data that is a digital recording of a user's speech and is acquired through a microphone.

[1722] "Text data" is text information that has been converted by analyzing voice data and converted into a format that is easy for conversational AI to understand.

[1723] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to generate appropriate responses based on input prompts.

[1724] A "prompt sentence" is an instruction sentence input into a generative AI model, and is generated based on the user's speech.

[1725] The "disease diagnosis AI model" is an artificial intelligence system that analyzes vocal cord data to find signs of disease and makes diagnoses based on medical knowledge.

[1726] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[1727] The "care industry" refers to the industry that provides care services for the elderly and disabled, and includes care facilities and home care services.

[1728] A "security company" is a company that provides security services, responds to emergencies, and manages safety.

[1729] This invention is a system that acquires vocal cord data through dialogue with elderly people and uses that data to detect early signs of illness. The system includes conversational AI, speech recognition software, generative AI models, speech synthesis software, and disease diagnosis AI models.

[1730] The server launches a conversational AI to engage in natural conversations with the elderly. The conversational AI uses a generative AI model, such as OpenAI's GPT-4. This AI model contains a vast amount of data on medical knowledge and topics of interest to the elderly.

[1731] The terminal is a device used by the elderly person and is equipped with a microphone and speaker. The terminal receives voice input from the user (elderly person) and sends it to the server. The server analyzes the received voice data and converts it into text data using voice recognition software (e.g., Google Cloud Speech-to-Text API).

[1732] The server inputs a prompt sentence into the generative AI model based on the converted text data. An example of a prompt sentence is, "The user is asking for advice about knee pain. Please provide some advice on what to do about knee pain."

[1733] The generative AI model generates an appropriate response based on the prompt, such as, "There are several ways to treat knee pain. First, it is important to engage in moderate exercise. Walking and water exercises, which do not put strain on the knee, are particularly effective. You can also use cold or hot compresses to relieve pain. If necessary, we recommend that you consult a doctor and receive appropriate treatment."

[1734] The server sends the generated response to the device, which converts the received text data into voice data using speech synthesis software (e.g., Amazon Polly), and transmits the converted voice data to the user through the speaker.

[1735] The server stores the vocal cord data acquired during the dialogue process and performs vocal cord analysis using a disease diagnosis AI model. The disease diagnosis AI model analyzes the vocal cord data and detects signs of illness. Any signs of illness detected are notified to medical institutions, the nursing care industry, and security companies.

[1736] As a concrete example, a user speaks to the device saying, "My knees have been hurting lately. Is there anything I can do about it?" The device captures the user's voice with a microphone and sends it to the server as digital voice data. The server uses the Google Cloud Speech-to-Text API to convert the voice data into text data and inputs the prompt sentence into the generative AI model. The generative AI model generates an appropriate response, and the server sends that response to the device. The device then converts the response into voice data using Amazon Polly and communicates it to the user through the speaker.

[1737] In this way, vocal cord data can be effectively obtained through natural conversation with the elderly, and signs of illness can be detected early based on that data.

[1738] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1739] Step 1:

[1740] The user speaks into the terminal.

[1741] The user asks the device a question such as, "My knees have been hurting lately. Is there anything I can do about it?" The input is the user's voice, and the output is digital voice data captured by the device's microphone.

[1742] Step 2:

[1743] The terminal transmits the voice data to the server.

[1744] The device transmits the captured digital audio data to a server via an internet connection, with the input being the digital audio data and the output being the audio data transmitted to the server.

[1745] Step 3:

[1746] The server analyzes the voice data and converts it into text data.

[1747] The server converts the received voice data into text data using speech recognition software (for example, Google Cloud Speech-to-Text API). The input is voice data, and the output is text data such as "I've been having trouble with knee pain lately. Is there anything I can do about it?"

[1748] Step 4:

[1749] The server inputs a prompt sentence into the generative AI model.

[1750] The server inputs a prompt sentence into a generative AI model (for example, OpenAI's GPT-4) based on the converted text data. The prompt sentence is, "The user is asking for advice about knee pain. Please provide advice on what to do about knee pain." The input is text data, and the output is the prompt sentence.

[1751] Step 5:

[1752] A generative AI model generates a response.

[1753] The generative AI model generates an appropriate response based on the prompt sentence. For example, it generates a response such as, "There are several measures to treat knee pain. First, it is important to engage in moderate exercise. Walking and water exercises, which do not put strain on the knee, are particularly effective. You can also use cold or hot compresses to relieve pain. If necessary, we recommend that you consult a doctor and receive appropriate treatment." The input is the prompt sentence, and the output is the generated response text.

[1754] Step 6:

[1755] The server sends the generated response to the terminal.

[1756] The server sends the response obtained from the generative AI model to the terminal. The response is sent as text data. The input is the generated response text, and the output is the text data sent to the terminal.

[1757] Step 7:

[1758] The terminal will then vocalize the response to the user.

[1759] The device converts the received text data into voice data using speech synthesis software (e.g., Amazon Polly), and transmits the converted voice data to the user through a speaker. The input is text data, and the output is voice data.

[1760] Step 8:

[1761] The server stores the vocal cord data acquired during the dialogue.

[1762] The server stores the voice and text data acquired during the dialogue, which can then be used for later analysis and system improvement. The input is the voice and text data, and the output is the stored data.

[1763] (Application example 2)

[1764] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1765] When elderly people shop in physical stores, it is difficult for them to obtain product information and health advice. There is also the issue of difficulty in collecting data to monitor the health status of elderly people. To solve these problems, a system that allows elderly people to interact naturally is needed.

[1766] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1767] In this invention, the server includes means for the conversational AI to acquire vocal cord data from a conversation with the elderly person, means for the disease diagnostic AI model to detect signs of illness from vocal cord analysis, means for notifying medical institutions, the nursing care industry, and security companies of the detected signs of illness, means for the elderly person to receive product information and health advice by talking to the conversational AI via smartphone while shopping in a physical store, and means for acquiring the elderly person's vocal cord data through the conversation and monitoring their health condition.This will support the elderly in making shopping in physical stores more comfortable while also enabling health condition monitoring.

[1768] "Conversational AI" is an artificial intelligence system that provides information and collects data through natural dialogue with the elderly.

[1769] "Vocal cord data" is audio information obtained by analyzing the voices of elderly people, and is data used to monitor their health status.

[1770] The "disease diagnosis AI model" is an artificial intelligence model that analyzes vocal cord data to find signs of disease.

[1771] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[1772] The "care industry" is an industry that provides care services to the elderly and disabled.

[1773] A "security company" is a company that provides security services to protect the safety of facilities and individuals.

[1774] A "brick and mortar store" is a store that exists in a physical location where customers can visit and purchase products in person.

[1775] A "smartphone" is a mobile phone that can connect to the Internet and use applications.

[1776] "Product information" refers to information about the product's features, price, usage, etc.

[1777] "Health advice" is advice or information about maintaining health and preventing disease.

[1778] "Health monitoring" means continuously observing an individual's health status to check for any abnormalities.

[1779] The system for implementing this invention includes an interactive AI, a disease diagnosis AI model, a smartphone, and a server. The specific configuration and operation of the system are described below.

[1780] System configuration

[1781] 1. Conversational AI: Provides information and collects vocal data through natural conversations with seniors. Conversational AI possesses vast knowledge, including medical knowledge and topics of interest to seniors.

[1782] 2. Disease diagnosis AI model: Conversational AI analyzes collected vocal data to detect signs of illness. Generative AI is used to provide more accurate diagnoses.

[1783] 3. Smartphones: These devices are used by seniors while shopping in physical stores and provide an interface for interacting with conversational AI.

[1784] 4. Server: Hosts the conversational AI and disease diagnosis AI models, and processes and stores data.

[1785] System Operation

[1786] 1. Acquiring voice input: The elderly person speaks to the conversational AI through their smartphone. The smartphone's microphone captures the voice and converts it into text using speech recognition software (e.g., SpeechRecognition library).

[1787] 2. Response generation by conversational AI: Based on the converted text, conversational AI generates an appropriate response. This is done using OpenAI's API.

[1788] 3. Voice output: The responses generated by the conversational AI are output as voice using speech synthesis software (e.g., the pyttsx3 library).

[1789] 4. Collection and analysis of vocal cord data: The AI ​​disease diagnosis model analyzes vocal cord data collected during the conversation to detect signs of illness.

[1790] 5. Notification: Any signs of illness detected will be notified to medical institutions, the care industry, and security companies.

[1791] Specific examples

[1792] For example, if an elderly person asks, "What effect does this supplement have?", the conversational AI will generate a response such as, "This supplement is rich in vitamin D and supports bone health," and output it in voice. At the same time, the vocal cord data collected through this dialogue will be analyzed by a disease diagnosis AI model, and if any abnormalities are detected, a medical institution will be notified.

[1793] Prompt Sentence Examples

[1794] User: What does this supplement do?

[1795] AI: This supplement is rich in vitamin D and supports bone health.

[1796] In this way, it is possible to help seniors feel more comfortable shopping in physical stores while also monitoring their health.

[1797] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1798] Step 1:

[1799] The user speaks to the conversational AI through their smartphone. The smartphone's microphone captures the voice and uses speech recognition software (e.g., the SpeechRecognition library) to convert the voice data into text data.

[1800] Input: User's voice

[1801] Output: Text data

[1802] What it does: Your smartphone's microphone captures your voice, and the SpeechRecognition library converts it into text.

[1803] Step 2:

[1804] The server receives the text data and the conversational AI generates an appropriate response, using OpenAI's API.

[1805] Input: Text data

[1806] Output: AI response text

[1807] Specific operation: The server sends text data to the OpenAI API and receives the generated response text.

[1808] Step 3:

[1809] The server sends the generated response text to the smartphone, which then outputs it aloud using speech synthesis software (e.g., the pyttsx3 library).

[1810] Input: AI response text

[1811] Output: Voice response

[1812] Specific operation: The server sends the response text to the smartphone, and the pyttsx3 library converts the text into speech and outputs it.

[1813] Step 4:

[1814] The server sends the vocal cord data collected during the conversation to a disease diagnosis AI model, which analyzes the vocal cord data to detect signs of disease.

[1815] Input: Vocal cord data

[1816] Output: Presence or absence of signs of illness

[1817] Specific operation: The server inputs vocal cord data into a disease diagnosis AI model and performs analysis using generative AI.

[1818] Step 5:

[1819] The server notifies medical institutions, the nursing care industry, and security companies of any signs of illness it detects.

[1820] Input: Presence or absence of signs of illness

[1821] Output: Information message

[1822] What it does: If the server detects any signs of illness, it will send a notification message to pre-defined contacts.

[1823] In this way, the system can help seniors feel more comfortable shopping in physical stores while simultaneously monitoring their health.

[1824] Example 3

[1825] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1826] Conventional disease diagnosis systems have had difficulty accurately detecting symptoms of illness from voice data. Furthermore, there was a lack of means to quickly provide diagnostic results to users, which could delay early medical treatment. Furthermore, there was a problem of insufficient preprocessing and feature extraction of voice data, which reduced the accuracy of diagnosis.

[1827] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1828] In this invention, the server includes means for a user to record voice data, means for a terminal to transmit the voice data to the server, means for the server to preprocess the voice data, means for the server to input data to a generative AI model, means for the generative AI model to generate symptoms of illness, means for the server to input the generated symptoms to a disease diagnostic AI model, means for the disease diagnostic AI model to generate a diagnostic result, means for the server to transmit the diagnostic result to the terminal, and means for the terminal to display the diagnostic result to the user. This makes it possible to accurately detect symptoms of illness from voice data and quickly provide a diagnostic result to the user.

[1829] "User" refers to an individual who utilizes the system to record voice data and receive diagnostic results.

[1830] "Audio data" refers to digital data containing voice information recorded by a user.

[1831] "Terminal" refers to a device that records voice data, transmits it to the server, and displays the diagnosis results. Specifically, this includes smartphones and tablets.

[1832] "Server" refers to a computer system that receives voice data, performs preprocessing, and runs the generative AI model and the disease diagnosis AI model.

[1833] "Preprocessing" refers to the process of removing noise from the audio data and extracting necessary features.

[1834] A "generative AI model" refers to an artificial intelligence model that learns patterns in voice data and generates symptoms of disease from those patterns.

[1835] "Disease symptoms" refer to features or patterns detected in voice data that indicate possible illness.

[1836] A "disease diagnosis AI model" refers to an artificial intelligence model that analyzes the symptoms of disease generated by a generative AI model and generates a final diagnosis result.

[1837] "Diagnosis results" refers to the assessment and diagnostic information regarding the user's health condition generated by the disease diagnosis AI model.

[1838] "Display" refers to the act of the terminal visually presenting the diagnostic results to the user.

[1839] This invention is a system that detects signs of illness using voice data and provides a diagnosis result to the user. The system operates by having the user record voice data and send it to a server via a terminal. The server preprocesses the voice data, generates signs of illness using a generative AI model, and generates a diagnosis result using a disease diagnosis AI model. The diagnosis result is sent to the terminal and displayed to the user.

[1840] Hardware and software used

[1841] Hardware: smartphones, tablets, servers

[1842] Software: TensorFlow, PyTorch, Librosa, audio recording app

[1843] Specific operation of the system

[1844] User operations

[1845] A user uses a device such as a smartphone or tablet to launch a voice recording app. For example, they record a phrase such as "Hello, I'm fine." Once the recording is complete, the device sends the voice data to the server.

[1846] Server Processing

[1847] The server receives the voice data sent from the device. The received voice data is first preprocessed. Preprocessing includes noise removal and Mel-Frequency Cepstral Coefficient (MFCC) extraction using Librosa. The preprocessed data is then input into a generative AI model. This model is built using TensorFlow and PyTorch and learns patterns from the voice data.

[1848] The generative AI model generates signs of illness from input voice data. For example, it analyzes voice tremors and pitch fluctuations to find patterns that indicate possible illness. The generated signs of illness are then input into a disease diagnosis AI model. This model evaluates the likelihood of illness based on past diagnostic data.

[1849] The disease diagnosis AI model analyzes the generated symptoms and generates a final diagnosis. For example, it evaluates the possibility of a cold, flu, or vocal cord polyps. The diagnosis result is sent from the server to the user's device. The result is encrypted and transmitted to protect the user's privacy.

[1850] Terminal display

[1851] The device displays the received diagnosis results to the user, who can then check the results via a smartphone app and consult a medical institution if necessary.

[1852] Specific examples

[1853] For example, if a user feels signs of a cold, they can record their vocal cord data on their smartphone and receive a diagnosis through the system, allowing them to receive appropriate medical treatment early.

[1854] Prompt Sentence Examples

[1855] "The user uses a smartphone to record vocal data and send it to the server. The server uses TensorFlow to run a generative AI model to generate disease symptoms. The generated symptoms are passed to a disease diagnosis AI model, which generates a final diagnosis result. The diagnosis result is then sent to the user's smartphone, where the user can view it."

[1856] This system allows users to easily collect vocal cord data and detect signs of illness at an early stage. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1857] Step 1:

[1858] The user records audio data

[1859] The user uses a voice recording app on their smartphone to record their own voice, for example, by saying a phrase such as "Hello, I'm fine." The recorded voice data is then stored digitally on the device.

[1860] Input: User's voice

[1861] Output: Recorded audio data (digital format)

[1862] Step 2:

[1863] The device sends the audio data to the server.

[1864] The device sends the recorded audio data to a server via the Internet, and the data is encrypted for security purposes.

[1865] Input: Recorded audio data

[1866] Output: Audio data sent to the server

[1867] Step 3:

[1868] The server preprocesses the audio data

[1869] The server preprocesses the received audio data, specifically using the Python library Librosa to remove noise and extract Mel-Frequency Cepstral Coefficients (MFCCs).

[1870] Input: Audio data sent to the server

[1871] Output: Preprocessed speech data (after feature extraction)

[1872] Step 4:

[1873] The server inputs data into the generative AI model

[1874] The preprocessed data is then fed into a generative AI model built using TensorFlow and PyTorch, which learns patterns from the audio data.

[1875] Input: Preprocessed audio data

[1876] Output: Data input into a generative AI model

[1877] Step 5:

[1878] Generative AI model generates disease symptoms

[1879] The generative AI model generates symptoms of illness from input voice data, for example by analyzing voice tremors and pitch fluctuations to find patterns that may indicate illness.

[1880] Input: Data input to a generative AI model

[1881] Output: Generated disease symptoms

[1882] Step 6:

[1883] The server inputs the generated symptoms into a disease diagnosis AI model.

[1884] The generated disease symptoms are then fed into a disease diagnosis AI model, which assesses the likelihood of disease based on past diagnostic data.

[1885] Input: Generated disease symptoms

[1886] Output: Data input into the disease diagnosis AI model

[1887] Step 7:

[1888] Disease diagnosis AI model generates diagnosis results ...

Claims

1. A means for acquiring voice data including a quality, tone, and volume of a user's voice through a dialogue with the user by a conversational AI; A means for detecting symptoms of illness by inputting the voice data into a generative AI; means for notifying at least one of a medical institution, a care facility, and a security company of the detected symptoms of the disease; A system including:

2. means for analyzing the voice data by an emotion recognition engine to analyze the emotion of a user; A means for quantifying the level of stress and sadness from the analyzed emotions of the user; means for generating a comprehensive analysis result based on the detected symptoms of illness and the quantified stress and sadness levels; The system according to claim 1 , wherein the analysis results are notified to at least one of a medical institution, a nursing facility, and a security company.

3. The system of claim 2, wherein the analysis results are notified to at least one of a medical institution, a nursing home, and a security company when the quantified stress and sadness levels exceed a predetermined threshold and when signs of illness are detected.

Images