System

Abstract

A system is provided.SOLUTION: A system, comprising: means for receiving profile information and answers to questions from a user as input; means for collecting and receiving health information from smart devices; means for pre-processing and converting the collected information into a format that can be input into a generated AI model; means for inputting the pre-processed information into the generated AI model and diagnosing a mental health condition of the user; and means for providing the user with diagnostic results and information of recommended medical institutions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the number of people suffering from mental illnesses such as depression has been increasing. However, these illnesses often have no noticeable symptoms and are difficult to detect without visiting a medical institution. Existing online diagnostic tools are based on predetermined items, making it difficult to make an accurate diagnosis. As a result, many potential depression patients are unable to receive appropriate treatment. This invention aims to solve this problem by providing an accurate diagnosis for potential depression patients and encouraging them to visit an appropriate medical institution. [Means for solving the problem]

[0005] This invention provides a system that includes a means for receiving profile information and answers to questions from a user as input, a means for collecting and receiving health data from a smart device, a means for preprocessing the collected data and converting it into a format that can be input to a generative AI model, a means for inputting the preprocessed data into the generative AI model to diagnose the user's mental health condition, and a means for providing the user with the diagnosis results and information on recommended medical institutions, thereby encouraging potential depression patients to seek treatment at appropriate medical institutions and aiming to reduce the number of patients with mental illness.

[0006] "Profile Information" refers to basic information about a user, such as their age, gender, and occupation.

[0007] "Question Answers" are responses to a series of questions that a user answers within the application, including information about their recent mood and physical condition.

[0008] A "smart device" is an electronic device, such as a smartphone or wearable device, that can collect health data such as a user's heart rate, activity level, and sleep patterns.

[0009] "Health data" refers to data about health conditions such as heart rate, activity, and sleep patterns collected from smart devices.

[0010] "Preprocessing" refers to the process of converting collected data into a format suitable for analysis, and includes data cleaning and normalization.

[0011] A "generative AI model" is an artificial intelligence model that analyzes collected data as input and is used to diagnose a user's mental health status.

[0012] "Diagnosis result" is the conclusion regarding the user's mental health status obtained through analysis by the generative AI model.

[0013] "Information about recommended medical institutions" refers to information such as contact details and appointment links for appropriate medical institutions or clinics that is provided to users based on their diagnosis results.

[0014] "Self-care resources" are information such as coping strategies and relaxation techniques that users can implement themselves based on their diagnostic results. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical attention at an appropriate medical institution. The processing and operation of the program of this system will be explained below in natural language.

[0037] System configuration

[0038] This system mainly consists of a user terminal, a server, and a smart device. The operation of each component is as follows:

[0039] Explanation of program processing

[0040] 1. Collection of User Data

[0041] User: Launches the application, enters profile information (age, gender, occupation, etc.), and answers questions related to current mood and physical condition.

[0042] Device: Sends the entered profile information and answers to questions to the server.

[0043] Server: Stores the received data in a database.

[0044] 2. Health data synchronization

[0045] User: Allows the application to access health data, which is then collected from smart devices (e.g., smartphones, wearable devices).

[0046] Device: Health data such as heart rate, exercise volume, and sleep patterns from the past week is collected from the smart device and sent to the server.

[0047] Server: Stores the received health data in a database.

[0048] 3. Data Preprocessing

[0049] Server: Retrieves stored profile information, question answers, and health data, filtering out incomplete data and outliers.

[0050] Server: Normalizes the data and converts it into a format suitable for AI models.

[0051] 4. Input to the diagnostic model

[0052] Server: Generates a feature set from the preprocessed data and inputs it into the generative AI model.

[0053] 5. Generating diagnostic results

[0054] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0055] Server: Interprets the diagnostic results and converts them into an understandable format.

[0056] 6. Distribution of Results

[0057] Server: Packages the diagnosis results and information on recommended medical institutions.

[0058] Server: Sends packaged information to the terminal.

[0059] On the device: The diagnosis results are displayed to the user, along with hospital contact information and a link to make an appointment.

[0060] 7. Follow-up

[0061] User: Based on the diagnosis, the user can use the displayed clinic information to book an appointment online or contact the clinic directly.

[0062] Users: Refer to the self-care resources provided by the app as needed.

[0063] Specific examples

[0064] For example, suppose a 28-year-old female user launches an application for the first time.

[0065] Device: The user enters their age, gender, and whether they work in sales, and then answers questions such as whether they worry a lot or have a decreased appetite.

[0066] Terminal: Sends this information to the server, which stores it in a database.

[0067] User: Then wears the smartwatch and syncs heart rate and sleep pattern data to the application.

[0068] Server: Preprocesses the received health data and inputs it into the generative AI model.

[0069] Server: The AI ​​model analyzes the user's situation and diagnoses them as having "mild depression."

[0070] Server: Along with the diagnosis results, it sends information about nearby clinics and a link to make an appointment to the user's device.

[0071] User: Checks diagnosis results and clinic information and makes online appointment with clinic.

[0072] In this way, users can receive an accurate diagnosis and smoothly consult with an appropriate medical institution. This system could be an effective means of promoting early detection and appropriate treatment for potential depression patients.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] User: Launches the application and enters profile information (age, gender, occupation, etc.) on the initial registration screen.

[0076] Terminal: Receives the entered profile information and sends it to the server.

[0077] Server: Stores the received profile information in a database.

[0078] Step 2:

[0079] User: Next, the user answers questions in the application. For example, they answer "yes" or "no" to questions like "Have you been feeling depressed for the past two weeks?"

[0080] Terminal: Receives the user's response and sends it to the server.

[0081] Server: Stores the received answers in a database.

[0082] Step 3:

[0083] User: Follows the instructions in the application and allows collection of health data from smartphones and wearable devices.

[0084] Device: Collects health data from your smart device over the past week (heart rate, activity, sleep patterns, etc.).

[0085] Terminal: Sends collected health data to the server.

[0086] Server: Stores the received health data in a database.

[0087] Step 4:

[0088] Server: Retrieves stored profile information, question answers, and health data.

[0089] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[0090] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[0091] Step 5:

[0092] Server: Generates a set of features from the preprocessed data, including, for example, the mean heart rate, standard deviation, sleep duration, and scores for answers to questions about specific emotions.

[0093] Server: Inputs the generated feature set into the generative AI model.

[0094] Step 6:

[0095] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0096] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[0097] Step 7:

[0098] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[0099] Server: Sends packaged information to the terminal.

[0100] Step 8:

[0101] On the device: Display the diagnosis and related information to the user. For example, display a message such as "You are experiencing moderate depression. We recommend the following clinics" along with the clinic's contact information.

[0102] User: Review the diagnosis and click on the recommended clinic contact and booking link to book an appointment.

[0103] Users: You can also consult self-care resources within the application (relaxation techniques, exercise programs, etc.).

[0104] Through this series of steps, users are able to accurately diagnose their mental health condition and are encouraged to seek medical attention from an appropriate medical institution.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Mental health issues are a serious social challenge in modern society. Early detection is particularly important for mental illnesses such as depression, and consultation at an appropriate medical institution is required. However, many people do not have the means to accurately assess their own mental health status, and access to appropriate medical institutions is limited. The objective of the present invention is to provide a system that accurately diagnoses a user's mental health status and encourages potential depression patients to seek treatment at an appropriate medical institution.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for generating prompt sentences, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical institutions, and means for validating the received data and detecting and correcting missing data and outliers, thereby enabling an accurate diagnosis of the user's mental health condition and promoting consultation at an appropriate medical institution.

[0110] "Means for receiving profile information and answers to questions as input from the user" is a function for collecting personal information and answer data to questions that the user has entered into the application and sending it to the server.

[0111] "Means for collecting and receiving health data from smart devices" refers to a function for transmitting health data such as heart rate, exercise volume, and sleep patterns obtained from smartphones and wearable devices to a server.

[0112] "Means for preprocessing collected data and converting it into a format that can be input into a generative AI model" refers to a function for formalizing and standardizing data obtained from users and smart devices and converting it into a form that can be used by an AI model.

[0113] "Means for inputting preprocessed data into a generative AI model and diagnosing the user's mental health state" refers to a function for inputting preprocessed data into a generative AI model and using AI to evaluate and diagnose the user's mental state.

[0114] "Means for providing users with information on diagnostic results and recommended medical institutions" refers to a function for notifying users of the diagnostic results obtained by AI and information on recommended medical institutions based on those results.

[0115] "Means for validating received data and detecting and correcting missing data and abnormal values" refers to a function for checking the accuracy of data sent to the server, finding missing or abnormal data, and supplementing or correcting it.

[0116] The "means for generating prompt sentences" is a function for creating specific instruction sentences (prompt sentences) in a format that can be analyzed by the generative AI model based on preprocessed data.

[0117] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at an appropriate medical institution. This system is mainly composed of a user terminal, a server, and a smart device.

[0118] User Data Collection

[0119] First, the user launches the application and enters profile information such as age, gender, and occupation. They also answer questions about their mood and physical condition. For example, they answer questions such as, "Have you been worrying a lot lately?" or "Have you lost your appetite?" The information entered by the user is sent to the server by the device and stored in a database.

[0120] Health Data Sync

[0121] Next, the user grants the application permission to access health data from their smart device (e.g., smartphone, smartwatch). This allows the application to collect health-related data such as heart rate, exercise volume, and sleep patterns from the past week and send it to the server. The health data is also stored in a database.

[0122] Data Preprocessing

[0123] The server retrieves the stored profile information, question answers, and health data, and detects and corrects incomplete data and outliers. Specifically, it performs data validation to detect and correct missing data and outliers. It also normalizes the data and converts it into a format that can be input into the generative AI model. For example, it standardizes numerical data and encodes categorical data.

[0124] Input to the diagnostic model

[0125] Next, a feature set is generated from the preprocessed data, and a prompt sentence is created. For example, the generated features might be "28-year-old female, sales position, high heart rate variability, light sleep." This prompt sentence is input into a generative AI model to diagnose the user's mental health condition.

[0126] Generating diagnostic results

[0127] The generative AI model evaluates the user's mental health state based on the input feature set and generates a diagnosis such as "mild depression" or "normal." The diagnosis result is interpreted by the server and converted into a format that is easy for humans to understand. Specifically, a message is generated saying, "Based on your current health data, you are suspected of having mild depression."

[0128] Results distribution

[0129] The server sends this information back to the user's device, which then displays the diagnosis, including contact details for the specific hospital and a link to make an online appointment.

[0130] Follow-up

[0131] Based on the diagnosis results, the user can use the displayed clinic information to make an appointment online or contact the clinic directly. If necessary, the user can also refer to the self-care resources provided by the application. The server monitors the user's usage and provides follow-up reminders and additional resources as needed.

[0132] Specific examples

[0133] For example, when a 28-year-old female user launches an application for the first time, the following scenario occurs:

[0134] Users enter profile information such as age, gender, and occupation, and answer questions about concerns and loss of appetite.

[0135] This information is sent from the terminal to the server and stored in a database.

[0136] The user then wears the smartwatch and syncs data on their heart rate and sleep patterns with the application.

[0137] The server preprocesses the received health data and inputs it into a generative AI model.

[0138] The AI ​​model analyzes the patient and diagnoses him as having "mild depression."

[0139] The server sends the diagnosis results along with information about nearby clinics and an online reservation link to the user's terminal.

[0140] The user checks the diagnosis results and clinic information and makes an online appointment with the clinic.

[0141] Prompt Sentence Examples

[0142] "A 28-year-old woman has recently been feeling anxious and has experienced a decrease in her appetite. Please assess this user's mental health status by taking into account her heart rate data and sleep patterns over the past week."

[0143] By using this system, users can receive an accurate diagnosis and smoothly consult with an appropriate medical institution. This system is an effective means of promoting early detection of potential depression patients and appropriate treatment.

[0144] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0145] Step 1: Collect user data

[0146] Input: Profile information (such as age, gender, and occupation) and answers to questions you enter into the application (e.g., "Have you been feeling a lot of anxiety lately?", "Have you lost your appetite?", etc.).

[0147] How it works: The user launches the application on their smartphone or PC and follows the instructions to provide the required profile information and answer health questions.

[0148] Data processing and output: These input data are converted into the appropriate format within the terminal and sent to the server. The terminal performs a data consistency check to ensure there are no inconsistencies.

[0149] Output: The user data, whose integrity has been verified, is sent to the server, which stores it in a database.

[0150] Step 2: Syncing health data

[0151] Input: Health data such as heart rate, activity, and sleep patterns obtained from smart devices (smartphones, wearable devices, etc.).

[0152] What it does: The user grants the application permission to access their smart device. The application periodically syncs with the smart device and collects health data from the past week.

[0153] Data processing and output: The device normalizes the acquired health data and performs consistency checks before sending it to the server. For example, it checks whether the heart rate is too high.

[0154] Output: The health data, whose integrity has been verified, is sent to the server, which stores it in a database.

[0155] Step 3: Data Preprocessing

[0156] Input: Saved profile information, question answers and health data.

[0157] What it does: The server retrieves these data from the database, detects and removes incomplete data and outliers, and performs other tasks, such as standardizing numeric data and encoding categorical data.

[0158] Data processing and output: The server completes incomplete data, corrects outliers, and converts the data into a format suitable for the AI ​​model. For example, it treats "age" as a numeric value and encodes "gender" as a categorical value.

[0159] Output: Preprocessed data is produced and can be input into subsequent processing steps.

[0160] Step 4: Input to the diagnostic model

[0161] Input: Preprocessed data (profile information, question answers, health data).

[0162] Specific operation: The server generates a feature set from the preprocessed data and creates a prompt based on this. The prompt generated is "28-year-old female, sales, high heart rate variability, light sleeper."

[0163] Data processing and output: The generated prompt sentences are input into a generative AI model, which analyzes them and diagnoses the user's mental health status.

[0164] Output: The generative AI model outputs a diagnosis of the user's mental health status.

[0165] Step 5: Generate diagnostic results

[0166] Input: Diagnostic results from a generative AI model.

[0167] Specific operation: The server converts the diagnosis results received from the generative AI model into a format that is easy for humans to understand. For example, it generates a message saying, "Mild depression is suspected."

[0168] Output: The interpreted diagnostic results are generated in a format that can be presented to the user.

[0169] Step 6: Delivering results

[0170] Input: Interpreted diagnosis and recommended medical provider information.

[0171] Specific operation: The server packages the diagnosis results and information on recommended medical institutions and sends them back to the terminal, which receives them and displays them to the user.

[0172] Data processing and output: For example, generating a message saying, "You are suspected of having mild depression. Here is the nearest clinic," and providing a link to make an online appointment.

[0173] Output: The diagnosis and medical institution information are displayed to the user.

[0174] Step 7: Follow up

[0175] Input: diagnostic results and self-care resources.

[0176] What happens: The user makes an online appointment using the provided clinic information and also accesses self-care resources provided by the application (e.g., mindfulness exercises, stress management guides).

[0177] Data processing and output: The server monitors the user's usage and provides follow-up reminders and additional resources as needed. For example, a reminder to "re-diagnose in a week."

[0178] Output: User receives follow-up information and resources for ongoing mental health management.

[0179] (Application example 1)

[0180] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0181] Conventional mental health diagnostic systems only target individual data, and lack mechanisms for efficiently monitoring the mental health status of employees in large organizations such as corporations and providing early and appropriate support. This can result in employees with mental health problems not receiving appropriate support, leading to reduced productivity and a worsening working environment. Another challenge is the difficulty of providing appropriate support while ensuring employee privacy and data security.

[0182] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0183] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical institutions, and means for monitoring the mental health condition of company employees and providing appropriate support early when their mental condition is unstable. This makes it possible to efficiently monitor the mental health condition of company employees and provide appropriate support early.

[0184] "Profile information" is data that includes personal information such as a user's age, gender, and occupation.

[0185] "Answers to questions" are response data entered by the user in response to questions related to mental and physical conditions presented in the application.

[0186] "Smart devices" refers to wearable devices and smartphones that can collect health data such as a user's heart rate, exercise volume, and sleep patterns.

[0187] "Health data" refers to data that indicates a user's health status, such as heart rate, exercise volume, and sleep patterns.

[0188] "Preprocessing" is the process of removing incomplete data and outliers from collected data and converting it into a format suitable for generative AI models.

[0189] A "generative AI model" is an artificial intelligence model that analyzes input data and generates a specific output (in this case, a mental health diagnosis).

[0190] "Diagnosis" refers to assessing a user's mental health status based on the output results obtained by the generative AI model.

[0191] "Information about recommended medical institutions" refers to information including contact information and appointment links for appropriate medical institutions suggested to the user based on the diagnosis results.

[0192] The "measures for monitoring the mental health status of employees in companies" is a mechanism for continuously monitoring the mental health status of employees within a company and providing appropriate support early on if a risk is determined.

[0193] "Self-care resources" refers to information and tools that users can use to manage their own mental health based on their diagnosis.

[0194] This invention is a system that diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at an appropriate medical institution, and also has a function for monitoring the mental health of company employees. This system is mainly composed of a user terminal, a server, and a smart device. The operation of each component will be described below.

[0195] First, the user launches the application and enters their profile information (age, gender, occupation, etc.) and answers questions related to their current mood and physical condition. The device then sends this information to the server, which then stores the received data in a database.

[0196] Next, the user allows the application to access their health data, which allows the application to collect health data such as heart rate, exercise volume, and sleep patterns from the past week from their smart device (e.g., smartphone or wearable device) and send it to the server. The server then stores the received health data in a database.

[0197] The server retrieves saved profile information, question answers, and health data and performs data preprocessing to remove incomplete data and outliers. The preprocessed data is converted into a format suitable for the generative AI model. The server generates a feature set from this preprocessed data and inputs it into the generative AI model. The generative AI model analyzes the input data and diagnoses the user's mental health condition. The diagnosis result is interpreted and converted into an easy-to-understand format. The server packages the diagnosis result and information on recommended medical institutions and sends it to the device. The device displays the diagnosis result to the user and also provides hospital contact information and a link to make an appointment.

[0198] In addition, the system has a function to monitor the mental health status of company employees. Companies provide this application to their employees and have them regularly check their mental health status. Employees enter data, and if there is a mental risk based on the analysis results, appropriate support is provided early. This process incorporates mechanisms to ensure data security while protecting employee privacy.

[0199] As a specific example, a 28-year-old female user launches an application for the first time, inputs her age, gender, and the fact that she works in sales, and then answers questions such as whether she worries a lot or has a decreased appetite. This information is sent to the server, which stores it in a database. The user then wears the smartwatch and synchronizes her heart rate and sleep pattern data with the application. The server preprocesses the received health data and inputs it into a generative AI model. The AI ​​model analyzes the user's condition and diagnoses her with "mild depression." Along with the diagnosis, information about nearby clinics and a link to make an appointment are sent to the user's device. The user confirms the diagnosis and clinic information and makes an online appointment with the clinic.

[0200] Example prompt sentence:

[0201] User profile data:

[0202] Age: 30

[0203] Gender: Male

[0204] Occupation: Engineer

[0205] User response data:

[0206] Mood: Stressed

[0207] Appetite: decreased

[0208] Health Data:

[0209] Heart Rate: [70, 75, 68, 74, 72, 78, 80]

[0210] Sleep Pattern: [6, 5, 7, 6, 5, 6, 4]

[0211] Activity: [2, 2, 1, 3, 2, 1, 2]

[0212] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0213] Step 1:

[0214] Collection and transmission of user data

[0215] The user starts the application, inputs their profile information (age, gender, occupation, etc.), and answers questions related to their current mood and physical condition. The device then sends this input data to the server.

[0216] Input: Profile information, answers to questions

[0217] Output: Send data to the server

[0218] Specific behavior:

[0219] The user accesses the application and enters information. The device validates the input and sends it to the server.

[0220] Step 2:

[0221] Health data synchronization and transmission

[0222] Users grant the application permission to access their health data, which allows the application to collect health data such as heart rate, exercise volume, and sleep patterns from the past week from their smart device and send it to a server.

[0223] Input: Health data from smart devices

[0224] Output: Send health data to the server

[0225] Specific behavior:

[0226] The user synchronizes their smart device with the application, and the device collects health data for a specified period and sends it to the server.

[0227] Step 3:

[0228] Data Preprocessing

[0229] The server retrieves stored profile information, question answers, and health data, performs data preprocessing to remove incomplete data and outliers, and converts the preprocessed data into a format suitable for the generative AI model.

[0230] Input: Profile information, answers to questions, health data

[0231] Output: Preprocessed data

[0232] Specific behavior:

[0233] The server reads the incoming data, performs consistency checks, removes outliers and incomplete data, and normalizes the data for the generative AI model.

[0234] Step 4:

[0235] Data input and diagnosis for generative AI models

[0236] The server generates a feature set from the preprocessed data and inputs it into a generative AI model, which analyzes the input data and diagnoses the user's mental health status.

[0237] Input: Preprocessed data

[0238] Output: Mental health status diagnosis

[0239] Specific behavior:

[0240] The server extracts a set of features and inputs them into a generative AI model, which then performs diagnostic analysis and generates results.

[0241] Step 5:

[0242] Providing diagnostic results and medical institution information

[0243] The server interprets the diagnosis results, converts them into an easy-to-understand format, and packages and sends information about recommended medical institutions to the device.

[0244] Input: Diagnostic results from generative AI model

[0245] Output: Diagnosis results and medical institution information to be presented to the user

[0246] Specific behavior:

[0247] The server interprets the diagnosis results, formats them into text, and sends them to the device along with information on recommended medical institutions.

[0248] Step 6:

[0249] Notification of results and follow-up with users

[0250] The device displays the diagnosis results and information on recommended medical institutions to the user, who can then check the results and make an appointment with a medical institution if necessary.

[0251] Input: Diagnosis results and medical institution information

[0252] Output: User presentation and follow-up action

[0253] Specific behavior:

[0254] The device displays the received information to the user, who then views the diagnostic results and clicks on the online reservation link.

[0255] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0256] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical attention at appropriate medical institutions. This system incorporates an emotion engine that recognizes the user's emotions. The program processing and operation of this system are explained below in natural language.

[0257] System configuration

[0258] This system mainly consists of the user's terminal, server, emotion engine, and smart device. The operation of each component is as follows.

[0259] Explanation of program processing

[0260] 1. Collection of User Data

[0261] User: Launches the application, enters profile information (age, gender, occupation, etc.), and answers questions related to current mood and physical condition.

[0262] Device: Sends the entered profile information and answers to questions to the server.

[0263] Server: Stores the received data in a database.

[0264] 2. Health data synchronization

[0265] User: Allows the application to access health data, which is then collected from smart devices (e.g., smartphones, wearable devices).

[0266] Device: Health data such as heart rate, exercise volume, and sleep patterns from the past week is collected from the smart device and sent to the server.

[0267] Server: Stores the received health data in a database.

[0268] 3. Collecting Emotional Data

[0269] User: The emotion engine recognizes the user's emotions based on their actions and inputs in the application.

[0270] Emotion engine: Analyzes user text input and other interactions to generate emotion data.

[0271] Terminal: Sends the generated emotion data to the server.

[0272] Server: Stores emotion data in a database.

[0273] 4. Data Preprocessing

[0274] Server: Retrieves stored profile information, question answers, health data, and emotional data.

[0275] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[0276] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[0277] 5. Input to the diagnostic model

[0278] Server: Generates a feature set from the preprocessed data, including, for example, average heart rate, standard deviation, sleep duration, answer scores to questions about specific emotions, and emotional data.

[0279] Server: Inputs the generated feature set into the generative AI model.

[0280] 6. Generating diagnostic results

[0281] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0282] Server: Customize the diagnosis results based on the emotional data recognized by the emotion engine. For example, if the emotional data indicates strong anxiety, include that information in the diagnosis results.

[0283] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[0284] 7. Distribution of Results

[0285] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[0286] Server: Sends packaged information to the terminal.

[0287] On the device: The diagnosis results are displayed to the user. For example, a message like "You are experiencing moderate depression. We recommend the following clinics" is displayed along with the clinic's contact information.

[0288] 8. Follow-up

[0289] User: Based on the diagnosis, the user can use the displayed clinic information to book an appointment online or contact the clinic directly.

[0290] User: If necessary, consult the self-care resources provided by the application (e.g., relaxation techniques, exercise programs, etc.).

[0291] Specific examples

[0292] For example, a 28-year-old male user launches an application for the first time and enters his profile information.

[0293] User: Enters age, gender, and whether they work in sales, then answers questions such as, "Have you often felt depressed in the last two weeks?"

[0294] Terminal: Sends this information to the server, which stores it in a database.

[0295] User: Then wears the smartwatch and syncs heart rate and sleep pattern data to the application.

[0296] Server: Preprocesses the received health data and inputs it into the generative AI model.

[0297] Emotion engine: Generates emotional data from user text input and other interactions.

[0298] Server: The generated emotion data is input into the AI ​​model to generate a diagnosis result, such as "The user is in a state of mild depression and has a high level of anxiety."

[0299] Server: Along with the diagnosis results, it sends information about nearby clinics and a link to make an appointment to the user's device.

[0300] Terminal: Displays the diagnosis results and allows the user to make an appointment based on the clinic information.

[0301] In this way, by analyzing various information, including the user's emotional data, it is possible to provide a more accurate diagnosis and help the user receive treatment at an appropriate medical institution. This system could be an effective means of promoting early detection and appropriate treatment of potential depression patients.

[0302] The processing flow will be explained below.

[0303] Step 1:

[0304] User: Launches the application and enters profile information (age, gender, occupation, etc.) on the initial registration screen.

[0305] Terminal: Receives the entered profile information and sends it to the server.

[0306] Server: Stores the received profile information in a database.

[0307] Step 2:

[0308] User: Next, the user answers questions in the application. For example, they answer "yes" or "no" to questions like "Have you been feeling depressed for the past two weeks?"

[0309] Terminal: Receives the user's response and sends it to the server.

[0310] Server: Stores the received answers in a database.

[0311] Step 3:

[0312] User: Follows the instructions in the application and allows collection of health data from smartphones and wearable devices.

[0313] Device: Collects health data from your smart device over the past week (heart rate, activity, sleep patterns, etc.).

[0314] Terminal: Sends collected health data to the server.

[0315] Server: Stores the received health data in a database.

[0316] Step 4:

[0317] Emotion engine: Analyzes user text input and other interactions with the application to generate emotion data.

[0318] Terminal: Sends the generated emotion data to the server.

[0319] Server: Stores emotion data in a database.

[0320] Step 5:

[0321] Server: Retrieves stored profile information, question answers, health data, and emotion data.

[0322] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[0323] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[0324] Step 6:

[0325] Server: Generates a feature set from the preprocessed data, including, for example, average heart rate, standard deviation, sleep duration, answer scores to questions about specific emotions, and emotional data.

[0326] Server: Inputs the generated feature set into the generative AI model.

[0327] Step 7:

[0328] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0329] Server: Customize the diagnosis results based on the emotional data recognized by the emotion engine. For example, if the emotional data indicates strong anxiety, include that information in the diagnosis results.

[0330] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[0331] Step 8:

[0332] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[0333] Server: Sends packaged information to the terminal.

[0334] Step 9:

[0335] On the device: Display the diagnosis and related information to the user. For example, display a message such as "You are experiencing moderate depression. We recommend the following clinics" along with the clinic's contact information.

[0336] User: Review the diagnosis and click on the recommended clinic contact and booking link to book an appointment.

[0337] Users: You can also consult self-care resources within the application (relaxation techniques, exercise programs, etc.).

[0338] Through this series of steps, users can not only receive an accurate diagnosis of their mental health condition, but also receive customized diagnostic results and recommended medical information based on their emotional data.

[0339] Example 2

[0340] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0341] In recent years, mental health problems have been increasing, and early detection of depression and anxiety disorders in particular is becoming increasingly important. However, it is difficult for users to accurately recognize their own mental state and seek medical attention at an appropriate medical institution. Furthermore, conventional systems are primarily limited to in-hospital diagnosis and are unable to comprehensively analyze diverse data from daily life. Therefore, there is a need for a system that can comprehensively analyze a user's emotions and health data, accurately diagnose their mental health state, and, if necessary, encourage them to seek medical attention at an appropriate medical institution.

[0342] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0343] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for analyzing the user's text input and interactions and generating emotion data, means for preprocessing the collected and generated data and converting it into a format that can be input to the generative AI model, means for generating a feature set from the preprocessed data and inputting it into the generative AI model, means for the generative AI model to analyze the input data and diagnose the user's mental health condition, and means for providing the user with the diagnosis results and information on recommended medical institutions. This makes it possible to comprehensively analyze various data obtained from the user's daily life, more accurately and quickly diagnose the user's mental health condition, and encourage the user to visit an appropriate medical institution.

[0344] "Profile information" refers to basic personal information such as a user's age, gender, and occupation.

[0345] "Answers to questions" are response information provided by a user in response to questions related to mental health conditions presented by the system.

[0346] A "smart device" is an electronic device that can collect health data such as heart rate, activity, and sleep patterns.

[0347] "Health data" is information about a user's physical condition, such as their heart rate, activity level, and sleep patterns.

[0348] "Emotion data" refers to information about emotions recognized and analyzed by the emotion engine from user text input and interactions.

[0349] "Data preprocessing" is the process of organizing and normalizing collected profile information, answers to questions, health data, and emotion data, and removing missing or outliers.

[0350] A "feature set" is a collection of data points that are important for analysis and are derived from preprocessed data.

[0351] A "generative AI model" is an artificial intelligence model that uses preprocessed data to diagnose a user's mental health status.

[0352] "Diagnosis results" are assessments and conclusions about the user's mental health status that are displayed as a result of the generative AI model's analysis.

[0353] "Information about recommended medical institutions" refers to information about appropriate medical institutions and specialists provided to users based on diagnostic results.

[0354] This invention is a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at appropriate medical institutions. The system is composed of a user terminal, a server, an emotion engine, and a smart device.

[0355] System configuration

[0356] Users launch the application using a device such as a smartphone or tablet. When launching the application, the user enters profile information (age, gender, occupation, etc.) and answers questions related to their mental health. In addition, the user allows the application to access their smart device (smartphone, smartwatch, etc.) to collect health data, such as heart rate, exercise volume, and sleep patterns.

[0357] Data collection and analysis

[0358] The server collects profile information, answers to questions, health data, and emotion data from the user's device and stores them in a centralized database. The emotion engine analyzes text input and other interactions the user has within the application to generate emotion data. For example, if a user inputs "I've been feeling tired lately," the emotion engine will analyze the emotions of "tired" and "anxiety."

[0359] Data preprocessing and feature generation

[0360] The server preprocesses all collected data and removes missing or outliers. From the preprocessed data, a feature set is generated to be input into the generative AI model. The feature set includes average heart rate, exercise volume, sleep duration, and emotion score. The generative AI model uses this feature set to diagnose the user's mental health condition.

[0361] Diagnostic results and medical institution recommendations

[0362] The generative AI model analyzes the user's data and generates a diagnosis, which includes specific descriptions such as "moderate depression" or "high anxiety level." Based on this diagnosis, the server recommends information about appropriate nearby medical institutions and reservation links. The diagnosis and medical institution information are sent to the user's device and displayed to the user through the application.

[0363] Specific examples

[0364] For example, a 28-year-old male user launches the application for the first time, enters his age, gender, and job title in sales, and then answers "Yes" to the question, "Have you often felt depressed in the last two weeks?" When the user wears a smartwatch and syncs it with the application, data on his heart rate and sleep patterns over the past week is sent to the server. The emotion engine analyzes "fatigue" and "anxiety" from the text entered by the user and generates emotion data.

[0365] The server preprocesses this data, generates a feature set, and inputs it into the generative AI model. The diagnosis results indicate that the user is in a state of mild depression and has a high level of anxiety, and information about nearby clinics and a link to make an appointment are sent to the user's device. The user can then make an appointment based on this information.

[0366] Prompt Sentence Examples

[0367] 1. Introducing a sentiment analysis tool using natural language processing algorithms

[0368] 2. Write a program to collect health data from smart devices

[0369] 3. Explain the architecture of the health checkup system based on profile information and emotion data.

[0370] By analyzing a variety of user data, this system can more accurately diagnose mental health conditions and encourage people to seek medical attention at appropriate medical institutions.

[0371] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0372] Step 1:

[0373] User Data Collection

[0374] User: Launches the application, enters profile information (e.g., age, gender, occupation), and answers questions related to mental health.

[0375] Input: Age, Gender, Occupation, Answer to Question

[0376] Action: A user enters information into an application and presses the submit button.

[0377] Output: Input information is saved on the device and sent to the server.

[0378] Device: Sends this information to the server.

[0379] Input: Information entered by the user

[0380] How it works: The device temporarily stores the user's input and sends an HTTP request to the server.

[0381] Output: Server receives user profile data and question answers

[0382] Server: Stores the received data in a database.

[0383] Input: User profile data and question answers

[0384] How it works: The server executes an SQL query to the database and saves the data.

[0385] Output: User data stored in the database

[0386] Step 2:

[0387] Health Data Sync

[0388] User: Allows the application to access health data.

[0389] Input: User action (access permission)

[0390] What happens: The user enables health data permission in the application settings.

[0391] Output: The permission settings change takes effect.

[0392] Device: Collects health data such as heart rate, exercise volume, and sleep patterns from smart devices and sends it to a server.

[0393] Input: Health data from smart devices

[0394] Operation: The terminal acquires data from the smart device and sends it to the server.

[0395] Output: Health data sent to the server

[0396] Server: Stores the received health data in a database.

[0397] Input: Health data

[0398] What it does: The server executes an SQL query to store in the database

[0399] Output: Health data stored in a database

[0400] Step 3:

[0401] Collecting Emotional Data

[0402] User: Enters text and otherwise interacts with the application.

[0403] Input: User text input or operation

[0404] Action: User enters text and presses submit, or interacts with a button within the app

[0405] Output: Text entered into the device and interaction data

[0406] Emotion engine: Analyzes text input and operation data to generate emotion data.

[0407] Input: User text input and operational data

[0408] How it works: The emotion engine uses natural language processing algorithms to analyze the input and generate emotion labels.

[0409] Output: Emotion data

[0410] Terminal: Sends the generated emotion data to the server.

[0411] Input: Emotion data generated by the emotion engine

[0412] Behavior: Issues an HTTP request to send emotion data to the server.

[0413] Output: Emotion data sent to the server

[0414] Server: Stores emotion data in a database.

[0415] Input: Emotion data

[0416] What it does: The server executes an SQL query to store in the database

[0417] Output: Emotion data stored in a database

[0418] Step 4:

[0419] Data Preprocessing

[0420] Server: Retrieves stored profile information, question answers, health data, and emotional data.

[0421] Input: Information retrieved from the database

[0422] What happens: The server executes a retrieval SQL query to retrieve the data.

[0423] Output: Multiple datasets retrieved for processing on the server side

[0424] Server: Performs data cleaning to remove incomplete data and outliers.

[0425] Input: Retrieved data

[0426] Action: Detects and filters data anomalies

[0427] Output: Cleaned dataset

[0428] Server: Performs data normalization.

[0429] Input: Cleaned dataset

[0430] What it does: It applies a data normalization algorithm to convert the data into a uniform format.

[0431] Output: Normalized dataset

[0432] Step 5:

[0433] Input to the diagnostic model

[0434] Server: Generates a feature set from the preprocessed data.

[0435] Input: Preprocessed dataset

[0436] Action: Runs an algorithm to extract features from the data.

[0437] Output: Feature set

[0438] Server: Inputs the feature set into the generative AI model.

[0439] Input: Feature set

[0440] Operation: Convert the feature set into an input format for the AI ​​model and input it into the model.

[0441] Output: Features input to the AI ​​model

[0442] Step 6:

[0443] Generating diagnostic results

[0444] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0445] Input: Feature set

[0446] How it works: The AI ​​model performs analysis and generates a diagnosis.

[0447] Output: Diagnostic results

[0448] Server: Converts diagnostic results into an easy-to-understand format.

[0449] Input: Diagnostic results

[0450] What it does: Converts diagnostic results into a text format for the user

[0451] Output: Diagnostic results for the user

[0452] Step 7:

[0453] Results distribution

[0454] Server: Packages the diagnosis results and information on recommended medical institutions.

[0455] Input: Diagnosis results, medical institution information

[0456] How it works: Combines diagnostic results and medical information into a single package

[0457] Output: Package information

[0458] Server: Sends packaged information to the device.

[0459] Input: Package information

[0460] Action: Sends package information to the device via HTTP request

[0461] Output: Package information received on the device

[0462] Terminal: Display diagnostic results to the user.

[0463] Input: Package information

[0464] Behavior: Updates the UI in the application to display diagnosis results and medical provider information.

[0465] Output: Diagnosis results and medical institution information displayed to the user

[0466] Step 8:

[0467] Follow-up

[0468] User: Based on the diagnosis, the user can use the displayed medical information to book an appointment online or contact the clinic directly.

[0469] Input: Diagnosis results, medical institution information

[0470] What happens: User clicks on clinic appointment link and fills out appointment form

[0471] Output: Clinic appointment completed

[0472] Users: Consult the self-care resources provided by the app as needed.

[0473] Input: Diagnostic results, self-care resources

[0474] Action: User views and practices self-care resources

[0475] Output: User self-care practices

[0476] (Application example 2)

[0477] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0478] Many systems exist that accurately diagnose mental health conditions and encourage potential depression patients to seek medical treatment at appropriate medical institutions. However, there are insufficient means to effectively collect and analyze users' emotional data and present it to users in a visually and intuitive manner. Furthermore, there are also insufficient means to analyze the collected data from multiple angles and provide users with information on improvement measures and medical facilities. This has led to issues that make it difficult for users to understand the diagnosis results and take appropriate action.

[0479] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0480] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving biometric data from the smart device, means for collecting and receiving the user's emotional data through an emotion recognition engine, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical facilities, and means for visually presenting the diagnosis results to the user through a head-mounted display. This enables the collection and analysis of multifaceted data, including the user's emotional data, and enables the user to intuitively and visually receive diagnosis results, promoting consultation at an appropriate medical institution.

[0481] "User" means an individual who uses this system and provides their profile information and biometric data.

[0482] "Profile Information" means basic personal information about a user, such as age, gender, and occupation.

[0483] "Question Answers" are responses that users make to mental health-related questions posed within the system.

[0484] "Smart device" is a general term for devices that can measure and collect a user's biometric information (e.g., heart rate, exercise volume, sleep patterns).

[0485] "Biometric data" means physiological data of a user (e.g., heart rate, sleep patterns) collected from a smart device.

[0486] An "emotion recognition engine" is software or algorithms that recognize emotions from user text input, voice, gestures, etc. and generate data.

[0487] "Preprocessing collected data" refers to data processing measures to convert collected profile information, biometric data, and emotional data into a format that can be input into a generative AI model.

[0488] A "generative AI model" is a machine learning or artificial intelligence model that assesses and diagnoses a user's mental health based on input data.

[0489] "Means for diagnosing a user's mental health condition" means means for analyzing collected and pre-processed data using a generative AI model to assess a user's mental health condition.

[0490] "Medical facility" is a general term for medical institutions such as hospitals and clinics where users can receive mental health consultations.

[0491] A "head-mounted display" is a display device that provides visual information when worn by a user.

[0492] This invention is a system that uses a head-mounted display (HMD) to diagnose a user's mental health condition and encourage them to seek appropriate medical advice. The system combines the user's profile information, answers to questions, biometric data from a smart device, and emotion data obtained using an emotion recognition engine to provide a diagnosis based on a generative AI model.

[0493] Hardware / Software used

[0494] Hardware:

[0495] Head-mounted display (HMD): General-purpose HMD device

[0496] Smart devices: devices that collect biometric data such as heart rate, activity, and sleep patterns (e.g., wearable devices)

[0497] software:

[0498] Emotion Recognition Engine: An engine that recognizes emotions from user text input, voice, and gestures.

[0499] Database: Firebase Firestore or other cloud databases

[0500] Generative AI model: A machine learning model for assessing mental health status

[0501] Web framework: Server-side framework such as Flask

[0502] System operation explanation

[0503] User Data Collection:

[0504] The user puts on the HMD device and launches the application. They input their profile information through the HMD interface using voice input and gestures. They also answer questions related to their mood and physical condition. This input data is sent from the HMD to the server and stored in a database.

[0505] Health Data Sync:

[0506] Users use smart devices (e.g., wearable devices) to collect biometric data such as heart rate, exercise volume, and sleep patterns. This data is then sent to a server via the HMD and stored in a database.

[0507] Emotion data collection:

[0508] The emotion recognition engine recognizes emotions from the user's text or voice input and generates emotion data, which is also sent to the server and stored in a database.

[0509] Data preprocessing:

[0510] The server preprocesses the collected profile information, biometric data, and emotion data, converting them into a format that can be input into the generative AI model, specifically by performing data cleaning and normalization.

[0511] Inputs to generative AI models and diagnosis:

[0512] The server inputs the preprocessed data into a generative AI model to diagnose the user's mental health status, and the results are converted into an intuitively understandable format.

[0513] Delivery of diagnostic results:

[0514] The server packages the diagnosis results and recommended medical facility information and presents them visually to the user through the HMD. The diagnosis results include contact information for nearby clinics and specialists.

[0515] Follow-up:

[0516] Users can review their diagnosis and use the HMD interface to find nearby clinics, schedule appointments online if needed, and access self-care resources (e.g., relaxation techniques, exercise programs).

[0517] Examples of specific examples and prompts

[0518] Examples:

[0519] A 28-year-old male user uses the app for the first time using a general-purpose HMD device. He registers his profile information by voice input, and then synchronizes his biometric data with his smart device. The emotion recognition engine generates emotion data based on the text input, and the diagnosis is "mild depression." The user then checks the information of nearby specialists through the HMD and makes an appointment.

[0520] Example prompt sentence:

[0521] "What steps would an HMD application that collects and analyzes a user's profile information and health data to diagnose their mental state take? Please explain using a concrete example."

[0522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0523] Step 1:

[0524] User Data Collection

[0525] The user puts on the head-mounted display and starts the application. They enter their profile information (age, gender, occupation, etc.) and answers to questions related to their mood and physical condition using voice input or gestures. This input data is sent from the device to the server and stored in a database.

[0526] Input: Profile information, answers to questions

[0527] Data calculation: voice recognition, data format conversion

[0528] Output: Profile information and question and answer data (sent to server)

[0529] Step 2:

[0530] Health Data Sync

[0531] Users collect biometric data such as heart rate, exercise volume, and sleep patterns from the past week from their smart devices (e.g., wearable devices). This data is sent from the device to a server and stored in a database.

[0532] Input: Heart rate data, exercise data, sleep pattern data

[0533] Data processing: Data synchronization, data format conversion

[0534] Output: Biometric data (sent to server)

[0535] Step 3:

[0536] Collecting Emotional Data

[0537] The emotion recognition engine analyzes the user's text and voice input to generate emotion data, which is then sent from the device to a server and stored in a database.

[0538] Input: User text input, voice input

[0539] Data Computation: Emotion Recognition, Data Generation

[0540] Output: Emotion data (sent to server)

[0541] Step 4:

[0542] Data Preprocessing

[0543] The server takes the collected profile information, question responses, biometric data, and emotional data, and performs data cleaning and normalization, converting the data into a format that can be input into a generative AI model.

[0544] Input: Profile information, question and answer data, biometric data, emotional data

[0545] Data calculations: data cleaning, data normalization

[0546] Output: Preprocessed data

[0547] Step 5:

[0548] Input to generative AI models and diagnostics

[0549] The server inputs the preprocessed data into a generative AI model to diagnose the user's mental health status, and the results are converted into an easy-to-understand format.

[0550] Input: Preprocessed data

[0551] Data calculation: feature extraction, model inference

[0552] Output: Diagnostic results

[0553] Step 6:

[0554] Distribution of diagnostic results

[0555] The server packages the diagnosis and recommended medical facility information and presents it visually to the user through the HMD, including contact information for clinics and specialists.

[0556] Input: Diagnosis results, medical facility information

[0557] Data Calculation: Data Packaging

[0558] Output: Visual presentation data (sent to HMD)

[0559] Step 7:

[0560] Follow-up

[0561] Users can check their diagnosis and medical facility information, make appointments online if necessary, and also view and use self-care resources (relaxation techniques, exercise programs, etc.).

[0562] Input: diagnosis results, medical facility information, self-care resource information

[0563] Output: Reservation information, self-care resource usage status

[0564] In this way, the system collects and analyzes data from multiple angles, provides users with intuitive and visual diagnostic results, and encourages them to seek medical attention at an appropriate medical institution, thereby achieving comprehensive mental health support.

[0565] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0566] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0567] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0568] [Second embodiment]

[0569] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0570] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0571] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0572] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0573] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0574] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0575] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0576] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0577] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0578] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0579] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0580] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0581] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical attention at an appropriate medical institution. The processing and operation of the program of this system will be explained below in natural language.

[0582] System configuration

[0583] This system mainly consists of a user terminal, a server, and a smart device. The operation of each component is as follows:

[0584] Explanation of program processing

[0585] 1. Collection of User Data

[0586] User: Launches the application, enters profile information (age, gender, occupation, etc.), and answers questions related to current mood and physical condition.

[0587] Device: Sends the entered profile information and answers to questions to the server.

[0588] Server: Stores the received data in a database.

[0589] 2. Health data synchronization

[0590] User: Allows the application to access health data, which is then collected from smart devices (e.g., smartphones, wearable devices).

[0591] Device: Health data such as heart rate, exercise volume, and sleep patterns from the past week is collected from the smart device and sent to the server.

[0592] Server: Stores the received health data in a database.

[0593] 3. Data Preprocessing

[0594] Server: Retrieves stored profile information, question answers, and health data, filtering out incomplete data and outliers.

[0595] Server: Normalizes the data and converts it into a format suitable for AI models.

[0596] 4. Input to the diagnostic model

[0597] Server: Generates a feature set from the preprocessed data and inputs it into the generative AI model.

[0598] 5. Generating diagnostic results

[0599] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0600] Server: Interprets the diagnostic results and converts them into an understandable format.

[0601] 6. Distribution of Results

[0602] Server: Packages the diagnosis results and information on recommended medical institutions.

[0603] Server: Sends packaged information to the terminal.

[0604] On the device: The diagnosis results are displayed to the user, along with hospital contact information and a link to make an appointment.

[0605] 7. Follow-up

[0606] User: Based on the diagnosis, the user can use the displayed clinic information to book an appointment online or contact the clinic directly.

[0607] Users: Refer to the self-care resources provided by the app as needed.

[0608] Specific examples

[0609] For example, suppose a 28-year-old female user launches an application for the first time.

[0610] Device: The user enters their age, gender, and whether they work in sales, and then answers questions such as whether they worry a lot or have a decreased appetite.

[0611] Terminal: Sends this information to the server, which stores it in a database.

[0612] User: Then wears the smartwatch and syncs heart rate and sleep pattern data to the application.

[0613] Server: Preprocesses the received health data and inputs it into the generative AI model.

[0614] Server: The AI ​​model analyzes the user's situation and diagnoses them as having "mild depression."

[0615] Server: Along with the diagnosis results, it sends information about nearby clinics and a link to make an appointment to the user's device.

[0616] User: Checks diagnosis results and clinic information and makes online appointment with clinic.

[0617] In this way, users can receive an accurate diagnosis and smoothly consult with an appropriate medical institution. This system could be an effective means of promoting early detection and appropriate treatment for potential depression patients.

[0618] The processing flow will be explained below.

[0619] Step 1:

[0620] User: Launches the application and enters profile information (age, gender, occupation, etc.) on the initial registration screen.

[0621] Terminal: Receives the entered profile information and sends it to the server.

[0622] Server: Stores the received profile information in a database.

[0623] Step 2:

[0624] User: Next, the user answers questions in the application. For example, they answer "yes" or "no" to questions like "Have you been feeling depressed for the past two weeks?"

[0625] Terminal: Receives the user's response and sends it to the server.

[0626] Server: Stores the received answers in a database.

[0627] Step 3:

[0628] User: Follows the instructions in the application and allows collection of health data from smartphones and wearable devices.

[0629] Device: Collects health data from your smart device over the past week (heart rate, activity, sleep patterns, etc.).

[0630] Terminal: Sends collected health data to the server.

[0631] Server: Stores the received health data in a database.

[0632] Step 4:

[0633] Server: Retrieves stored profile information, question answers, and health data.

[0634] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[0635] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[0636] Step 5:

[0637] Server: Generates a set of features from the preprocessed data, including, for example, the mean heart rate, standard deviation, sleep duration, and scores for answers to questions about specific emotions.

[0638] Server: Inputs the generated feature set into the generative AI model.

[0639] Step 6:

[0640] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0641] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[0642] Step 7:

[0643] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[0644] Server: Sends packaged information to the terminal.

[0645] Step 8:

[0646] On the device: Display the diagnosis and related information to the user. For example, display a message such as "You are experiencing moderate depression. We recommend the following clinics" along with the clinic's contact information.

[0647] User: Review the diagnosis and click on the recommended clinic contact and booking link to book an appointment.

[0648] Users: You can also consult self-care resources within the application (relaxation techniques, exercise programs, etc.).

[0649] Through this series of steps, users are able to accurately diagnose their mental health condition and are encouraged to seek medical attention from an appropriate medical institution.

[0650] Example 1

[0651] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0652] Mental health issues are a serious social challenge in modern society. Early detection is particularly important for mental illnesses such as depression, and consultation at an appropriate medical institution is required. However, many people do not have the means to accurately assess their own mental health status, and access to appropriate medical institutions is limited. The objective of the present invention is to provide a system that accurately diagnoses a user's mental health status and encourages potential depression patients to seek treatment at an appropriate medical institution.

[0653] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0654] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for generating prompt sentences, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical institutions, and means for validating the received data and detecting and correcting missing data and outliers, thereby enabling an accurate diagnosis of the user's mental health condition and promoting consultation at an appropriate medical institution.

[0655] "Means for receiving profile information and answers to questions as input from the user" is a function for collecting personal information and answer data to questions that the user has entered into the application and sending it to the server.

[0656] "Means for collecting and receiving health data from smart devices" refers to a function for transmitting health data such as heart rate, exercise volume, and sleep patterns obtained from smartphones and wearable devices to a server.

[0657] "Means for preprocessing collected data and converting it into a format that can be input into a generative AI model" refers to a function for formalizing and standardizing data obtained from users and smart devices and converting it into a form that can be used by an AI model.

[0658] "Means for inputting preprocessed data into a generative AI model and diagnosing the user's mental health state" refers to a function for inputting preprocessed data into a generative AI model and using AI to evaluate and diagnose the user's mental state.

[0659] "Means for providing users with information on diagnostic results and recommended medical institutions" refers to a function for notifying users of the diagnostic results obtained by AI and information on recommended medical institutions based on those results.

[0660] "Means for validating received data and detecting and correcting missing data and abnormal values" refers to a function for checking the accuracy of data sent to the server, finding missing or abnormal data, and supplementing or correcting it.

[0661] The "means for generating prompt sentences" is a function for creating specific instruction sentences (prompt sentences) in a format that can be analyzed by the generative AI model based on preprocessed data.

[0662] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at an appropriate medical institution. This system is mainly composed of a user terminal, a server, and a smart device.

[0663] User Data Collection

[0664] First, the user launches the application and enters profile information such as age, gender, and occupation. They also answer questions about their mood and physical condition. For example, they answer questions such as, "Have you been worrying a lot lately?" or "Have you lost your appetite?" The information entered by the user is sent to the server by the device and stored in a database.

[0665] Health Data Sync

[0666] Next, the user grants the application permission to access health data from their smart device (e.g., smartphone, smartwatch). This allows the application to collect health-related data such as heart rate, exercise volume, and sleep patterns from the past week and send it to the server. The health data is also stored in a database.

[0667] Data Preprocessing

[0668] The server retrieves the stored profile information, question answers, and health data, and detects and corrects incomplete data and outliers. Specifically, it performs data validation to detect and correct missing data and outliers. It also normalizes the data and converts it into a format that can be input into the generative AI model. For example, it standardizes numerical data and encodes categorical data.

[0669] Input to the diagnostic model

[0670] Next, a feature set is generated from the preprocessed data, and a prompt sentence is created. For example, the generated features might be "28-year-old female, sales position, high heart rate variability, light sleep." This prompt sentence is input into a generative AI model to diagnose the user's mental health condition.

[0671] Generating diagnostic results

[0672] The generative AI model evaluates the user's mental health state based on the input feature set and generates a diagnosis such as "mild depression" or "normal." The diagnosis result is interpreted by the server and converted into a format that is easy for humans to understand. Specifically, a message is generated saying, "Based on your current health data, you are suspected of having mild depression."

[0673] Results distribution

[0674] The server sends this information back to the user's device, which then displays the diagnosis, including contact details for the specific hospital and a link to make an online appointment.

[0675] Follow-up

[0676] Based on the diagnosis results, the user can use the displayed clinic information to make an appointment online or contact the clinic directly. If necessary, the user can also refer to the self-care resources provided by the application. The server monitors the user's usage and provides follow-up reminders and additional resources as needed.

[0677] Specific examples

[0678] For example, when a 28-year-old female user launches an application for the first time, the following scenario occurs:

[0679] Users enter profile information such as age, gender, and occupation, and answer questions about concerns and loss of appetite.

[0680] This information is sent from the terminal to the server and stored in a database.

[0681] The user then wears the smartwatch and syncs data on their heart rate and sleep patterns with the application.

[0682] The server preprocesses the received health data and inputs it into a generative AI model.

[0683] The AI ​​model analyzes the patient and diagnoses him as having "mild depression."

[0684] The server sends the diagnosis results along with information about nearby clinics and an online reservation link to the user's terminal.

[0685] The user checks the diagnosis results and clinic information and makes an online appointment with the clinic.

[0686] Prompt Sentence Examples

[0687] "A 28-year-old woman has recently been feeling anxious and has experienced a decrease in her appetite. Please assess this user's mental health status by taking into account her heart rate data and sleep patterns over the past week."

[0688] By using this system, users can receive an accurate diagnosis and smoothly consult with an appropriate medical institution. This system is an effective means of promoting early detection of potential depression patients and appropriate treatment.

[0689] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0690] Step 1: Collect user data

[0691] Input: Profile information (such as age, gender, and occupation) and answers to questions you enter into the application (e.g., "Have you been feeling a lot of anxiety lately?", "Have you lost your appetite?", etc.).

[0692] How it works: The user launches the application on their smartphone or PC and follows the instructions to provide the required profile information and answer health questions.

[0693] Data processing and output: These input data are converted into the appropriate format within the terminal and sent to the server. The terminal performs a data consistency check to ensure there are no inconsistencies.

[0694] Output: The user data, whose integrity has been verified, is sent to the server, which stores it in a database.

[0695] Step 2: Syncing health data

[0696] Input: Health data such as heart rate, activity, and sleep patterns obtained from smart devices (smartphones, wearable devices, etc.).

[0697] What it does: The user grants the application permission to access their smart device. The application periodically syncs with the smart device and collects health data from the past week.

[0698] Data processing and output: The device normalizes the acquired health data and performs consistency checks before sending it to the server. For example, it checks whether the heart rate is too high.

[0699] Output: The health data, whose integrity has been verified, is sent to the server, which stores it in a database.

[0700] Step 3: Data Preprocessing

[0701] Input: Saved profile information, question answers and health data.

[0702] What it does: The server retrieves these data from the database, detects and removes incomplete data and outliers, and performs other tasks, such as standardizing numeric data and encoding categorical data.

[0703] Data processing and output: The server completes incomplete data, corrects outliers, and converts the data into a format suitable for the AI ​​model. For example, it treats "age" as a numeric value and encodes "gender" as a categorical value.

[0704] Output: Preprocessed data is produced and can be input into subsequent processing steps.

[0705] Step 4: Input to the diagnostic model

[0706] Input: Preprocessed data (profile information, question answers, health data).

[0707] Specific operation: The server generates a feature set from the preprocessed data and creates a prompt based on this. The prompt generated is "28-year-old female, sales, high heart rate variability, light sleeper."

[0708] Data processing and output: The generated prompt sentences are input into a generative AI model, which analyzes them and diagnoses the user's mental health status.

[0709] Output: The generative AI model outputs a diagnosis of the user's mental health status.

[0710] Step 5: Generate diagnostic results

[0711] Input: Diagnostic results from a generative AI model.

[0712] Specific operation: The server converts the diagnosis results received from the generative AI model into a format that is easy for humans to understand. For example, it generates a message saying, "Mild depression is suspected."

[0713] Output: The interpreted diagnostic results are generated in a format that can be presented to the user.

[0714] Step 6: Delivering results

[0715] Input: Interpreted diagnosis and recommended medical provider information.

[0716] Specific operation: The server packages the diagnosis results and information on recommended medical institutions and sends them back to the terminal, which receives them and displays them to the user.

[0717] Data processing and output: For example, generating a message saying, "You are suspected of having mild depression. Here is the nearest clinic," and providing a link to make an online appointment.

[0718] Output: The diagnosis and medical institution information are displayed to the user.

[0719] Step 7: Follow up

[0720] Input: diagnostic results and self-care resources.

[0721] What happens: The user makes an online appointment using the provided clinic information and also accesses self-care resources provided by the application (e.g., mindfulness exercises, stress management guides).

[0722] Data processing and output: The server monitors the user's usage and provides follow-up reminders and additional resources as needed. For example, a reminder to "re-diagnose in a week."

[0723] Output: User receives follow-up information and resources for ongoing mental health management.

[0724] (Application example 1)

[0725] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0726] Conventional mental health diagnostic systems only target individual data, and lack mechanisms for efficiently monitoring the mental health status of employees in large organizations such as corporations and providing early and appropriate support. This can result in employees with mental health problems not receiving appropriate support, leading to reduced productivity and a worsening working environment. Another challenge is the difficulty of providing appropriate support while ensuring employee privacy and data security.

[0727] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0728] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical institutions, and means for monitoring the mental health condition of company employees and providing appropriate support early when their mental condition is unstable. This makes it possible to efficiently monitor the mental health condition of company employees and provide appropriate support early.

[0729] "Profile information" is data that includes personal information such as a user's age, gender, and occupation.

[0730] "Answers to questions" are response data entered by the user in response to questions related to mental and physical conditions presented in the application.

[0731] "Smart devices" refers to wearable devices and smartphones that can collect health data such as a user's heart rate, exercise volume, and sleep patterns.

[0732] "Health data" refers to data that indicates a user's health status, such as heart rate, exercise volume, and sleep patterns.

[0733] "Preprocessing" is the process of removing incomplete data and outliers from collected data and converting it into a format suitable for generative AI models.

[0734] A "generative AI model" is an artificial intelligence model that analyzes input data and generates a specific output (in this case, a mental health diagnosis).

[0735] "Diagnosis" refers to assessing a user's mental health status based on the output results obtained by the generative AI model.

[0736] "Information about recommended medical institutions" refers to information including contact information and appointment links for appropriate medical institutions suggested to the user based on the diagnosis results.

[0737] The "measures for monitoring the mental health status of employees in companies" is a mechanism for continuously monitoring the mental health status of employees within a company and providing appropriate support early on if a risk is determined.

[0738] "Self-care resources" refers to information and tools that users can use to manage their own mental health based on their diagnosis.

[0739] This invention is a system that diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at an appropriate medical institution, and also has a function for monitoring the mental health of company employees. This system is mainly composed of a user terminal, a server, and a smart device. The operation of each component will be described below.

[0740] First, the user launches the application and enters their profile information (age, gender, occupation, etc.) and answers questions related to their current mood and physical condition. The device then sends this information to the server, which then stores the received data in a database.

[0741] Next, the user allows the application to access their health data, which allows the application to collect health data such as heart rate, exercise volume, and sleep patterns from the past week from their smart device (e.g., smartphone or wearable device) and send it to the server. The server then stores the received health data in a database.

[0742] The server retrieves saved profile information, question answers, and health data and performs data preprocessing to remove incomplete data and outliers. The preprocessed data is converted into a format suitable for the generative AI model. The server generates a feature set from this preprocessed data and inputs it into the generative AI model. The generative AI model analyzes the input data and diagnoses the user's mental health condition. The diagnosis result is interpreted and converted into an easy-to-understand format. The server packages the diagnosis result and information on recommended medical institutions and sends it to the device. The device displays the diagnosis result to the user and also provides hospital contact information and a link to make an appointment.

[0743] In addition, the system has a function to monitor the mental health status of company employees. Companies provide this application to their employees and have them regularly check their mental health status. Employees enter data, and if there is a mental risk based on the analysis results, appropriate support is provided early. This process incorporates mechanisms to ensure data security while protecting employee privacy.

[0744] As a specific example, a 28-year-old female user launches an application for the first time, inputs her age, gender, and the fact that she works in sales, and then answers questions such as whether she worries a lot or has a decreased appetite. This information is sent to the server, which stores it in a database. The user then wears the smartwatch and synchronizes her heart rate and sleep pattern data with the application. The server preprocesses the received health data and inputs it into a generative AI model. The AI ​​model analyzes the user's condition and diagnoses her with "mild depression." Along with the diagnosis, information about nearby clinics and a link to make an appointment are sent to the user's device. The user confirms the diagnosis and clinic information and makes an online appointment with the clinic.

[0745] Example prompt sentence:

[0746] User profile data:

[0747] Age: 30

[0748] Gender: Male

[0749] Occupation: Engineer

[0750] User response data:

[0751] Mood: Stressed

[0752] Appetite: decreased

[0753] Health Data:

[0754] Heart Rate: [70, 75, 68, 74, 72, 78, 80]

[0755] Sleep Pattern: [6, 5, 7, 6, 5, 6, 4]

[0756] Activity: [2, 2, 1, 3, 2, 1, 2]

[0757] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0758] Step 1:

[0759] Collection and transmission of user data

[0760] The user starts the application, inputs their profile information (age, gender, occupation, etc.), and answers questions related to their current mood and physical condition. The device then sends this input data to the server.

[0761] Input: Profile information, answers to questions

[0762] Output: Send data to the server

[0763] Specific behavior:

[0764] The user accesses the application and enters information. The device validates the input and sends it to the server.

[0765] Step 2:

[0766] Health data synchronization and transmission

[0767] Users grant the application permission to access their health data, which allows the application to collect health data such as heart rate, exercise volume, and sleep patterns from the past week from their smart device and send it to a server.

[0768] Input: Health data from smart devices

[0769] Output: Send health data to the server

[0770] Specific behavior:

[0771] The user synchronizes their smart device with the application, and the device collects health data for a specified period and sends it to the server.

[0772] Step 3:

[0773] Data Preprocessing

[0774] The server retrieves stored profile information, question answers, and health data, performs data preprocessing to remove incomplete data and outliers, and converts the preprocessed data into a format suitable for the generative AI model.

[0775] Input: Profile information, answers to questions, health data

[0776] Output: Preprocessed data

[0777] Specific behavior:

[0778] The server reads the incoming data, performs consistency checks, removes outliers and incomplete data, and normalizes the data for the generative AI model.

[0779] Step 4:

[0780] Data input and diagnosis for generative AI models

[0781] The server generates a feature set from the preprocessed data and inputs it into a generative AI model, which analyzes the input data and diagnoses the user's mental health status.

[0782] Input: Preprocessed data

[0783] Output: Mental health status diagnosis

[0784] Specific behavior:

[0785] The server extracts a set of features and inputs them into a generative AI model, which then performs diagnostic analysis and generates results.

[0786] Step 5:

[0787] Providing diagnostic results and medical institution information

[0788] The server interprets the diagnosis results, converts them into an easy-to-understand format, and packages and sends information about recommended medical institutions to the device.

[0789] Input: Diagnostic results from generative AI model

[0790] Output: Diagnosis results and medical institution information to be presented to the user

[0791] Specific behavior:

[0792] The server interprets the diagnosis results, formats them into text, and sends them to the device along with information on recommended medical institutions.

[0793] Step 6:

[0794] Notification of results and follow-up with users

[0795] The device displays the diagnosis results and information on recommended medical institutions to the user, who can then check the results and make an appointment with a medical institution if necessary.

[0796] Input: Diagnosis results and medical institution information

[0797] Output: User presentation and follow-up action

[0798] Specific behavior:

[0799] The device displays the received information to the user, who then views the diagnostic results and clicks on the online reservation link.

[0800] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0801] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical attention at appropriate medical institutions. This system incorporates an emotion engine that recognizes the user's emotions. The program processing and operation of this system are explained below in natural language.

[0802] System configuration

[0803] This system mainly consists of the user's terminal, server, emotion engine, and smart device. The operation of each component is as follows.

[0804] Explanation of program processing

[0805] 1. Collection of User Data

[0806] User: Launches the application, enters profile information (age, gender, occupation, etc.), and answers questions related to current mood and physical condition.

[0807] Device: Sends the entered profile information and answers to questions to the server.

[0808] Server: Stores the received data in a database.

[0809] 2. Health data synchronization

[0810] User: Allows the application to access health data, which is then collected from smart devices (e.g., smartphones, wearable devices).

[0811] Device: Health data such as heart rate, exercise volume, and sleep patterns from the past week is collected from the smart device and sent to the server.

[0812] Server: Stores the received health data in a database.

[0813] 3. Collecting Emotional Data

[0814] User: The emotion engine recognizes the user's emotions based on their actions and inputs in the application.

[0815] Emotion engine: Analyzes user text input and other interactions to generate emotion data.

[0816] Terminal: Sends the generated emotion data to the server.

[0817] Server: Stores emotion data in a database.

[0818] 4. Data Preprocessing

[0819] Server: Retrieves stored profile information, question answers, health data, and emotional data.

[0820] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[0821] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[0822] 5. Input to the diagnostic model

[0823] Server: Generates a feature set from the preprocessed data, including, for example, average heart rate, standard deviation, sleep duration, answer scores to questions about specific emotions, and emotional data.

[0824] Server: Inputs the generated feature set into the generative AI model.

[0825] 6. Generating diagnostic results

[0826] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0827] Server: Customize the diagnosis results based on the emotional data recognized by the emotion engine. For example, if the emotional data indicates strong anxiety, include that information in the diagnosis results.

[0828] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[0829] 7. Distribution of Results

[0830] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[0831] Server: Sends packaged information to the terminal.

[0832] On the device: The diagnosis results are displayed to the user. For example, a message like "You are experiencing moderate depression. We recommend the following clinics" is displayed along with the clinic's contact information.

[0833] 8. Follow-up

[0834] User: Based on the diagnosis, the user can use the displayed clinic information to book an appointment online or contact the clinic directly.

[0835] User: If necessary, consult the self-care resources provided by the application (e.g., relaxation techniques, exercise programs, etc.).

[0836] Specific examples

[0837] For example, a 28-year-old male user launches an application for the first time and enters his profile information.

[0838] User: Enters age, gender, and whether they work in sales, then answers questions such as, "Have you often felt depressed in the last two weeks?"

[0839] Terminal: Sends this information to the server, which stores it in a database.

[0840] User: Then wears the smartwatch and syncs heart rate and sleep pattern data to the application.

[0841] Server: Preprocesses the received health data and inputs it into the generative AI model.

[0842] Emotion engine: Generates emotional data from user text input and other interactions.

[0843] Server: The generated emotion data is input into the AI ​​model to generate a diagnosis result, such as "The user is in a state of mild depression and has a high level of anxiety."

[0844] Server: Along with the diagnosis results, it sends information about nearby clinics and a link to make an appointment to the user's device.

[0845] Terminal: Displays the diagnosis results and allows the user to make an appointment based on the clinic information.

[0846] In this way, by analyzing various information, including the user's emotional data, it is possible to provide a more accurate diagnosis and help the user receive treatment at an appropriate medical institution. This system could be an effective means of promoting early detection and appropriate treatment of potential depression patients.

[0847] The processing flow will be explained below.

[0848] Step 1:

[0849] User: Launches the application and enters profile information (age, gender, occupation, etc.) on the initial registration screen.

[0850] Terminal: Receives the entered profile information and sends it to the server.

[0851] Server: Stores the received profile information in a database.

[0852] Step 2:

[0853] User: Next, the user answers questions in the application. For example, they answer "yes" or "no" to questions like "Have you been feeling depressed for the past two weeks?"

[0854] Terminal: Receives the user's response and sends it to the server.

[0855] Server: Stores the received answers in a database.

[0856] Step 3:

[0857] User: Follows the instructions in the application and allows collection of health data from smartphones and wearable devices.

[0858] Device: Collects health data from your smart device over the past week (heart rate, activity, sleep patterns, etc.).

[0859] Terminal: Sends collected health data to the server.

[0860] Server: Stores the received health data in a database.

[0861] Step 4:

[0862] Emotion engine: Analyzes user text input and other interactions with the application to generate emotion data.

[0863] Terminal: Sends the generated emotion data to the server.

[0864] Server: Stores emotion data in a database.

[0865] Step 5:

[0866] Server: Retrieves stored profile information, question answers, health data, and emotion data.

[0867] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[0868] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[0869] Step 6:

[0870] Server: Generates a feature set from the preprocessed data, including, for example, average heart rate, standard deviation, sleep duration, answer scores to questions about specific emotions, and emotional data.

[0871] Server: Inputs the generated feature set into the generative AI model.

[0872] Step 7:

[0873] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0874] Server: Customize the diagnosis results based on the emotional data recognized by the emotion engine. For example, if the emotional data indicates strong anxiety, include that information in the diagnosis results.

[0875] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[0876] Step 8:

[0877] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[0878] Server: Sends packaged information to the terminal.

[0879] Step 9:

[0880] On the device: Display the diagnosis and related information to the user. For example, display a message such as "You are experiencing moderate depression. We recommend the following clinics" along with the clinic's contact information.

[0881] User: Review the diagnosis and click on the recommended clinic contact and booking link to book an appointment.

[0882] Users: You can also consult self-care resources within the application (relaxation techniques, exercise programs, etc.).

[0883] Through this series of steps, users can not only receive an accurate diagnosis of their mental health condition, but also receive customized diagnostic results and recommended medical information based on their emotional data.

[0884] Example 2

[0885] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0886] In recent years, mental health problems have been increasing, and early detection of depression and anxiety disorders in particular is becoming increasingly important. However, it is difficult for users to accurately recognize their own mental state and seek medical attention at an appropriate medical institution. Furthermore, conventional systems are primarily limited to in-hospital diagnosis and are unable to comprehensively analyze diverse data from daily life. Therefore, there is a need for a system that can comprehensively analyze a user's emotions and health data, accurately diagnose their mental health state, and, if necessary, encourage them to seek medical attention at an appropriate medical institution.

[0887] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0888] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for analyzing the user's text input and interactions and generating emotion data, means for preprocessing the collected and generated data and converting it into a format that can be input to the generative AI model, means for generating a feature set from the preprocessed data and inputting it into the generative AI model, means for the generative AI model to analyze the input data and diagnose the user's mental health condition, and means for providing the user with the diagnosis results and information on recommended medical institutions. This makes it possible to comprehensively analyze various data obtained from the user's daily life, more accurately and quickly diagnose the user's mental health condition, and encourage the user to visit an appropriate medical institution.

[0889] "Profile information" refers to basic personal information such as a user's age, gender, and occupation.

[0890] "Answers to questions" are response information provided by a user in response to questions related to mental health conditions presented by the system.

[0891] A "smart device" is an electronic device that can collect health data such as heart rate, activity, and sleep patterns.

[0892] "Health data" is information about a user's physical condition, such as their heart rate, activity level, and sleep patterns.

[0893] "Emotion data" refers to information about emotions recognized and analyzed by the emotion engine from user text input and interactions.

[0894] "Data preprocessing" is the process of organizing and normalizing collected profile information, answers to questions, health data, and emotion data, and removing missing or outliers.

[0895] A "feature set" is a collection of data points that are important for analysis and are derived from preprocessed data.

[0896] A "generative AI model" is an artificial intelligence model that uses preprocessed data to diagnose a user's mental health status.

[0897] "Diagnosis results" are assessments and conclusions about the user's mental health status that are displayed as a result of the generative AI model's analysis.

[0898] "Information about recommended medical institutions" refers to information about appropriate medical institutions and specialists provided to users based on diagnostic results.

[0899] This invention is a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at appropriate medical institutions. The system is composed of a user terminal, a server, an emotion engine, and a smart device.

[0900] System configuration

[0901] Users launch the application using a device such as a smartphone or tablet. When launching the application, the user enters profile information (age, gender, occupation, etc.) and answers questions related to their mental health. In addition, the user allows the application to access their smart device (smartphone, smartwatch, etc.) to collect health data, such as heart rate, exercise volume, and sleep patterns.

[0902] Data collection and analysis

[0903] The server collects profile information, answers to questions, health data, and emotion data from the user's device and stores them in a centralized database. The emotion engine analyzes text input and other interactions the user has within the application to generate emotion data. For example, if a user inputs "I've been feeling tired lately," the emotion engine will analyze the emotions of "tired" and "anxiety."

[0904] Data preprocessing and feature generation

[0905] The server preprocesses all collected data and removes missing or outliers. From the preprocessed data, a feature set is generated to be input into the generative AI model. The feature set includes average heart rate, exercise volume, sleep duration, and emotion score. The generative AI model uses this feature set to diagnose the user's mental health condition.

[0906] Diagnostic results and medical institution recommendations

[0907] The generative AI model analyzes the user's data and generates a diagnosis, which includes specific descriptions such as "moderate depression" or "high anxiety level." Based on this diagnosis, the server recommends information about appropriate nearby medical institutions and reservation links. The diagnosis and medical institution information are sent to the user's device and displayed to the user through the application.

[0908] Specific examples

[0909] For example, a 28-year-old male user launches the application for the first time, enters his age, gender, and job title in sales, and then answers "Yes" to the question, "Have you often felt depressed in the last two weeks?" When the user wears a smartwatch and syncs it with the application, data on his heart rate and sleep patterns over the past week is sent to the server. The emotion engine analyzes "fatigue" and "anxiety" from the text entered by the user and generates emotion data.

[0910] The server preprocesses this data, generates a feature set, and inputs it into the generative AI model. The diagnosis results indicate that the user is in a state of mild depression and has a high level of anxiety, and information about nearby clinics and a link to make an appointment are sent to the user's device. The user can then make an appointment based on this information.

[0911] Prompt Sentence Examples

[0912] 1. Introducing a sentiment analysis tool using natural language processing algorithms

[0913] 2. Write a program to collect health data from smart devices

[0914] 3. Explain the architecture of the health checkup system based on profile information and emotion data.

[0915] By analyzing a variety of user data, this system can more accurately diagnose mental health conditions and encourage people to seek medical attention at appropriate medical institutions.

[0916] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0917] Step 1:

[0918] User Data Collection

[0919] User: Launches the application, enters profile information (e.g., age, gender, occupation), and answers questions related to mental health.

[0920] Input: Age, Gender, Occupation, Answer to Question

[0921] Action: A user enters information into an application and presses the submit button.

[0922] Output: Input information is saved on the device and sent to the server.

[0923] Device: Sends this information to the server.

[0924] Input: Information entered by the user

[0925] How it works: The device temporarily stores the user's input and sends an HTTP request to the server.

[0926] Output: Server receives user profile data and question answers

[0927] Server: Stores the received data in a database.

[0928] Input: User profile data and question answers

[0929] How it works: The server executes an SQL query to the database and saves the data.

[0930] Output: User data stored in the database

[0931] Step 2:

[0932] Health Data Sync

[0933] User: Allows the application to access health data.

[0934] Input: User action (access permission)

[0935] What happens: The user enables health data permission in the application settings.

[0936] Output: The permission settings change takes effect.

[0937] Device: Collects health data such as heart rate, exercise volume, and sleep patterns from smart devices and sends it to a server.

[0938] Input: Health data from smart devices

[0939] Operation: The terminal acquires data from the smart device and sends it to the server.

[0940] Output: Health data sent to the server

[0941] Server: Stores the received health data in a database.

[0942] Input: Health data

[0943] What it does: The server executes an SQL query to store in the database

[0944] Output: Health data stored in a database

[0945] Step 3:

[0946] Collecting Emotional Data

[0947] User: Enters text and otherwise interacts with the application.

[0948] Input: User text input or operation

[0949] Action: User enters text and presses submit, or interacts with a button within the app

[0950] Output: Text entered into the device and interaction data

[0951] Emotion engine: Analyzes text input and operation data to generate emotion data.

[0952] Input: User text input and operational data

[0953] How it works: The emotion engine uses natural language processing algorithms to analyze the input and generate emotion labels.

[0954] Output: Emotion data

[0955] Terminal: Sends the generated emotion data to the server.

[0956] Input: Emotion data generated by the emotion engine

[0957] Behavior: Issues an HTTP request to send emotion data to the server.

[0958] Output: Emotion data sent to the server

[0959] Server: Stores emotion data in a database.

[0960] Input: Emotion data

[0961] What it does: The server executes an SQL query to store in the database

[0962] Output: Emotion data stored in a database

[0963] Step 4:

[0964] Data Preprocessing

[0965] Server: Retrieves stored profile information, question answers, health data, and emotional data.

[0966] Input: Information retrieved from the database

[0967] What happens: The server executes a retrieval SQL query to retrieve the data.

[0968] Output: Multiple datasets retrieved for processing on the server side

[0969] Server: Performs data cleaning to remove incomplete data and outliers.

[0970] Input: Retrieved data

[0971] Action: Detects and filters data anomalies

[0972] Output: Cleaned dataset

[0973] Server: Performs data normalization.

[0974] Input: Cleaned dataset

[0975] What it does: It applies a data normalization algorithm to convert the data into a uniform format.

[0976] Output: Normalized dataset

[0977] Step 5:

[0978] Input to the diagnostic model

[0979] Server: Generates a feature set from the preprocessed data.

[0980] Input: Preprocessed dataset

[0981] Action: Runs an algorithm to extract features from the data.

[0982] Output: Feature set

[0983] Server: Inputs the feature set into the generative AI model.

[0984] Input: Feature set

[0985] Operation: Convert the feature set into an input format for the AI ​​model and input it into the model.

[0986] Output: Features input to the AI ​​model

[0987] Step 6:

[0988] Generating diagnostic results

[0989] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[0990] Input: Feature set

[0991] How it works: The AI ​​model performs analysis and generates a diagnosis.

[0992] Output: Diagnostic results

[0993] Server: Converts diagnostic results into an easy-to-understand format.

[0994] Input: Diagnostic results

[0995] What it does: Converts diagnostic results into a text format for the user

[0996] Output: Diagnostic results for the user

[0997] Step 7:

[0998] Results distribution

[0999] Server: Packages the diagnosis results and information on recommended medical institutions.

[1000] Input: Diagnosis results, medical institution information

[1001] How it works: Combines diagnostic results and medical information into a single package

[1002] Output: Package information

[1003] Server: Sends packaged information to the device.

[1004] Input: Package information

[1005] Action: Sends package information to the device via HTTP request

[1006] Output: Package information received on the device

[1007] Terminal: Display diagnostic results to the user.

[1008] Input: Package information

[1009] Behavior: Updates the UI in the application to display diagnosis results and medical provider information.

[1010] Output: Diagnosis results and medical institution information displayed to the user

[1011] Step 8:

[1012] Follow-up

[1013] User: Based on the diagnosis, the user can use the displayed medical information to book an appointment online or contact the clinic directly.

[1014] Input: Diagnosis results, medical institution information

[1015] What happens: User clicks on clinic appointment link and fills out appointment form

[1016] Output: Clinic appointment completed

[1017] Users: Consult the self-care resources provided by the app as needed.

[1018] Input: Diagnostic results, self-care resources

[1019] Action: User views and practices self-care resources

[1020] Output: User self-care practices

[1021] (Application example 2)

[1022] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1023] Many systems exist that accurately diagnose mental health conditions and encourage potential depression patients to seek medical treatment at appropriate medical institutions. However, there are insufficient means to effectively collect and analyze users' emotional data and present it to users in a visually and intuitive manner. Furthermore, there are also insufficient means to analyze the collected data from multiple angles and provide users with information on improvement measures and medical facilities. This has led to issues that make it difficult for users to understand the diagnosis results and take appropriate action.

[1024] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1025] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving biometric data from the smart device, means for collecting and receiving the user's emotional data through an emotion recognition engine, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical facilities, and means for visually presenting the diagnosis results to the user through a head-mounted display. This enables the collection and analysis of multifaceted data, including the user's emotional data, and enables the user to intuitively and visually receive diagnosis results, promoting consultation at an appropriate medical institution.

[1026] "User" means an individual who uses this system and provides their profile information and biometric data.

[1027] "Profile Information" means basic personal information about a user, such as age, gender, and occupation.

[1028] "Question Answers" are responses that users make to mental health-related questions posed within the system.

[1029] "Smart device" is a general term for devices that can measure and collect a user's biometric information (e.g., heart rate, exercise volume, sleep patterns).

[1030] "Biometric data" means physiological data of a user (e.g., heart rate, sleep patterns) collected from a smart device.

[1031] An "emotion recognition engine" is software or algorithms that recognize emotions from user text input, voice, gestures, etc. and generate data.

[1032] "Preprocessing collected data" refers to data processing measures to convert collected profile information, biometric data, and emotional data into a format that can be input into a generative AI model.

[1033] A "generative AI model" is a machine learning or artificial intelligence model that assesses and diagnoses a user's mental health based on input data.

[1034] "Means for diagnosing a user's mental health condition" means means for analyzing collected and pre-processed data using a generative AI model to assess a user's mental health condition.

[1035] "Medical facility" is a general term for medical institutions such as hospitals and clinics where users can receive mental health consultations.

[1036] A "head-mounted display" is a display device that provides visual information when worn by a user.

[1037] This invention is a system that uses a head-mounted display (HMD) to diagnose a user's mental health condition and encourage them to seek appropriate medical advice. The system combines the user's profile information, answers to questions, biometric data from a smart device, and emotion data obtained using an emotion recognition engine to provide a diagnosis based on a generative AI model.

[1038] Hardware / Software used

[1039] Hardware:

[1040] Head-mounted display (HMD): General-purpose HMD device

[1041] Smart devices: devices that collect biometric data such as heart rate, activity, and sleep patterns (e.g., wearable devices)

[1042] software:

[1043] Emotion Recognition Engine: An engine that recognizes emotions from user text input, voice, and gestures.

[1044] Database: Firebase Firestore or other cloud databases

[1045] Generative AI model: A machine learning model for assessing mental health status

[1046] Web framework: Server-side framework such as Flask

[1047] System operation explanation

[1048] User Data Collection:

[1049] The user puts on the HMD device and launches the application. They input their profile information through the HMD interface using voice input and gestures. They also answer questions related to their mood and physical condition. This input data is sent from the HMD to the server and stored in a database.

[1050] Health Data Sync:

[1051] Users use smart devices (e.g., wearable devices) to collect biometric data such as heart rate, exercise volume, and sleep patterns. This data is then sent to a server via the HMD and stored in a database.

[1052] Emotion data collection:

[1053] The emotion recognition engine recognizes emotions from the user's text or voice input and generates emotion data, which is also sent to the server and stored in a database.

[1054] Data preprocessing:

[1055] The server preprocesses the collected profile information, biometric data, and emotion data, converting them into a format that can be input into the generative AI model, specifically by performing data cleaning and normalization.

[1056] Inputs to generative AI models and diagnosis:

[1057] The server inputs the preprocessed data into a generative AI model to diagnose the user's mental health status, and the results are converted into an intuitively understandable format.

[1058] Delivery of diagnostic results:

[1059] The server packages the diagnosis results and recommended medical facility information and presents them visually to the user through the HMD. The diagnosis results include contact information for nearby clinics and specialists.

[1060] Follow-up:

[1061] Users can review their diagnosis and use the HMD interface to find nearby clinics, schedule appointments online if needed, and access self-care resources (e.g., relaxation techniques, exercise programs).

[1062] Examples of specific examples and prompts

[1063] Examples:

[1064] A 28-year-old male user uses the app for the first time using a general-purpose HMD device. He registers his profile information by voice input, and then synchronizes his biometric data with his smart device. The emotion recognition engine generates emotion data based on the text input, and the diagnosis is "mild depression." The user then checks the information of nearby specialists through the HMD and makes an appointment.

[1065] Example prompt sentence:

[1066] "What steps would an HMD application that collects and analyzes a user's profile information and health data to diagnose their mental state take? Please explain using a concrete example."

[1067] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1068] Step 1:

[1069] User Data Collection

[1070] The user puts on the head-mounted display and starts the application. They enter their profile information (age, gender, occupation, etc.) and answers to questions related to their mood and physical condition using voice input or gestures. This input data is sent from the device to the server and stored in a database.

[1071] Input: Profile information, answers to questions

[1072] Data calculation: voice recognition, data format conversion

[1073] Output: Profile information and question and answer data (sent to server)

[1074] Step 2:

[1075] Health Data Sync

[1076] Users collect biometric data such as heart rate, exercise volume, and sleep patterns from the past week from their smart devices (e.g., wearable devices). This data is sent from the device to a server and stored in a database.

[1077] Input: Heart rate data, exercise data, sleep pattern data

[1078] Data processing: Data synchronization, data format conversion

[1079] Output: Biometric data (sent to server)

[1080] Step 3:

[1081] Collecting Emotional Data

[1082] The emotion recognition engine analyzes the user's text and voice input to generate emotion data, which is then sent from the device to a server and stored in a database.

[1083] Input: User text input, voice input

[1084] Data Computation: Emotion Recognition, Data Generation

[1085] Output: Emotion data (sent to server)

[1086] Step 4:

[1087] Data Preprocessing

[1088] The server takes the collected profile information, question responses, biometric data, and emotional data, and performs data cleaning and normalization, converting the data into a format that can be input into a generative AI model.

[1089] Input: Profile information, question and answer data, biometric data, emotional data

[1090] Data calculations: data cleaning, data normalization

[1091] Output: Preprocessed data

[1092] Step 5:

[1093] Input to generative AI models and diagnostics

[1094] The server inputs the preprocessed data into a generative AI model to diagnose the user's mental health status, and the results are converted into an easy-to-understand format.

[1095] Input: Preprocessed data

[1096] Data calculation: feature extraction, model inference

[1097] Output: Diagnostic results

[1098] Step 6:

[1099] Distribution of diagnostic results

[1100] The server packages the diagnosis and recommended medical facility information and presents it visually to the user through the HMD, including contact information for clinics and specialists.

[1101] Input: Diagnosis results, medical facility information

[1102] Data Calculation: Data Packaging

[1103] Output: Visual presentation data (sent to HMD)

[1104] Step 7:

[1105] Follow-up

[1106] Users can check their diagnosis and medical facility information, make appointments online if necessary, and also view and use self-care resources (relaxation techniques, exercise programs, etc.).

[1107] Input: diagnosis results, medical facility information, self-care resource information

[1108] Output: Reservation information, self-care resource usage status

[1109] In this way, the system collects and analyzes data from multiple angles, provides users with intuitive and visual diagnostic results, and encourages them to seek medical attention at an appropriate medical institution, thereby achieving comprehensive mental health support.

[1110] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1111] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1112] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1113] [Third embodiment]

[1114] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1115] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1116] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1117] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1118] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1119] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1120] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1121] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1122] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1123] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1124] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1125] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1126] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical attention at an appropriate medical institution. The processing and operation of the program of this system will be explained below in natural language.

[1127] System configuration

[1128] This system mainly consists of a user terminal, a server, and a smart device. The operation of each component is as follows:

[1129] Explanation of program processing

[1130] 1. Collection of User Data

[1131] User: Launches the application, enters profile information (age, gender, occupation, etc.), and answers questions related to current mood and physical condition.

[1132] Device: Sends the entered profile information and answers to questions to the server.

[1133] Server: Stores the received data in a database.

[1134] 2. Health data synchronization

[1135] User: Allows the application to access health data, which is then collected from smart devices (e.g., smartphones, wearable devices).

[1136] Device: Health data such as heart rate, exercise volume, and sleep patterns from the past week is collected from the smart device and sent to the server.

[1137] Server: Stores the received health data in a database.

[1138] 3. Data Preprocessing

[1139] Server: Retrieves stored profile information, question answers, and health data, filtering out incomplete data and outliers.

[1140] Server: Normalizes the data and converts it into a format suitable for AI models.

[1141] 4. Input to the diagnostic model

[1142] Server: Generates a feature set from the preprocessed data and inputs it into the generative AI model.

[1143] 5. Generating diagnostic results

[1144] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1145] Server: Interprets the diagnostic results and converts them into an understandable format.

[1146] 6. Distribution of Results

[1147] Server: Packages the diagnosis results and information on recommended medical institutions.

[1148] Server: Sends packaged information to the terminal.

[1149] On the device: The diagnosis results are displayed to the user, along with hospital contact information and a link to make an appointment.

[1150] 7. Follow-up

[1151] User: Based on the diagnosis, the user can use the displayed clinic information to book an appointment online or contact the clinic directly.

[1152] Users: Refer to the self-care resources provided by the app as needed.

[1153] Specific examples

[1154] For example, suppose a 28-year-old female user launches an application for the first time.

[1155] Device: The user enters their age, gender, and whether they work in sales, and then answers questions such as whether they worry a lot or have a decreased appetite.

[1156] Terminal: Sends this information to the server, which stores it in a database.

[1157] User: Then wears the smartwatch and syncs heart rate and sleep pattern data to the application.

[1158] Server: Preprocesses the received health data and inputs it into the generative AI model.

[1159] Server: The AI ​​model analyzes the user's situation and diagnoses them as having "mild depression."

[1160] Server: Along with the diagnosis results, it sends information about nearby clinics and a link to make an appointment to the user's device.

[1161] User: Checks diagnosis results and clinic information and makes online appointment with clinic.

[1162] In this way, users can receive an accurate diagnosis and smoothly consult with an appropriate medical institution. This system could be an effective means of promoting early detection and appropriate treatment for potential depression patients.

[1163] The processing flow will be explained below.

[1164] Step 1:

[1165] User: Launches the application and enters profile information (age, gender, occupation, etc.) on the initial registration screen.

[1166] Terminal: Receives the entered profile information and sends it to the server.

[1167] Server: Stores the received profile information in a database.

[1168] Step 2:

[1169] User: Next, the user answers questions in the application. For example, they answer "yes" or "no" to questions like "Have you been feeling depressed for the past two weeks?"

[1170] Terminal: Receives the user's response and sends it to the server.

[1171] Server: Stores the received answers in a database.

[1172] Step 3:

[1173] User: Follows the instructions in the application and allows collection of health data from smartphones and wearable devices.

[1174] Device: Collects health data from your smart device over the past week (heart rate, activity, sleep patterns, etc.).

[1175] Terminal: Sends collected health data to the server.

[1176] Server: Stores the received health data in a database.

[1177] Step 4:

[1178] Server: Retrieves stored profile information, question answers, and health data.

[1179] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[1180] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[1181] Step 5:

[1182] Server: Generates a set of features from the preprocessed data, including, for example, the mean heart rate, standard deviation, sleep duration, and scores for answers to questions about specific emotions.

[1183] Server: Inputs the generated feature set into the generative AI model.

[1184] Step 6:

[1185] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1186] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[1187] Step 7:

[1188] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[1189] Server: Sends packaged information to the terminal.

[1190] Step 8:

[1191] On the device: Display the diagnosis and related information to the user. For example, display a message such as "You are experiencing moderate depression. We recommend the following clinics" along with the clinic's contact information.

[1192] User: Review the diagnosis and click on the recommended clinic contact and booking link to book an appointment.

[1193] Users: You can also consult self-care resources within the application (relaxation techniques, exercise programs, etc.).

[1194] Through this series of steps, users are able to accurately diagnose their mental health condition and are encouraged to seek medical attention from an appropriate medical institution.

[1195] Example 1

[1196] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1197] Mental health issues are a serious social challenge in modern society. Early detection is particularly important for mental illnesses such as depression, and consultation at an appropriate medical institution is required. However, many people do not have the means to accurately assess their own mental health status, and access to appropriate medical institutions is limited. The objective of the present invention is to provide a system that accurately diagnoses a user's mental health status and encourages potential depression patients to seek treatment at an appropriate medical institution.

[1198] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1199] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for generating prompt sentences, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical institutions, and means for validating the received data and detecting and correcting missing data and outliers, thereby enabling an accurate diagnosis of the user's mental health condition and promoting consultation at an appropriate medical institution.

[1200] "Means for receiving profile information and answers to questions as input from the user" is a function for collecting personal information and answer data to questions that the user has entered into the application and sending it to the server.

[1201] "Means for collecting and receiving health data from smart devices" refers to a function for transmitting health data such as heart rate, exercise volume, and sleep patterns obtained from smartphones and wearable devices to a server.

[1202] "Means for preprocessing collected data and converting it into a format that can be input into a generative AI model" refers to a function for formalizing and standardizing data obtained from users and smart devices and converting it into a form that can be used by an AI model.

[1203] "Means for inputting preprocessed data into a generative AI model and diagnosing the user's mental health state" refers to a function for inputting preprocessed data into a generative AI model and using AI to evaluate and diagnose the user's mental state.

[1204] "Means for providing users with information on diagnostic results and recommended medical institutions" refers to a function for notifying users of the diagnostic results obtained by AI and information on recommended medical institutions based on those results.

[1205] "Means for validating received data and detecting and correcting missing data and abnormal values" refers to a function for checking the accuracy of data sent to the server, finding missing or abnormal data, and supplementing or correcting it.

[1206] The "means for generating prompt sentences" is a function for creating specific instruction sentences (prompt sentences) in a format that can be analyzed by the generative AI model based on preprocessed data.

[1207] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at an appropriate medical institution. This system is mainly composed of a user terminal, a server, and a smart device.

[1208] User Data Collection

[1209] First, the user launches the application and enters profile information such as age, gender, and occupation. They also answer questions about their mood and physical condition. For example, they answer questions such as, "Have you been worrying a lot lately?" or "Have you lost your appetite?" The information entered by the user is sent to the server by the device and stored in a database.

[1210] Health Data Sync

[1211] Next, the user grants the application permission to access health data from their smart device (e.g., smartphone, smartwatch). This allows the application to collect health-related data such as heart rate, exercise volume, and sleep patterns from the past week and send it to the server. The health data is also stored in a database.

[1212] Data Preprocessing

[1213] The server retrieves the stored profile information, question answers, and health data, and detects and corrects incomplete data and outliers. Specifically, it performs data validation to detect and correct missing data and outliers. It also normalizes the data and converts it into a format that can be input into the generative AI model. For example, it standardizes numerical data and encodes categorical data.

[1214] Input to the diagnostic model

[1215] Next, a feature set is generated from the preprocessed data, and a prompt sentence is created. For example, the generated features might be "28-year-old female, sales position, high heart rate variability, light sleep." This prompt sentence is input into a generative AI model to diagnose the user's mental health condition.

[1216] Generating diagnostic results

[1217] The generative AI model evaluates the user's mental health state based on the input feature set and generates a diagnosis such as "mild depression" or "normal." The diagnosis result is interpreted by the server and converted into a format that is easy for humans to understand. Specifically, a message is generated saying, "Based on your current health data, you are suspected of having mild depression."

[1218] Results distribution

[1219] The server sends this information back to the user's device, which then displays the diagnosis, including contact details for the specific hospital and a link to make an online appointment.

[1220] Follow-up

[1221] Based on the diagnosis results, the user can use the displayed clinic information to make an appointment online or contact the clinic directly. If necessary, the user can also refer to the self-care resources provided by the application. The server monitors the user's usage and provides follow-up reminders and additional resources as needed.

[1222] Specific examples

[1223] For example, when a 28-year-old female user launches an application for the first time, the following scenario occurs:

[1224] Users enter profile information such as age, gender, and occupation, and answer questions about concerns and loss of appetite.

[1225] This information is sent from the terminal to the server and stored in a database.

[1226] The user then wears the smartwatch and syncs data on their heart rate and sleep patterns with the application.

[1227] The server preprocesses the received health data and inputs it into a generative AI model.

[1228] The AI ​​model analyzes the patient and diagnoses him as having "mild depression."

[1229] The server sends the diagnosis results along with information about nearby clinics and an online reservation link to the user's terminal.

[1230] The user checks the diagnosis results and clinic information and makes an online appointment with the clinic.

[1231] Prompt Sentence Examples

[1232] "A 28-year-old woman has recently been feeling anxious and has experienced a decrease in her appetite. Please assess this user's mental health status by taking into account her heart rate data and sleep patterns over the past week."

[1233] By using this system, users can receive an accurate diagnosis and smoothly consult with an appropriate medical institution. This system is an effective means of promoting early detection of potential depression patients and appropriate treatment.

[1234] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1235] Step 1: Collect user data

[1236] Input: Profile information (such as age, gender, and occupation) and answers to questions you enter into the application (e.g., "Have you been feeling a lot of anxiety lately?", "Have you lost your appetite?", etc.).

[1237] How it works: The user launches the application on their smartphone or PC and follows the instructions to provide the required profile information and answer health questions.

[1238] Data processing and output: These input data are converted into the appropriate format within the terminal and sent to the server. The terminal performs a data consistency check to ensure there are no inconsistencies.

[1239] Output: The user data, whose integrity has been verified, is sent to the server, which stores it in a database.

[1240] Step 2: Syncing health data

[1241] Input: Health data such as heart rate, activity, and sleep patterns obtained from smart devices (smartphones, wearable devices, etc.).

[1242] What it does: The user grants the application permission to access their smart device. The application periodically syncs with the smart device and collects health data from the past week.

[1243] Data processing and output: The device normalizes the acquired health data and performs consistency checks before sending it to the server. For example, it checks whether the heart rate is too high.

[1244] Output: The health data, whose integrity has been verified, is sent to the server, which stores it in a database.

[1245] Step 3: Data Preprocessing

[1246] Input: Saved profile information, question answers and health data.

[1247] What it does: The server retrieves these data from the database, detects and removes incomplete data and outliers, and performs other tasks, such as standardizing numeric data and encoding categorical data.

[1248] Data processing and output: The server completes incomplete data, corrects outliers, and converts the data into a format suitable for the AI ​​model. For example, it treats "age" as a numeric value and encodes "gender" as a categorical value.

[1249] Output: Preprocessed data is produced and can be input into subsequent processing steps.

[1250] Step 4: Input to the diagnostic model

[1251] Input: Preprocessed data (profile information, question answers, health data).

[1252] Specific operation: The server generates a feature set from the preprocessed data and creates a prompt based on this. The prompt generated is "28-year-old female, sales, high heart rate variability, light sleeper."

[1253] Data processing and output: The generated prompt sentences are input into a generative AI model, which analyzes them and diagnoses the user's mental health status.

[1254] Output: The generative AI model outputs a diagnosis of the user's mental health status.

[1255] Step 5: Generate diagnostic results

[1256] Input: Diagnostic results from a generative AI model.

[1257] Specific operation: The server converts the diagnosis results received from the generative AI model into a format that is easy for humans to understand. For example, it generates a message saying, "Mild depression is suspected."

[1258] Output: The interpreted diagnostic results are generated in a format that can be presented to the user.

[1259] Step 6: Delivering results

[1260] Input: Interpreted diagnosis and recommended medical provider information.

[1261] Specific operation: The server packages the diagnosis results and information on recommended medical institutions and sends them back to the terminal, which receives them and displays them to the user.

[1262] Data processing and output: For example, generating a message saying, "You are suspected of having mild depression. Here is the nearest clinic," and providing a link to make an online appointment.

[1263] Output: The diagnosis and medical institution information are displayed to the user.

[1264] Step 7: Follow up

[1265] Input: diagnostic results and self-care resources.

[1266] What happens: The user makes an online appointment using the provided clinic information and also accesses self-care resources provided by the application (e.g., mindfulness exercises, stress management guides).

[1267] Data processing and output: The server monitors the user's usage and provides follow-up reminders and additional resources as needed. For example, a reminder to "re-diagnose in a week."

[1268] Output: User receives follow-up information and resources for ongoing mental health management.

[1269] (Application example 1)

[1270] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1271] Conventional mental health diagnostic systems only target individual data, and lack mechanisms for efficiently monitoring the mental health status of employees in large organizations such as corporations and providing early and appropriate support. This can result in employees with mental health problems not receiving appropriate support, leading to reduced productivity and a worsening working environment. Another challenge is the difficulty of providing appropriate support while ensuring employee privacy and data security.

[1272] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1273] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical institutions, and means for monitoring the mental health condition of company employees and providing appropriate support early when their mental condition is unstable. This makes it possible to efficiently monitor the mental health condition of company employees and provide appropriate support early.

[1274] "Profile information" is data that includes personal information such as a user's age, gender, and occupation.

[1275] "Answers to questions" are response data entered by the user in response to questions related to mental and physical conditions presented in the application.

[1276] "Smart devices" refers to wearable devices and smartphones that can collect health data such as a user's heart rate, exercise volume, and sleep patterns.

[1277] "Health data" refers to data that indicates a user's health status, such as heart rate, exercise volume, and sleep patterns.

[1278] "Preprocessing" is the process of removing incomplete data and outliers from collected data and converting it into a format suitable for generative AI models.

[1279] A "generative AI model" is an artificial intelligence model that analyzes input data and generates a specific output (in this case, a mental health diagnosis).

[1280] "Diagnosis" refers to assessing a user's mental health status based on the output results obtained by the generative AI model.

[1281] "Information about recommended medical institutions" refers to information including contact information and appointment links for appropriate medical institutions suggested to the user based on the diagnosis results.

[1282] The "measures for monitoring the mental health status of employees in companies" is a mechanism for continuously monitoring the mental health status of employees within a company and providing appropriate support early on if a risk is determined.

[1283] "Self-care resources" refers to information and tools that users can use to manage their own mental health based on their diagnosis.

[1284] This invention is a system that diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at an appropriate medical institution, and also has a function for monitoring the mental health of company employees. This system is mainly composed of a user terminal, a server, and a smart device. The operation of each component will be described below.

[1285] First, the user launches the application and enters their profile information (age, gender, occupation, etc.) and answers questions related to their current mood and physical condition. The device then sends this information to the server, which then stores the received data in a database.

[1286] Next, the user allows the application to access their health data, which allows the application to collect health data such as heart rate, exercise volume, and sleep patterns from the past week from their smart device (e.g., smartphone or wearable device) and send it to the server. The server then stores the received health data in a database.

[1287] The server retrieves saved profile information, question answers, and health data and performs data preprocessing to remove incomplete data and outliers. The preprocessed data is converted into a format suitable for the generative AI model. The server generates a feature set from this preprocessed data and inputs it into the generative AI model. The generative AI model analyzes the input data and diagnoses the user's mental health condition. The diagnosis result is interpreted and converted into an easy-to-understand format. The server packages the diagnosis result and information on recommended medical institutions and sends it to the device. The device displays the diagnosis result to the user and also provides hospital contact information and a link to make an appointment.

[1288] In addition, the system has a function to monitor the mental health status of company employees. Companies provide this application to their employees and have them regularly check their mental health status. Employees enter data, and if there is a mental risk based on the analysis results, appropriate support is provided early. This process incorporates mechanisms to ensure data security while protecting employee privacy.

[1289] As a specific example, a 28-year-old female user launches an application for the first time, inputs her age, gender, and the fact that she works in sales, and then answers questions such as whether she worries a lot or has a decreased appetite. This information is sent to the server, which stores it in a database. The user then wears the smartwatch and synchronizes her heart rate and sleep pattern data with the application. The server preprocesses the received health data and inputs it into a generative AI model. The AI ​​model analyzes the user's condition and diagnoses her with "mild depression." Along with the diagnosis, information about nearby clinics and a link to make an appointment are sent to the user's device. The user confirms the diagnosis and clinic information and makes an online appointment with the clinic.

[1290] Example prompt sentence:

[1291] User profile data:

[1292] Age: 30

[1293] Gender: Male

[1294] Occupation: Engineer

[1295] User response data:

[1296] Mood: Stressed

[1297] Appetite: decreased

[1298] Health Data:

[1299] Heart Rate: [70, 75, 68, 74, 72, 78, 80]

[1300] Sleep Pattern: [6, 5, 7, 6, 5, 6, 4]

[1301] Activity: [2, 2, 1, 3, 2, 1, 2]

[1302] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1303] Step 1:

[1304] Collection and transmission of user data

[1305] The user starts the application, inputs their profile information (age, gender, occupation, etc.), and answers questions related to their current mood and physical condition. The device then sends this input data to the server.

[1306] Input: Profile information, answers to questions

[1307] Output: Send data to the server

[1308] Specific behavior:

[1309] The user accesses the application and enters information. The device validates the input and sends it to the server.

[1310] Step 2:

[1311] Health data synchronization and transmission

[1312] Users grant the application permission to access their health data, which allows the application to collect health data such as heart rate, exercise volume, and sleep patterns from the past week from their smart device and send it to a server.

[1313] Input: Health data from smart devices

[1314] Output: Send health data to the server

[1315] Specific behavior:

[1316] The user synchronizes their smart device with the application, and the device collects health data for a specified period and sends it to the server.

[1317] Step 3:

[1318] Data Preprocessing

[1319] The server retrieves stored profile information, question answers, and health data, performs data preprocessing to remove incomplete data and outliers, and converts the preprocessed data into a format suitable for the generative AI model.

[1320] Input: Profile information, answers to questions, health data

[1321] Output: Preprocessed data

[1322] Specific behavior:

[1323] The server reads the incoming data, performs consistency checks, removes outliers and incomplete data, and normalizes the data for the generative AI model.

[1324] Step 4:

[1325] Data input and diagnosis for generative AI models

[1326] The server generates a feature set from the preprocessed data and inputs it into a generative AI model, which analyzes the input data and diagnoses the user's mental health status.

[1327] Input: Preprocessed data

[1328] Output: Mental health status diagnosis

[1329] Specific behavior:

[1330] The server extracts a set of features and inputs them into a generative AI model, which then performs diagnostic analysis and generates results.

[1331] Step 5:

[1332] Providing diagnostic results and medical institution information

[1333] The server interprets the diagnosis results, converts them into an easy-to-understand format, and packages and sends information about recommended medical institutions to the device.

[1334] Input: Diagnostic results from generative AI model

[1335] Output: Diagnosis results and medical institution information to be presented to the user

[1336] Specific behavior:

[1337] The server interprets the diagnosis results, formats them into text, and sends them to the device along with information on recommended medical institutions.

[1338] Step 6:

[1339] Notification of results and follow-up with users

[1340] The device displays the diagnosis results and information on recommended medical institutions to the user, who can then check the results and make an appointment with a medical institution if necessary.

[1341] Input: Diagnosis results and medical institution information

[1342] Output: User presentation and follow-up action

[1343] Specific behavior:

[1344] The device displays the received information to the user, who then views the diagnostic results and clicks on the online reservation link.

[1345] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1346] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical attention at appropriate medical institutions. This system incorporates an emotion engine that recognizes the user's emotions. The program processing and operation of this system are explained below in natural language.

[1347] System configuration

[1348] This system mainly consists of the user's terminal, server, emotion engine, and smart device. The operation of each component is as follows.

[1349] Explanation of program processing

[1350] 1. Collection of User Data

[1351] User: Launches the application, enters profile information (age, gender, occupation, etc.), and answers questions related to current mood and physical condition.

[1352] Device: Sends the entered profile information and answers to questions to the server.

[1353] Server: Stores the received data in a database.

[1354] 2. Health data synchronization

[1355] User: Allows the application to access health data, which is then collected from smart devices (e.g., smartphones, wearable devices).

[1356] Device: Health data such as heart rate, exercise volume, and sleep patterns from the past week is collected from the smart device and sent to the server.

[1357] Server: Stores the received health data in a database.

[1358] 3. Collecting Emotional Data

[1359] User: The emotion engine recognizes the user's emotions based on their actions and inputs in the application.

[1360] Emotion engine: Analyzes user text input and other interactions to generate emotion data.

[1361] Terminal: Sends the generated emotion data to the server.

[1362] Server: Stores emotion data in a database.

[1363] 4. Data Preprocessing

[1364] Server: Retrieves stored profile information, question answers, health data, and emotional data.

[1365] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[1366] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[1367] 5. Input to the diagnostic model

[1368] Server: Generates a feature set from the preprocessed data, including, for example, average heart rate, standard deviation, sleep duration, answer scores to questions about specific emotions, and emotional data.

[1369] Server: Inputs the generated feature set into the generative AI model.

[1370] 6. Generating diagnostic results

[1371] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1372] Server: Customize the diagnosis results based on the emotional data recognized by the emotion engine. For example, if the emotional data indicates strong anxiety, include that information in the diagnosis results.

[1373] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[1374] 7. Distribution of Results

[1375] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[1376] Server: Sends packaged information to the terminal.

[1377] On the device: The diagnosis results are displayed to the user. For example, a message like "You are experiencing moderate depression. We recommend the following clinics" is displayed along with the clinic's contact information.

[1378] 8. Follow-up

[1379] User: Based on the diagnosis, the user can use the displayed clinic information to book an appointment online or contact the clinic directly.

[1380] User: If necessary, consult the self-care resources provided by the application (e.g., relaxation techniques, exercise programs, etc.).

[1381] Specific examples

[1382] For example, a 28-year-old male user launches an application for the first time and enters his profile information.

[1383] User: Enters age, gender, and whether they work in sales, then answers questions such as, "Have you often felt depressed in the last two weeks?"

[1384] Terminal: Sends this information to the server, which stores it in a database.

[1385] User: Then wears the smartwatch and syncs heart rate and sleep pattern data to the application.

[1386] Server: Preprocesses the received health data and inputs it into the generative AI model.

[1387] Emotion engine: Generates emotional data from user text input and other interactions.

[1388] Server: The generated emotion data is input into the AI ​​model to generate a diagnosis result, such as "The user is in a state of mild depression and has a high level of anxiety."

[1389] Server: Along with the diagnosis results, it sends information about nearby clinics and a link to make an appointment to the user's device.

[1390] Terminal: Displays the diagnosis results and allows the user to make an appointment based on the clinic information.

[1391] In this way, by analyzing various information, including the user's emotional data, it is possible to provide a more accurate diagnosis and help the user receive treatment at an appropriate medical institution. This system could be an effective means of promoting early detection and appropriate treatment of potential depression patients.

[1392] The processing flow will be explained below.

[1393] Step 1:

[1394] User: Launches the application and enters profile information (age, gender, occupation, etc.) on the initial registration screen.

[1395] Terminal: Receives the entered profile information and sends it to the server.

[1396] Server: Stores the received profile information in a database.

[1397] Step 2:

[1398] User: Next, the user answers questions in the application. For example, they answer "yes" or "no" to questions like "Have you been feeling depressed for the past two weeks?"

[1399] Terminal: Receives the user's response and sends it to the server.

[1400] Server: Stores the received answers in a database.

[1401] Step 3:

[1402] User: Follows the instructions in the application and allows collection of health data from smartphones and wearable devices.

[1403] Device: Collects health data from your smart device over the past week (heart rate, activity, sleep patterns, etc.).

[1404] Terminal: Sends collected health data to the server.

[1405] Server: Stores the received health data in a database.

[1406] Step 4:

[1407] Emotion engine: Analyzes user text input and other interactions with the application to generate emotion data.

[1408] Terminal: Sends the generated emotion data to the server.

[1409] Server: Stores emotion data in a database.

[1410] Step 5:

[1411] Server: Retrieves stored profile information, question answers, health data, and emotion data.

[1412] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[1413] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[1414] Step 6:

[1415] Server: Generates a feature set from the preprocessed data, including, for example, average heart rate, standard deviation, sleep duration, answer scores to questions about specific emotions, and emotional data.

[1416] Server: Inputs the generated feature set into the generative AI model.

[1417] Step 7:

[1418] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1419] Server: Customize the diagnosis results based on the emotional data recognized by the emotion engine. For example, if the emotional data indicates strong anxiety, include that information in the diagnosis results.

[1420] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[1421] Step 8:

[1422] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[1423] Server: Sends packaged information to the terminal.

[1424] Step 9:

[1425] On the device: Display the diagnosis and related information to the user. For example, display a message such as "You are experiencing moderate depression. We recommend the following clinics" along with the clinic's contact information.

[1426] User: Review the diagnosis and click on the recommended clinic contact and booking link to book an appointment.

[1427] Users: You can also consult self-care resources within the application (relaxation techniques, exercise programs, etc.).

[1428] Through this series of steps, users can not only receive an accurate diagnosis of their mental health condition, but also receive customized diagnostic results and recommended medical information based on their emotional data.

[1429] Example 2

[1430] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1431] In recent years, mental health problems have been increasing, and early detection of depression and anxiety disorders in particular is becoming increasingly important. However, it is difficult for users to accurately recognize their own mental state and seek medical attention at an appropriate medical institution. Furthermore, conventional systems are primarily limited to in-hospital diagnosis and are unable to comprehensively analyze diverse data from daily life. Therefore, there is a need for a system that can comprehensively analyze a user's emotions and health data, accurately diagnose their mental health state, and, if necessary, encourage them to seek medical attention at an appropriate medical institution.

[1432] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1433] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for analyzing the user's text input and interactions and generating emotion data, means for preprocessing the collected and generated data and converting it into a format that can be input to the generative AI model, means for generating a feature set from the preprocessed data and inputting it into the generative AI model, means for the generative AI model to analyze the input data and diagnose the user's mental health condition, and means for providing the user with the diagnosis results and information on recommended medical institutions. This makes it possible to comprehensively analyze various data obtained from the user's daily life, more accurately and quickly diagnose the user's mental health condition, and encourage the user to visit an appropriate medical institution.

[1434] "Profile information" refers to basic personal information such as a user's age, gender, and occupation.

[1435] "Answers to questions" are response information provided by a user in response to questions related to mental health conditions presented by the system.

[1436] A "smart device" is an electronic device that can collect health data such as heart rate, activity, and sleep patterns.

[1437] "Health data" is information about a user's physical condition, such as their heart rate, activity level, and sleep patterns.

[1438] "Emotion data" refers to information about emotions recognized and analyzed by the emotion engine from user text input and interactions.

[1439] "Data preprocessing" is the process of organizing and normalizing collected profile information, answers to questions, health data, and emotion data, and removing missing or outliers.

[1440] A "feature set" is a collection of data points that are important for analysis and are derived from preprocessed data.

[1441] A "generative AI model" is an artificial intelligence model that uses preprocessed data to diagnose a user's mental health status.

[1442] "Diagnosis results" are assessments and conclusions about the user's mental health status that are displayed as a result of the generative AI model's analysis.

[1443] "Information about recommended medical institutions" refers to information about appropriate medical institutions and specialists provided to users based on diagnostic results.

[1444] This invention is a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at appropriate medical institutions. The system is composed of a user terminal, a server, an emotion engine, and a smart device.

[1445] System configuration

[1446] Users launch the application using a device such as a smartphone or tablet. When launching the application, the user enters profile information (age, gender, occupation, etc.) and answers questions related to their mental health. In addition, the user allows the application to access their smart device (smartphone, smartwatch, etc.) to collect health data, such as heart rate, exercise volume, and sleep patterns.

[1447] Data collection and analysis

[1448] The server collects profile information, answers to questions, health data, and emotion data from the user's device and stores them in a centralized database. The emotion engine analyzes text input and other interactions the user has within the application to generate emotion data. For example, if a user inputs "I've been feeling tired lately," the emotion engine will analyze the emotions of "tired" and "anxiety."

[1449] Data preprocessing and feature generation

[1450] The server preprocesses all collected data and removes missing or outliers. From the preprocessed data, a feature set is generated to be input into the generative AI model. The feature set includes average heart rate, exercise volume, sleep duration, and emotion score. The generative AI model uses this feature set to diagnose the user's mental health condition.

[1451] Diagnostic results and medical institution recommendations

[1452] The generative AI model analyzes the user's data and generates a diagnosis, which includes specific descriptions such as "moderate depression" or "high anxiety level." Based on this diagnosis, the server recommends information about appropriate nearby medical institutions and reservation links. The diagnosis and medical institution information are sent to the user's device and displayed to the user through the application.

[1453] Specific examples

[1454] For example, a 28-year-old male user launches the application for the first time, enters his age, gender, and job title in sales, and then answers "Yes" to the question, "Have you often felt depressed in the last two weeks?" When the user wears a smartwatch and syncs it with the application, data on his heart rate and sleep patterns over the past week is sent to the server. The emotion engine analyzes "fatigue" and "anxiety" from the text entered by the user and generates emotion data.

[1455] The server preprocesses this data, generates a feature set, and inputs it into the generative AI model. The diagnosis results indicate that the user is in a state of mild depression and has a high level of anxiety, and information about nearby clinics and a link to make an appointment are sent to the user's device. The user can then make an appointment based on this information.

[1456] Prompt Sentence Examples

[1457] 1. Introducing a sentiment analysis tool using natural language processing algorithms

[1458] 2. Write a program to collect health data from smart devices

[1459] 3. Explain the architecture of the health checkup system based on profile information and emotion data.

[1460] By analyzing a variety of user data, this system can more accurately diagnose mental health conditions and encourage people to seek medical attention at appropriate medical institutions.

[1461] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1462] Step 1:

[1463] User Data Collection

[1464] User: Launches the application, enters profile information (e.g., age, gender, occupation), and answers questions related to mental health.

[1465] Input: Age, Gender, Occupation, Answer to Question

[1466] Action: A user enters information into an application and presses the submit button.

[1467] Output: Input information is saved on the device and sent to the server.

[1468] Device: Sends this information to the server.

[1469] Input: Information entered by the user

[1470] How it works: The device temporarily stores the user's input and sends an HTTP request to the server.

[1471] Output: Server receives user profile data and question answers

[1472] Server: Stores the received data in a database.

[1473] Input: User profile data and question answers

[1474] How it works: The server executes an SQL query to the database and saves the data.

[1475] Output: User data stored in the database

[1476] Step 2:

[1477] Health Data Sync

[1478] User: Allows the application to access health data.

[1479] Input: User action (access permission)

[1480] What happens: The user enables health data permission in the application settings.

[1481] Output: The permission settings change takes effect.

[1482] Device: Collects health data such as heart rate, exercise volume, and sleep patterns from smart devices and sends it to a server.

[1483] Input: Health data from smart devices

[1484] Operation: The terminal acquires data from the smart device and sends it to the server.

[1485] Output: Health data sent to the server

[1486] Server: Stores the received health data in a database.

[1487] Input: Health data

[1488] What it does: The server executes an SQL query to store in the database

[1489] Output: Health data stored in a database

[1490] Step 3:

[1491] Collecting Emotional Data

[1492] User: Enters text and otherwise interacts with the application.

[1493] Input: User text input or operation

[1494] Action: User enters text and presses submit, or interacts with a button within the app

[1495] Output: Text entered into the device and interaction data

[1496] Emotion engine: Analyzes text input and operation data to generate emotion data.

[1497] Input: User text input and operational data

[1498] How it works: The emotion engine uses natural language processing algorithms to analyze the input and generate emotion labels.

[1499] Output: Emotion data

[1500] Terminal: Sends the generated emotion data to the server.

[1501] Input: Emotion data generated by the emotion engine

[1502] Behavior: Issues an HTTP request to send emotion data to the server.

[1503] Output: Emotion data sent to the server

[1504] Server: Stores emotion data in a database.

[1505] Input: Emotion data

[1506] What it does: The server executes an SQL query to store in the database

[1507] Output: Emotion data stored in a database

[1508] Step 4:

[1509] Data Preprocessing

[1510] Server: Retrieves stored profile information, question answers, health data, and emotional data.

[1511] Input: Information retrieved from the database

[1512] What happens: The server executes a retrieval SQL query to retrieve the data.

[1513] Output: Multiple datasets retrieved for processing on the server side

[1514] Server: Performs data cleaning to remove incomplete data and outliers.

[1515] Input: Retrieved data

[1516] Action: Detects and filters data anomalies

[1517] Output: Cleaned dataset

[1518] Server: Performs data normalization.

[1519] Input: Cleaned dataset

[1520] What it does: It applies a data normalization algorithm to convert the data into a uniform format.

[1521] Output: Normalized dataset

[1522] Step 5:

[1523] Input to the diagnostic model

[1524] Server: Generates a feature set from the preprocessed data.

[1525] Input: Preprocessed dataset

[1526] Action: Runs an algorithm to extract features from the data.

[1527] Output: Feature set

[1528] Server: Inputs the feature set into the generative AI model.

[1529] Input: Feature set

[1530] Operation: Convert the feature set into an input format for the AI ​​model and input it into the model.

[1531] Output: Features input to the AI ​​model

[1532] Step 6:

[1533] Generating diagnostic results

[1534] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1535] Input: Feature set

[1536] How it works: The AI ​​model performs analysis and generates a diagnosis.

[1537] Output: Diagnostic results

[1538] Server: Converts diagnostic results into an easy-to-understand format.

[1539] Input: Diagnostic results

[1540] What it does: Converts diagnostic results into a text format for the user

[1541] Output: Diagnostic results for the user

[1542] Step 7:

[1543] Results distribution

[1544] Server: Packages the diagnosis results and information on recommended medical institutions.

[1545] Input: Diagnosis results, medical institution information

[1546] How it works: Combines diagnostic results and medical information into a single package

[1547] Output: Package information

[1548] Server: Sends packaged information to the device.

[1549] Input: Package information

[1550] Action: Sends package information to the device via HTTP request

[1551] Output: Package information received on the device

[1552] Terminal: Display diagnostic results to the user.

[1553] Input: Package information

[1554] Behavior: Updates the UI in the application to display diagnosis results and medical provider information.

[1555] Output: Diagnosis results and medical institution information displayed to the user

[1556] Step 8:

[1557] Follow-up

[1558] User: Based on the diagnosis, the user can use the displayed medical information to book an appointment online or contact the clinic directly.

[1559] Input: Diagnosis results, medical institution information

[1560] What happens: User clicks on clinic appointment link and fills out appointment form

[1561] Output: Clinic appointment completed

[1562] Users: Consult the self-care resources provided by the app as needed.

[1563] Input: Diagnostic results, self-care resources

[1564] Action: User views and practices self-care resources

[1565] Output: User self-care practices

[1566] (Application example 2)

[1567] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1568] Many systems exist that accurately diagnose mental health conditions and encourage potential depression patients to seek medical treatment at appropriate medical institutions. However, there are insufficient means to effectively collect and analyze users' emotional data and present it to users in a visually and intuitive manner. Furthermore, there are also insufficient means to analyze the collected data from multiple angles and provide users with information on improvement measures and medical facilities. This has led to issues that make it difficult for users to understand the diagnosis results and take appropriate action.

[1569] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1570] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving biometric data from the smart device, means for collecting and receiving the user's emotional data through an emotion recognition engine, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical facilities, and means for visually presenting the diagnosis results to the user through a head-mounted display. This enables the collection and analysis of multifaceted data, including the user's emotional data, and enables the user to intuitively and visually receive diagnosis results, promoting consultation at an appropriate medical institution.

[1571] "User" means an individual who uses this system and provides their profile information and biometric data.

[1572] "Profile Information" means basic personal information about a user, such as age, gender, and occupation.

[1573] "Question Answers" are responses that users make to mental health-related questions posed within the system.

[1574] "Smart device" is a general term for devices that can measure and collect a user's biometric information (e.g., heart rate, exercise volume, sleep patterns).

[1575] "Biometric data" means physiological data of a user (e.g., heart rate, sleep patterns) collected from a smart device.

[1576] An "emotion recognition engine" is software or algorithms that recognize emotions from user text input, voice, gestures, etc. and generate data.

[1577] "Preprocessing collected data" refers to data processing measures to convert collected profile information, biometric data, and emotional data into a format that can be input into a generative AI model.

[1578] A "generative AI model" is a machine learning or artificial intelligence model that assesses and diagnoses a user's mental health based on input data.

[1579] "Means for diagnosing a user's mental health condition" means means for analyzing collected and pre-processed data using a generative AI model to assess a user's mental health condition.

[1580] "Medical facility" is a general term for medical institutions such as hospitals and clinics where users can receive mental health consultations.

[1581] A "head-mounted display" is a display device that provides visual information when worn by a user.

[1582] This invention is a system that uses a head-mounted display (HMD) to diagnose a user's mental health condition and encourage them to seek appropriate medical advice. The system combines the user's profile information, answers to questions, biometric data from a smart device, and emotion data obtained using an emotion recognition engine to provide a diagnosis based on a generative AI model.

[1583] Hardware / Software used

[1584] Hardware:

[1585] Head-mounted display (HMD): General-purpose HMD device

[1586] Smart devices: devices that collect biometric data such as heart rate, activity, and sleep patterns (e.g., wearable devices)

[1587] software:

[1588] Emotion Recognition Engine: An engine that recognizes emotions from user text input, voice, and gestures.

[1589] Database: Firebase Firestore or other cloud databases

[1590] Generative AI model: A machine learning model for assessing mental health status

[1591] Web framework: Server-side framework such as Flask

[1592] System operation explanation

[1593] User Data Collection:

[1594] The user puts on the HMD device and launches the application. They input their profile information through the HMD interface using voice input and gestures. They also answer questions related to their mood and physical condition. This input data is sent from the HMD to the server and stored in a database.

[1595] Health Data Sync:

[1596] Users use smart devices (e.g., wearable devices) to collect biometric data such as heart rate, exercise volume, and sleep patterns. This data is then sent to a server via the HMD and stored in a database.

[1597] Emotion data collection:

[1598] The emotion recognition engine recognizes emotions from the user's text or voice input and generates emotion data, which is also sent to the server and stored in a database.

[1599] Data preprocessing:

[1600] The server preprocesses the collected profile information, biometric data, and emotion data, converting them into a format that can be input into the generative AI model, specifically by performing data cleaning and normalization.

[1601] Inputs to generative AI models and diagnosis:

[1602] The server inputs the preprocessed data into a generative AI model to diagnose the user's mental health status, and the results are converted into an intuitively understandable format.

[1603] Delivery of diagnostic results:

[1604] The server packages the diagnosis results and recommended medical facility information and presents them visually to the user through the HMD. The diagnosis results include contact information for nearby clinics and specialists.

[1605] Follow-up:

[1606] Users can review their diagnosis and use the HMD interface to find nearby clinics, schedule appointments online if needed, and access self-care resources (e.g., relaxation techniques, exercise programs).

[1607] Examples of specific examples and prompts

[1608] Examples:

[1609] A 28-year-old male user uses the app for the first time using a general-purpose HMD device. He registers his profile information by voice input, and then synchronizes his biometric data with his smart device. The emotion recognition engine generates emotion data based on the text input, and the diagnosis is "mild depression." The user then checks the information of nearby specialists through the HMD and makes an appointment.

[1610] Example prompt sentence:

[1611] "What steps would an HMD application that collects and analyzes a user's profile information and health data to diagnose their mental state take? Please explain using a concrete example."

[1612] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1613] Step 1:

[1614] User Data Collection

[1615] The user puts on the head-mounted display and starts the application. They enter their profile information (age, gender, occupation, etc.) and answers to questions related to their mood and physical condition using voice input or gestures. This input data is sent from the device to the server and stored in a database.

[1616] Input: Profile information, answers to questions

[1617] Data calculation: voice recognition, data format conversion

[1618] Output: Profile information and question and answer data (sent to server)

[1619] Step 2:

[1620] Health Data Sync

[1621] Users collect biometric data such as heart rate, exercise volume, and sleep patterns from the past week from their smart devices (e.g., wearable devices). This data is sent from the device to a server and stored in a database.

[1622] Input: Heart rate data, exercise data, sleep pattern data

[1623] Data processing: Data synchronization, data format conversion

[1624] Output: Biometric data (sent to server)

[1625] Step 3:

[1626] Collecting Emotional Data

[1627] The emotion recognition engine analyzes the user's text and voice input to generate emotion data, which is then sent from the device to a server and stored in a database.

[1628] Input: User text input, voice input

[1629] Data Computation: Emotion Recognition, Data Generation

[1630] Output: Emotion data (sent to server)

[1631] Step 4:

[1632] Data Preprocessing

[1633] The server takes the collected profile information, question responses, biometric data, and emotional data, and performs data cleaning and normalization, converting the data into a format that can be input into a generative AI model.

[1634] Input: Profile information, question and answer data, biometric data, emotional data

[1635] Data calculations: data cleaning, data normalization

[1636] Output: Preprocessed data

[1637] Step 5:

[1638] Input to generative AI models and diagnostics

[1639] The server inputs the preprocessed data into a generative AI model to diagnose the user's mental health status, and the results are converted into an easy-to-understand format.

[1640] Input: Preprocessed data

[1641] Data calculation: feature extraction, model inference

[1642] Output: Diagnostic results

[1643] Step 6:

[1644] Distribution of diagnostic results

[1645] The server packages the diagnosis and recommended medical facility information and presents it visually to the user through the HMD, including contact information for clinics and specialists.

[1646] Input: Diagnosis results, medical facility information

[1647] Data Calculation: Data Packaging

[1648] Output: Visual presentation data (sent to HMD)

[1649] Step 7:

[1650] Follow-up

[1651] Users can check their diagnosis and medical facility information, make appointments online if necessary, and also view and use self-care resources (relaxation techniques, exercise programs, etc.).

[1652] Input: diagnosis results, medical facility information, self-care resource information

[1653] Output: Reservation information, self-care resource usage status

[1654] In this way, the system collects and analyzes data from multiple angles, provides users with intuitive and visual diagnostic results, and encourages them to seek medical attention at an appropriate medical institution, thereby achieving comprehensive mental health support.

[1655] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1656] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1657] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1658] [Fourth embodiment]

[1659] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1660] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1661] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1662] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1663] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1664] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1665] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1666] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1667] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1668] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1669] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1670] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1671] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1672] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical attention at an appropriate medical institution. The processing and operation of the program of this system will be explained below in natural language.

[1673] System configuration

[1674] This system mainly consists of a user terminal, a server, and a smart device. The operation of each component is as follows:

[1675] Explanation of program processing

[1676] 1. Collection of User Data

[1677] User: Launches the application, enters profile information (age, gender, occupation, etc.), and answers questions related to current mood and physical condition.

[1678] Device: Sends the entered profile information and answers to questions to the server.

[1679] Server: Stores the received data in a database.

[1680] 2. Health data synchronization

[1681] User: Allows the application to access health data, which is then collected from smart devices (e.g., smartphones, wearable devices).

[1682] Device: Health data such as heart rate, exercise volume, and sleep patterns from the past week is collected from the smart device and sent to the server.

[1683] Server: Stores the received health data in a database.

[1684] 3. Data Preprocessing

[1685] Server: Retrieves stored profile information, question answers, and health data, filtering out incomplete data and outliers.

[1686] Server: Normalizes the data and converts it into a format suitable for AI models.

[1687] 4. Input to the diagnostic model

[1688] Server: Generates a feature set from the preprocessed data and inputs it into the generative AI model.

[1689] 5. Generating diagnostic results

[1690] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1691] Server: Interprets the diagnostic results and converts them into an understandable format.

[1692] 6. Distribution of Results

[1693] Server: Packages the diagnosis results and information on recommended medical institutions.

[1694] Server: Sends packaged information to the terminal.

[1695] On the device: The diagnosis results are displayed to the user, along with hospital contact information and a link to make an appointment.

[1696] 7. Follow-up

[1697] User: Based on the diagnosis, the user can use the displayed clinic information to book an appointment online or contact the clinic directly.

[1698] Users: Refer to the self-care resources provided by the app as needed.

[1699] Specific examples

[1700] For example, suppose a 28-year-old female user launches an application for the first time.

[1701] Device: The user enters their age, gender, and whether they work in sales, and then answers questions such as whether they worry a lot or have a decreased appetite.

[1702] Terminal: Sends this information to the server, which stores it in a database.

[1703] User: Then wears the smartwatch and syncs heart rate and sleep pattern data to the application.

[1704] Server: Preprocesses the received health data and inputs it into the generative AI model.

[1705] Server: The AI ​​model analyzes the user's situation and diagnoses them as having "mild depression."

[1706] Server: Along with the diagnosis results, it sends information about nearby clinics and a link to make an appointment to the user's device.

[1707] User: Checks diagnosis results and clinic information and makes online appointment with clinic.

[1708] In this way, users can receive an accurate diagnosis and smoothly consult with an appropriate medical institution. This system could be an effective means of promoting early detection and appropriate treatment for potential depression patients.

[1709] The processing flow will be explained below.

[1710] Step 1:

[1711] User: Launches the application and enters profile information (age, gender, occupation, etc.) on the initial registration screen.

[1712] Terminal: Receives the entered profile information and sends it to the server.

[1713] Server: Stores the received profile information in a database.

[1714] Step 2:

[1715] User: Next, the user answers questions in the application. For example, they answer "yes" or "no" to questions like "Have you been feeling depressed for the past two weeks?"

[1716] Terminal: Receives the user's response and sends it to the server.

[1717] Server: Stores the received answers in a database.

[1718] Step 3:

[1719] User: Follows the instructions in the application and allows collection of health data from smartphones and wearable devices.

[1720] Device: Collects health data from your smart device over the past week (heart rate, activity, sleep patterns, etc.).

[1721] Terminal: Sends collected health data to the server.

[1722] Server: Stores the received health data in a database.

[1723] Step 4:

[1724] Server: Retrieves stored profile information, question answers, and health data.

[1725] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[1726] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[1727] Step 5:

[1728] Server: Generates a set of features from the preprocessed data, including, for example, the mean heart rate, standard deviation, sleep duration, and scores for answers to questions about specific emotions.

[1729] Server: Inputs the generated feature set into the generative AI model.

[1730] Step 6:

[1731] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1732] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[1733] Step 7:

[1734] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[1735] Server: Sends packaged information to the terminal.

[1736] Step 8:

[1737] On the device: Display the diagnosis and related information to the user. For example, display a message such as "You are experiencing moderate depression. We recommend the following clinics" along with the clinic's contact information.

[1738] User: Review the diagnosis and click on the recommended clinic contact and booking link to book an appointment.

[1739] Users: You can also consult self-care resources within the application (relaxation techniques, exercise programs, etc.).

[1740] Through this series of steps, users are able to accurately diagnose their mental health condition and are encouraged to seek medical attention from an appropriate medical institution.

[1741] Example 1

[1742] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1743] Mental health issues are a serious social challenge in modern society. Early detection is particularly important for mental illnesses such as depression, and consultation at an appropriate medical institution is required. However, many people do not have the means to accurately assess their own mental health status, and access to appropriate medical institutions is limited. The objective of the present invention is to provide a system that accurately diagnoses a user's mental health status and encourages potential depression patients to seek treatment at an appropriate medical institution.

[1744] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1745] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for generating prompt sentences, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical institutions, and means for validating the received data and detecting and correcting missing data and outliers, thereby enabling an accurate diagnosis of the user's mental health condition and promoting consultation at an appropriate medical institution.

[1746] "Means for receiving profile information and answers to questions as input from the user" is a function for collecting personal information and answer data to questions that the user has entered into the application and sending it to the server.

[1747] "Means for collecting and receiving health data from smart devices" refers to a function for transmitting health data such as heart rate, exercise volume, and sleep patterns obtained from smartphones and wearable devices to a server.

[1748] "Means for preprocessing collected data and converting it into a format that can be input into a generative AI model" refers to a function for formalizing and standardizing data obtained from users and smart devices and converting it into a form that can be used by an AI model.

[1749] "Means for inputting preprocessed data into a generative AI model and diagnosing the user's mental health state" refers to a function for inputting preprocessed data into a generative AI model and using AI to evaluate and diagnose the user's mental state.

[1750] "Means for providing users with information on diagnostic results and recommended medical institutions" refers to a function for notifying users of the diagnostic results obtained by AI and information on recommended medical institutions based on those results.

[1751] "Means for validating received data and detecting and correcting missing data and abnormal values" refers to a function for checking the accuracy of data sent to the server, finding missing or abnormal data, and supplementing or correcting it.

[1752] The "means for generating prompt sentences" is a function for creating specific instruction sentences (prompt sentences) in a format that can be analyzed by the generative AI model based on preprocessed data.

[1753] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at an appropriate medical institution. This system is mainly composed of a user terminal, a server, and a smart device.

[1754] User Data Collection

[1755] First, the user launches the application and enters profile information such as age, gender, and occupation. They also answer questions about their mood and physical condition. For example, they answer questions such as, "Have you been worrying a lot lately?" or "Have you lost your appetite?" The information entered by the user is sent to the server by the device and stored in a database.

[1756] Health Data Sync

[1757] Next, the user grants the application permission to access health data from their smart device (e.g., smartphone, smartwatch). This allows the application to collect health-related data such as heart rate, exercise volume, and sleep patterns from the past week and send it to the server. The health data is also stored in a database.

[1758] Data Preprocessing

[1759] The server retrieves the stored profile information, question answers, and health data, and detects and corrects incomplete data and outliers. Specifically, it performs data validation to detect and correct missing data and outliers. It also normalizes the data and converts it into a format that can be input into the generative AI model. For example, it standardizes numerical data and encodes categorical data.

[1760] Input to the diagnostic model

[1761] Next, a feature set is generated from the preprocessed data, and a prompt sentence is created. For example, the generated features might be "28-year-old female, sales position, high heart rate variability, light sleep." This prompt sentence is input into a generative AI model to diagnose the user's mental health condition.

[1762] Generating diagnostic results

[1763] The generative AI model evaluates the user's mental health state based on the input feature set and generates a diagnosis such as "mild depression" or "normal." The diagnosis result is interpreted by the server and converted into a format that is easy for humans to understand. Specifically, a message is generated saying, "Based on your current health data, you are suspected of having mild depression."

[1764] Results distribution

[1765] The server sends this information back to the user's device, which then displays the diagnosis, including contact details for the specific hospital and a link to make an online appointment.

[1766] Follow-up

[1767] Based on the diagnosis results, the user can use the displayed clinic information to make an appointment online or contact the clinic directly. If necessary, the user can also refer to the self-care resources provided by the application. The server monitors the user's usage and provides follow-up reminders and additional resources as needed.

[1768] Specific examples

[1769] For example, when a 28-year-old female user launches an application for the first time, the following scenario occurs:

[1770] Users enter profile information such as age, gender, and occupation, and answer questions about concerns and loss of appetite.

[1771] This information is sent from the terminal to the server and stored in a database.

[1772] The user then wears the smartwatch and syncs data on their heart rate and sleep patterns with the application.

[1773] The server preprocesses the received health data and inputs it into a generative AI model.

[1774] The AI ​​model analyzes the patient and diagnoses him as having "mild depression."

[1775] The server sends the diagnosis results along with information about nearby clinics and an online reservation link to the user's terminal.

[1776] The user checks the diagnosis results and clinic information and makes an online appointment with the clinic.

[1777] Prompt Sentence Examples

[1778] "A 28-year-old woman has recently been feeling anxious and has experienced a decrease in her appetite. Please assess this user's mental health status by taking into account her heart rate data and sleep patterns over the past week."

[1779] By using this system, users can receive an accurate diagnosis and smoothly consult with an appropriate medical institution. This system is an effective means of promoting early detection of potential depression patients and appropriate treatment.

[1780] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1781] Step 1: Collect user data

[1782] Input: Profile information (such as age, gender, and occupation) and answers to questions you enter into the application (e.g., "Have you been feeling a lot of anxiety lately?", "Have you lost your appetite?", etc.).

[1783] How it works: The user launches the application on their smartphone or PC and follows the instructions to provide the required profile information and answer health questions.

[1784] Data processing and output: These input data are converted into the appropriate format within the terminal and sent to the server. The terminal performs a data consistency check to ensure there are no inconsistencies.

[1785] Output: The user data, whose integrity has been verified, is sent to the server, which stores it in a database.

[1786] Step 2: Syncing health data

[1787] Input: Health data such as heart rate, activity, and sleep patterns obtained from smart devices (smartphones, wearable devices, etc.).

[1788] What it does: The user grants the application permission to access their smart device. The application periodically syncs with the smart device and collects health data from the past week.

[1789] Data processing and output: The device normalizes the acquired health data and performs consistency checks before sending it to the server. For example, it checks whether the heart rate is too high.

[1790] Output: The health data, whose integrity has been verified, is sent to the server, which stores it in a database.

[1791] Step 3: Data Preprocessing

[1792] Input: Saved profile information, question answers and health data.

[1793] What it does: The server retrieves these data from the database, detects and removes incomplete data and outliers, and performs other tasks, such as standardizing numeric data and encoding categorical data.

[1794] Data processing and output: The server completes incomplete data, corrects outliers, and converts the data into a format suitable for the AI ​​model. For example, it treats "age" as a numeric value and encodes "gender" as a categorical value.

[1795] Output: Preprocessed data is produced and can be input into subsequent processing steps.

[1796] Step 4: Input to the diagnostic model

[1797] Input: Preprocessed data (profile information, question answers, health data).

[1798] Specific operation: The server generates a feature set from the preprocessed data and creates a prompt based on this. The prompt generated is "28-year-old female, sales, high heart rate variability, light sleeper."

[1799] Data processing and output: The generated prompt sentences are input into a generative AI model, which analyzes them and diagnoses the user's mental health status.

[1800] Output: The generative AI model outputs a diagnosis of the user's mental health status.

[1801] Step 5: Generate diagnostic results

[1802] Input: Diagnostic results from a generative AI model.

[1803] Specific operation: The server converts the diagnosis results received from the generative AI model into a format that is easy for humans to understand. For example, it generates a message saying, "Mild depression is suspected."

[1804] Output: The interpreted diagnostic results are generated in a format that can be presented to the user.

[1805] Step 6: Delivering results

[1806] Input: Interpreted diagnosis and recommended medical provider information.

[1807] Specific operation: The server packages the diagnosis results and information on recommended medical institutions and sends them back to the terminal, which receives them and displays them to the user.

[1808] Data processing and output: For example, generating a message saying, "You are suspected of having mild depression. Here is the nearest clinic," and providing a link to make an online appointment.

[1809] Output: The diagnosis and medical institution information are displayed to the user.

[1810] Step 7: Follow up

[1811] Input: diagnostic results and self-care resources.

[1812] What happens: The user makes an online appointment using the provided clinic information and also accesses self-care resources provided by the application (e.g., mindfulness exercises, stress management guides).

[1813] Data processing and output: The server monitors the user's usage and provides follow-up reminders and additional resources as needed. For example, a reminder to "re-diagnose in a week."

[1814] Output: User receives follow-up information and resources for ongoing mental health management.

[1815] (Application example 1)

[1816] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1817] Conventional mental health diagnostic systems only target individual data, and lack mechanisms for efficiently monitoring the mental health status of employees in large organizations such as corporations and providing early and appropriate support. This can result in employees with mental health problems not receiving appropriate support, leading to reduced productivity and a worsening working environment. Another challenge is the difficulty of providing appropriate support while ensuring employee privacy and data security.

[1818] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1819] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical institutions, and means for monitoring the mental health condition of company employees and providing appropriate support early when their mental condition is unstable. This makes it possible to efficiently monitor the mental health condition of company employees and provide appropriate support early.

[1820] "Profile information" is data that includes personal information such as a user's age, gender, and occupation.

[1821] "Answers to questions" are response data entered by the user in response to questions related to mental and physical conditions presented in the application.

[1822] "Smart devices" refers to wearable devices and smartphones that can collect health data such as a user's heart rate, exercise volume, and sleep patterns.

[1823] "Health data" refers to data that indicates a user's health status, such as heart rate, exercise volume, and sleep patterns.

[1824] "Preprocessing" is the process of removing incomplete data and outliers from collected data and converting it into a format suitable for generative AI models.

[1825] A "generative AI model" is an artificial intelligence model that analyzes input data and generates a specific output (in this case, a mental health diagnosis).

[1826] "Diagnosis" refers to assessing a user's mental health status based on the output results obtained by the generative AI model.

[1827] "Information about recommended medical institutions" refers to information including contact information and appointment links for appropriate medical institutions suggested to the user based on the diagnosis results.

[1828] The "measures for monitoring the mental health status of employees in companies" is a mechanism for continuously monitoring the mental health status of employees within a company and providing appropriate support early on if a risk is determined.

[1829] "Self-care resources" refers to information and tools that users can use to manage their own mental health based on their diagnosis.

[1830] This invention is a system that diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at an appropriate medical institution, and also has a function for monitoring the mental health of company employees. This system is mainly composed of a user terminal, a server, and a smart device. The operation of each component will be described below.

[1831] First, the user launches the application and enters their profile information (age, gender, occupation, etc.) and answers questions related to their current mood and physical condition. The device then sends this information to the server, which then stores the received data in a database.

[1832] Next, the user allows the application to access their health data, which allows the application to collect health data such as heart rate, exercise volume, and sleep patterns from the past week from their smart device (e.g., smartphone or wearable device) and send it to the server. The server then stores the received health data in a database.

[1833] The server retrieves saved profile information, question answers, and health data and performs data preprocessing to remove incomplete data and outliers. The preprocessed data is converted into a format suitable for the generative AI model. The server generates a feature set from this preprocessed data and inputs it into the generative AI model. The generative AI model analyzes the input data and diagnoses the user's mental health condition. The diagnosis result is interpreted and converted into an easy-to-understand format. The server packages the diagnosis result and information on recommended medical institutions and sends it to the device. The device displays the diagnosis result to the user and also provides hospital contact information and a link to make an appointment.

[1834] In addition, the system has a function to monitor the mental health status of company employees. Companies provide this application to their employees and have them regularly check their mental health status. Employees enter data, and if there is a mental risk based on the analysis results, appropriate support is provided early. This process incorporates mechanisms to ensure data security while protecting employee privacy.

[1835] As a specific example, a 28-year-old female user launches an application for the first time, inputs her age, gender, and the fact that she works in sales, and then answers questions such as whether she worries a lot or has a decreased appetite. This information is sent to the server, which stores it in a database. The user then wears the smartwatch and synchronizes her heart rate and sleep pattern data with the application. The server preprocesses the received health data and inputs it into a generative AI model. The AI ​​model analyzes the user's condition and diagnoses her with "mild depression." Along with the diagnosis, information about nearby clinics and a link to make an appointment are sent to the user's device. The user confirms the diagnosis and clinic information and makes an online appointment with the clinic.

[1836] Example prompt sentence:

[1837] User profile data:

[1838] Age: 30

[1839] Gender: Male

[1840] Occupation: Engineer

[1841] User response data:

[1842] Mood: Stressed

[1843] Appetite: decreased

[1844] Health Data:

[1845] Heart Rate: [70, 75, 68, 74, 72, 78, 80]

[1846] Sleep Pattern: [6, 5, 7, 6, 5, 6, 4]

[1847] Activity: [2, 2, 1, 3, 2, 1, 2]

[1848] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1849] Step 1:

[1850] Collection and transmission of user data

[1851] The user starts the application, inputs their profile information (age, gender, occupation, etc.), and answers questions related to their current mood and physical condition. The device then sends this input data to the server.

[1852] Input: Profile information, answers to questions

[1853] Output: Send data to the server

[1854] Specific behavior:

[1855] The user accesses the application and enters information. The device validates the input and sends it to the server.

[1856] Step 2:

[1857] Health data synchronization and transmission

[1858] Users grant the application permission to access their health data, which allows the application to collect health data such as heart rate, exercise volume, and sleep patterns from the past week from their smart device and send it to a server.

[1859] Input: Health data from smart devices

[1860] Output: Send health data to the server

[1861] Specific behavior:

[1862] The user synchronizes their smart device with the application, and the device collects health data for a specified period and sends it to the server.

[1863] Step 3:

[1864] Data Preprocessing

[1865] The server retrieves stored profile information, question answers, and health data, performs data preprocessing to remove incomplete data and outliers, and converts the preprocessed data into a format suitable for the generative AI model.

[1866] Input: Profile information, answers to questions, health data

[1867] Output: Preprocessed data

[1868] Specific behavior:

[1869] The server reads the incoming data, performs consistency checks, removes outliers and incomplete data, and normalizes the data for the generative AI model.

[1870] Step 4:

[1871] Data input and diagnosis for generative AI models

[1872] The server generates a feature set from the preprocessed data and inputs it into a generative AI model, which analyzes the input data and diagnoses the user's mental health status.

[1873] Input: Preprocessed data

[1874] Output: Mental health status diagnosis

[1875] Specific behavior:

[1876] The server extracts a set of features and inputs them into a generative AI model, which then performs diagnostic analysis and generates results.

[1877] Step 5:

[1878] Providing diagnostic results and medical institution information

[1879] The server interprets the diagnosis results, converts them into an easy-to-understand format, and packages and sends information about recommended medical institutions to the device.

[1880] Input: Diagnostic results from generative AI model

[1881] Output: Diagnosis results and medical institution information to be presented to the user

[1882] Specific behavior:

[1883] The server interprets the diagnosis results, formats them into text, and sends them to the device along with information on recommended medical institutions.

[1884] Step 6:

[1885] Notification of results and follow-up with users

[1886] The device displays the diagnosis results and information on recommended medical institutions to the user, who can then check the results and make an appointment with a medical institution if necessary.

[1887] Input: Diagnosis results and medical institution information

[1888] Output: User presentation and follow-up action

[1889] Specific behavior:

[1890] The device displays the received information to the user, who then views the diagnostic results and clicks on the online reservation link.

[1891] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1892] This invention relates to a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical attention at appropriate medical institutions. This system incorporates an emotion engine that recognizes the user's emotions. The program processing and operation of this system are explained below in natural language.

[1893] System configuration

[1894] This system mainly consists of the user's terminal, server, emotion engine, and smart device. The operation of each component is as follows.

[1895] Explanation of program processing

[1896] 1. Collection of User Data

[1897] User: Launches the application, enters profile information (age, gender, occupation, etc.), and answers questions related to current mood and physical condition.

[1898] Device: Sends the entered profile information and answers to questions to the server.

[1899] Server: Stores the received data in a database.

[1900] 2. Health data synchronization

[1901] User: Allows the application to access health data, which is then collected from smart devices (e.g., smartphones, wearable devices).

[1902] Device: Health data such as heart rate, exercise volume, and sleep patterns from the past week is collected from the smart device and sent to the server.

[1903] Server: Stores the received health data in a database.

[1904] 3. Collecting Emotional Data

[1905] User: The emotion engine recognizes the user's emotions based on their actions and inputs in the application.

[1906] Emotion engine: Analyzes user text input and other interactions to generate emotion data.

[1907] Terminal: Sends the generated emotion data to the server.

[1908] Server: Stores emotion data in a database.

[1909] 4. Data Preprocessing

[1910] Server: Retrieves stored profile information, question answers, health data, and emotional data.

[1911] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[1912] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[1913] 5. Input to the diagnostic model

[1914] Server: Generates a feature set from the preprocessed data, including, for example, average heart rate, standard deviation, sleep duration, answer scores to questions about specific emotions, and emotional data.

[1915] Server: Inputs the generated feature set into the generative AI model.

[1916] 6. Generating diagnostic results

[1917] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1918] Server: Customize the diagnosis results based on the emotional data recognized by the emotion engine. For example, if the emotional data indicates strong anxiety, include that information in the diagnosis results.

[1919] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[1920] 7. Distribution of Results

[1921] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[1922] Server: Sends packaged information to the terminal.

[1923] On the device: The diagnosis results are displayed to the user. For example, a message like "You are experiencing moderate depression. We recommend the following clinics" is displayed along with the clinic's contact information.

[1924] 8. Follow-up

[1925] User: Based on the diagnosis, the user can use the displayed clinic information to book an appointment online or contact the clinic directly.

[1926] User: If necessary, consult the self-care resources provided by the application (e.g., relaxation techniques, exercise programs, etc.).

[1927] Specific examples

[1928] For example, a 28-year-old male user launches an application for the first time and enters his profile information.

[1929] User: Enters age, gender, and whether they work in sales, then answers questions such as, "Have you often felt depressed in the last two weeks?"

[1930] Terminal: Sends this information to the server, which stores it in a database.

[1931] User: Then wears the smartwatch and syncs heart rate and sleep pattern data to the application.

[1932] Server: Preprocesses the received health data and inputs it into the generative AI model.

[1933] Emotion engine: Generates emotional data from user text input and other interactions.

[1934] Server: The generated emotion data is input into the AI ​​model to generate a diagnosis result, such as "The user is in a state of mild depression and has a high level of anxiety."

[1935] Server: Along with the diagnosis results, it sends information about nearby clinics and a link to make an appointment to the user's device.

[1936] Terminal: Displays the diagnosis results and allows the user to make an appointment based on the clinic information.

[1937] In this way, by analyzing various information, including the user's emotional data, it is possible to provide a more accurate diagnosis and help the user receive treatment at an appropriate medical institution. This system could be an effective means of promoting early detection and appropriate treatment of potential depression patients.

[1938] The processing flow will be explained below.

[1939] Step 1:

[1940] User: Launches the application and enters profile information (age, gender, occupation, etc.) on the initial registration screen.

[1941] Terminal: Receives the entered profile information and sends it to the server.

[1942] Server: Stores the received profile information in a database.

[1943] Step 2:

[1944] User: Next, the user answers questions in the application. For example, they answer "yes" or "no" to questions like "Have you been feeling depressed for the past two weeks?"

[1945] Terminal: Receives the user's response and sends it to the server.

[1946] Server: Stores the received answers in a database.

[1947] Step 3:

[1948] User: Follows the instructions in the application and allows collection of health data from smartphones and wearable devices.

[1949] Device: Collects health data from your smart device over the past week (heart rate, activity, sleep patterns, etc.).

[1950] Terminal: Sends collected health data to the server.

[1951] Server: Stores the received health data in a database.

[1952] Step 4:

[1953] Emotion engine: Analyzes user text input and other interactions with the application to generate emotion data.

[1954] Terminal: Sends the generated emotion data to the server.

[1955] Server: Stores emotion data in a database.

[1956] Step 5:

[1957] Server: Retrieves stored profile information, question answers, health data, and emotion data.

[1958] Server: Performs data cleaning, e.g., removing incomplete data and outliers.

[1959] Server: Normalizes the data, for example, standardizing heart rate data along the time axis.

[1960] Step 6:

[1961] Server: Generates a feature set from the preprocessed data, including, for example, average heart rate, standard deviation, sleep duration, answer scores to questions about specific emotions, and emotional data.

[1962] Server: Inputs the generated feature set into the generative AI model.

[1963] Step 7:

[1964] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[1965] Server: Customize the diagnosis results based on the emotional data recognized by the emotion engine. For example, if the emotional data indicates strong anxiety, include that information in the diagnosis results.

[1966] Server: Generates the diagnostic results and converts them into an easy-to-understand format, such as "The user is likely to be moderately depressed."

[1967] Step 8:

[1968] Server: Packages the diagnosis results and recommended medical facilities. The package includes the diagnosis conclusion, contact information for nearby clinics and specialists, and a link to make an appointment.

[1969] Server: Sends packaged information to the terminal.

[1970] Step 9:

[1971] On the device: Display the diagnosis and related information to the user. For example, display a message such as "You are experiencing moderate depression. We recommend the following clinics" along with the clinic's contact information.

[1972] User: Review the diagnosis and click on the recommended clinic contact and booking link to book an appointment.

[1973] Users: You can also consult self-care resources within the application (relaxation techniques, exercise programs, etc.).

[1974] Through this series of steps, users can not only receive an accurate diagnosis of their mental health condition, but also receive customized diagnostic results and recommended medical information based on their emotional data.

[1975] Example 2

[1976] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1977] In recent years, mental health problems have been increasing, and early detection of depression and anxiety disorders in particular is becoming increasingly important. However, it is difficult for users to accurately recognize their own mental state and seek medical attention at an appropriate medical institution. Furthermore, conventional systems are primarily limited to in-hospital diagnosis and are unable to comprehensively analyze diverse data from daily life. Therefore, there is a need for a system that can comprehensively analyze a user's emotions and health data, accurately diagnose their mental health state, and, if necessary, encourage them to seek medical attention at an appropriate medical institution.

[1978] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1979] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving health data from the smart device, means for analyzing the user's text input and interactions and generating emotion data, means for preprocessing the collected and generated data and converting it into a format that can be input to the generative AI model, means for generating a feature set from the preprocessed data and inputting it into the generative AI model, means for the generative AI model to analyze the input data and diagnose the user's mental health condition, and means for providing the user with the diagnosis results and information on recommended medical institutions. This makes it possible to comprehensively analyze various data obtained from the user's daily life, more accurately and quickly diagnose the user's mental health condition, and encourage the user to visit an appropriate medical institution.

[1980] "Profile information" refers to basic personal information such as a user's age, gender, and occupation.

[1981] "Answers to questions" are response information provided by a user in response to questions related to mental health conditions presented by the system.

[1982] A "smart device" is an electronic device that can collect health data such as heart rate, activity, and sleep patterns.

[1983] "Health data" is information about a user's physical condition, such as their heart rate, activity level, and sleep patterns.

[1984] "Emotion data" refers to information about emotions recognized and analyzed by the emotion engine from user text input and interactions.

[1985] "Data preprocessing" is the process of organizing and normalizing collected profile information, answers to questions, health data, and emotion data, and removing missing or outliers.

[1986] A "feature set" is a collection of data points that are important for analysis and are derived from preprocessed data.

[1987] A "generative AI model" is an artificial intelligence model that uses preprocessed data to diagnose a user's mental health status.

[1988] "Diagnosis results" are assessments and conclusions about the user's mental health status that are displayed as a result of the generative AI model's analysis.

[1989] "Information about recommended medical institutions" refers to information about appropriate medical institutions and specialists provided to users based on diagnostic results.

[1990] This invention is a system that accurately diagnoses a user's mental health condition and encourages potential depression patients to seek medical treatment at appropriate medical institutions. The system is composed of a user terminal, a server, an emotion engine, and a smart device.

[1991] System configuration

[1992] Users launch the application using a device such as a smartphone or tablet. When launching the application, the user enters profile information (age, gender, occupation, etc.) and answers questions related to their mental health. In addition, the user allows the application to access their smart device (smartphone, smartwatch, etc.) to collect health data, such as heart rate, exercise volume, and sleep patterns.

[1993] Data collection and analysis

[1994] The server collects profile information, answers to questions, health data, and emotion data from the user's device and stores them in a centralized database. The emotion engine analyzes text input and other interactions the user has within the application to generate emotion data. For example, if a user inputs "I've been feeling tired lately," the emotion engine will analyze the emotions of "tired" and "anxiety."

[1995] Data preprocessing and feature generation

[1996] The server preprocesses all collected data and removes missing or outliers. From the preprocessed data, a feature set is generated to be input into the generative AI model. The feature set includes average heart rate, exercise volume, sleep duration, and emotion score. The generative AI model uses this feature set to diagnose the user's mental health condition.

[1997] Diagnostic results and medical institution recommendations

[1998] The generative AI model analyzes the user's data and generates a diagnosis, which includes specific descriptions such as "moderate depression" or "high anxiety level." Based on this diagnosis, the server recommends information about appropriate nearby medical institutions and reservation links. The diagnosis and medical institution information are sent to the user's device and displayed to the user through the application.

[1999] Specific examples

[2000] For example, a 28-year-old male user launches the application for the first time, enters his age, gender, and job title in sales, and then answers "Yes" to the question, "Have you often felt depressed in the last two weeks?" When the user wears a smartwatch and syncs it with the application, data on his heart rate and sleep patterns over the past week is sent to the server. The emotion engine analyzes "fatigue" and "anxiety" from the text entered by the user and generates emotion data.

[2001] The server preprocesses this data, generates a feature set, and inputs it into the generative AI model. The diagnosis results indicate that the user is in a state of mild depression and has a high level of anxiety, and information about nearby clinics and a link to make an appointment are sent to the user's device. The user can then make an appointment based on this information.

[2002] Prompt Sentence Examples

[2003] 1. Introducing a sentiment analysis tool using natural language processing algorithms

[2004] 2. Write a program to collect health data from smart devices

[2005] 3. Explain the architecture of the health checkup system based on profile information and emotion data.

[2006] By analyzing a variety of user data, this system can more accurately diagnose mental health conditions and encourage people to seek medical attention at appropriate medical institutions.

[2007] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2008] Step 1:

[2009] User Data Collection

[2010] User: Launches the application, enters profile information (e.g., age, gender, occupation), and answers questions related to mental health.

[2011] Input: Age, Gender, Occupation, Answer to Question

[2012] Action: A user enters information into an application and presses the submit button.

[2013] Output: Input information is saved on the device and sent to the server.

[2014] Device: Sends this information to the server.

[2015] Input: Information entered by the user

[2016] How it works: The device temporarily stores the user's input and sends an HTTP request to the server.

[2017] Output: Server receives user profile data and question answers

[2018] Server: Stores the received data in a database.

[2019] Input: User profile data and question answers

[2020] How it works: The server executes an SQL query to the database and saves the data.

[2021] Output: User data stored in the database

[2022] Step 2:

[2023] Health Data Sync

[2024] User: Allows the application to access health data.

[2025] Input: User action (access permission)

[2026] What happens: The user enables health data permission in the application settings.

[2027] Output: The permission settings change takes effect.

[2028] Device: Collects health data such as heart rate, exercise volume, and sleep patterns from smart devices and sends it to a server.

[2029] Input: Health data from smart devices

[2030] Operation: The terminal acquires data from the smart device and sends it to the server.

[2031] Output: Health data sent to the server

[2032] Server: Stores the received health data in a database.

[2033] Input: Health data

[2034] What it does: The server executes an SQL query to store in the database

[2035] Output: Health data stored in a database

[2036] Step 3:

[2037] Collecting Emotional Data

[2038] User: Enters text and otherwise interacts with the application.

[2039] Input: User text input or operation

[2040] Action: User enters text and presses submit, or interacts with a button within the app

[2041] Output: Text entered into the device and interaction data

[2042] Emotion engine: Analyzes text input and operation data to generate emotion data.

[2043] Input: User text input and operational data

[2044] How it works: The emotion engine uses natural language processing algorithms to analyze the input and generate emotion labels.

[2045] Output: Emotion data

[2046] Terminal: Sends the generated emotion data to the server.

[2047] Input: Emotion data generated by the emotion engine

[2048] Behavior: Issues an HTTP request to send emotion data to the server.

[2049] Output: Emotion data sent to the server

[2050] Server: Stores emotion data in a database.

[2051] Input: Emotion data

[2052] What it does: The server executes an SQL query to store in the database

[2053] Output: Emotion data stored in a database

[2054] Step 4:

[2055] Data Preprocessing

[2056] Server: Retrieves stored profile information, question answers, health data, and emotional data.

[2057] Input: Information retrieved from the database

[2058] What happens: The server executes a retrieval SQL query to retrieve the data.

[2059] Output: Multiple datasets retrieved for processing on the server side

[2060] Server: Performs data cleaning to remove incomplete data and outliers.

[2061] Input: Retrieved data

[2062] Action: Detects and filters data anomalies

[2063] Output: Cleaned dataset

[2064] Server: Performs data normalization.

[2065] Input: Cleaned dataset

[2066] What it does: It applies a data normalization algorithm to convert the data into a uniform format.

[2067] Output: Normalized dataset

[2068] Step 5:

[2069] Input to the diagnostic model

[2070] Server: Generates a feature set from the preprocessed data.

[2071] Input: Preprocessed dataset

[2072] Action: Runs an algorithm to extract features from the data.

[2073] Output: Feature set

[2074] Server: Inputs the feature set into the generative AI model.

[2075] Input: Feature set

[2076] Operation: Convert the feature set into an input format for the AI ​​model and input it into the model.

[2077] Output: Features input to the AI ​​model

[2078] Step 6:

[2079] Generating diagnostic results

[2080] Server: The generative AI model analyzes the input data and diagnoses the user's mental health status.

[2081] Input: Feature set

[2082] How it works: The AI ​​model performs analysis and generates a diagnosis.

[2083] Output: Diagnostic results

[2084] Server: Converts diagnostic results into an easy-to-understand format.

[2085] Input: Diagnostic results

[2086] What it does: Converts diagnostic results into a text format for the user

[2087] Output: Diagnostic results for the user

[2088] Step 7:

[2089] Results distribution

[2090] Server: Packages the diagnosis results and information on recommended medical institutions.

[2091] Input: Diagnosis results, medical institution information

[2092] How it works: Combines diagnostic results and medical information into a single package

[2093] Output: Package information

[2094] Server: Sends packaged information to the device.

[2095] Input: Package information

[2096] Action: Sends package information to the device via HTTP request

[2097] Output: Package information received on the device

[2098] Terminal: Display diagnostic results to the user.

[2099] Input: Package information

[2100] Behavior: Updates the UI in the application to display diagnosis results and medical provider information.

[2101] Output: Diagnosis results and medical institution information displayed to the user

[2102] Step 8:

[2103] Follow-up

[2104] User: Based on the diagnosis, the user can use the displayed medical information to book an appointment online or contact the clinic directly.

[2105] Input: Diagnosis results, medical institution information

[2106] What happens: User clicks on clinic appointment link and fills out appointment form

[2107] Output: Clinic appointment completed

[2108] Users: Consult the self-care resources provided by the app as needed.

[2109] Input: Diagnostic results, self-care resources

[2110] Action: User views and practices self-care resources

[2111] Output: User self-care practices

[2112] (Application example 2)

[2113] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2114] Many systems exist that accurately diagnose mental health conditions and encourage potential depression patients to seek medical treatment at appropriate medical institutions. However, there are insufficient means to effectively collect and analyze users' emotional data and present it to users in a visually and intuitive manner. Furthermore, there are also insufficient means to analyze the collected data from multiple angles and provide users with information on improvement measures and medical facilities. This has led to issues that make it difficult for users to understand the diagnosis results and take appropriate action.

[2115] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2116] In this invention, the server includes means for receiving profile information and answers to questions from the user as input, means for collecting and receiving biometric data from the smart device, means for collecting and receiving the user's emotional data through an emotion recognition engine, means for preprocessing the collected data and converting it into a format that can be input to the generative AI model, means for inputting the preprocessed data into the generative AI model and diagnosing the user's mental health condition, means for providing the user with the diagnosis results and information on recommended medical facilities, and means for visually presenting the diagnosis results to the user through a head-mounted display. This enables the collection and analysis of multifaceted data, including the user's emotional data, and enables the user to intuitively and visually receive diagnosis results, promoting consultation at an appropriate medical institution.

[2117] "User" means an individual who uses this system and provides their profile information and biometric data.

[2118] "Profile Information" means basic personal information about a user, such as age, gender, and occupation.

[2119] "Question Answers" are responses that users make to mental health-related questions posed within the system.

[2120] "Smart device" is a general term for devices that can measure and collect a user's biometric information (e.g., heart rate, exercise volume, sleep patterns).

[2121] "Biometric data" means physiological data of a user (e.g., heart rate, sleep patterns) collected from a smart device.

[2122] An "emotion recognition engine" is software or algorithms that recognize emotions from user text input, voice, gestures, etc. and generate data.

[2123] "Preprocessing collected data" refers to data processing measures to convert collected profile information, biometric data, and emotional data into a format that can be input into a generative AI model.

[2124] A "generative AI model" is a machine learning or artificial intelligence model that assesses and diagnoses a user's mental health based on input data.

[2125] "Means for diagnosing a user's mental health condition" means means for analyzing collected and pre-processed data using a generative AI model to assess a user's mental health condition.

[2126] "Medical facility" is a general term for medical institutions such as hospitals and clinics where users can receive mental health consultations.

[2127] A "head-mounted display" is a display device that provides visual information when worn by a user.

[2128] This invention is a system that uses a head-mounted display (HMD) to diagnose a user's mental health condition and encourage them to seek appropriate medical advice. The system combines the user's profile information, answers to questions, biometric data from a smart device, and emotion data obtained using an emotion recognition engine to provide a diagnosis based on a generative AI model.

[2129] Hardware / Software used

[2130] Hardware:

[2131] Head-mounted display (HMD): General-purpose HMD device

[2132] Smart devices: devices that collect biometric data such as heart rate, activity, and sleep patterns (e.g., wearable devices)

[2133] software:

[2134] Emotion Recognition Engine: An engine that recognizes emotions from user text input, voice, and gestures.

[2135] Database: Firebase Firestore or other cloud databases

[2136] Generative AI model: A machine learning model for assessing mental health status

[2137] Web framework: Server-side framework such as Flask

[2138] System operation explanation

[2139] User Data Collection:

[2140] The user puts on the HMD device and launches the application. They input their profile information through the HMD interface using voice input and gestures. They also answer questions related to their mood and physical condition. This input data is sent from the HMD to the server and stored in a database.

[2141] Health Data Sync:

[2142] Users use smart devices (e.g., wearable devices) to collect biometric data such as heart rate, exercise volume, and sleep patterns. This data is then sent to a server via the HMD and stored in a database.

[2143] Emotion data collection:

[2144] The emotion recognition engine recognizes emotions from the user's text or voice input and generates emotion data, which is also sent to the server and stored in a database.

[2145] Data preprocessing:

[2146] The server preprocesses the collected profile information, biometric data, and emotion data, converting them into a format that can be input into the generative AI model, specifically by performing data cleaning and normalization.

[2147] Inputs to generative AI models and diagnosis:

[2148] The server inputs the preprocessed data into a generative AI model to diagnose the user's mental health status, and the results are converted into an intuitively understandable format.

[2149] Delivery of diagnostic results:

[2150] The server packages the diagnosis results and recommended medical facility information and presents them visually to the user through the HMD. The diagnosis results include contact information for nearby clinics and specialists.

[2151] Follow-up:

[2152] Users can review their diagnosis and use the HMD interface to find nearby clinics, schedule appointments online if needed, and access self-care resources (e.g., relaxation techniques, exercise programs).

[2153] Examples of specific examples and prompts

[2154] Examples:

[2155] A 28-year-old male user uses the app for the first time using a general-purpose HMD device. He registers his profile information by voice input, and then synchronizes his biometric data with his smart device. The emotion recognition engine generates emotion data based on the text input, and the diagnosis is "mild depression." The user then checks the information of nearby specialists through the HMD and makes an appointment.

[2156] Example prompt sentence:

[2157] "What steps would an HMD application that collects and analyzes a user's profile information and health data to diagnose their mental state take? Please explain using a concrete example."

[2158] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2159] Step 1:

[2160] User Data Collection

[2161] The user puts on the head-mounted display and starts the application. They enter their profile information (age, gender, occupation, etc.) and answers to questions related to their mood and physical condition using voice input or gestures. This input data is sent from the device to the server and stored in a database.

[2162] Input: Profile information, answers to questions

[2163] Data calculation: voice recognition, data format conversion

[2164] Output: Profile information and question and answer data (sent to server)

[2165] Step 2:

[2166] Health Data Sync

[2167] Users collect biometric data such as heart rate, exercise volume, and sleep patterns from the past week from their smart devices (e.g., wearable devices). This data is sent from the device to a server and stored in a database.

[2168] Input: Heart rate data, exercise data, sleep pattern data

[2169] Data processing: Data synchronization, data format conversion

[2170] Output: Biometric data (sent to server)

[2171] Step 3:

[2172] Collecting Emotional Data

[2173] The emotion recognition engine analyzes the user's text and voice input to generate emotion data, which is then sent from the device to a server and stored in a database.

[2174] Input: User text input, voice input

[2175] Data Computation: Emotion Recognition, Data Generation

[2176] Output: Emotion data (sent to server)

[2177] Step 4:

[2178] Data Preprocessing

[2179] The server takes the collected profile information, question responses, biometric data, and emotional data, and performs data cleaning and normalization, converting the data into a format that can be input into a generative AI model.

[2180] Input: Profile information, question and answer data, biometric data, emotional data

[2181] Data calculations: data cleaning, data normalization

[2182] Output: Preprocessed data

[2183] Step 5:

[2184] Input to generative AI models and diagnostics

[2185] The server inputs the preprocessed data into a generative AI model to diagnose the user's mental health status, and the results are converted into an easy-to-understand format.

[2186] Input: Preprocessed data

[2187] Data calculation: feature extraction, model inference

[2188] Output: Diagnostic results

[2189] Step 6:

[2190] Distribution of diagnostic results

[2191] The server packages the diagnosis and recommended medical facility information and presents it visually to the user through the HMD, including contact information for clinics and specialists.

[2192] Input: Diagnosis results, medical facility information

[2193] Data Calculation: Data Packaging

[2194] Output: Visual presentation data (sent to HMD)

[2195] Step 7:

[2196] Follow-up

[2197] Users can check their diagnosis and medical facility information, make appointments online if necessary, and also view and use self-care resources (relaxation techniques, exercise programs, etc.).

[2198] Input: diagnosis results, medical facility information, self-care resource information

[2199] Output: Reservation information, self-care resource usage status

[2200] In this way, the system collects and analyzes data from multiple angles, provides users with intuitive and visual diagnostic results, and encourages them to seek medical attention at an appropriate medical institution, thereby achieving comprehensive mental health support.

[2201] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2202] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2203] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2204] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2205] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental s...

Claims

1. means for receiving profile information and responses to questions as input from a user; a means for collecting and receiving health data from a smart device; A means of preprocessing the collected data and converting it into a format that can be input into a generative AI model; A means of inputting the pre-processed data into a generative AI model to diagnose the user's mental health status; and A means for providing the user with information on the diagnosis results and recommended medical institutions; A system including:

2. The system of claim 1 , further comprising means for storing the user input data and health data in a database.

3. The system of claim 1 , further comprising means for providing self-care resources to the user based on the diagnosis.

Images