System

Abstract

A system is provided.SOLUTION: A system includes means for acquiring current medical images and lifestyle data, means for processing the acquired current medical images and lifestyle data and inputting the processed current medical images and lifestyle data to a generative artificial intelligence model, means for generating future medical images using the generative artificial intelligence model, means for visualizing the generated future medical images and providing the visualized future medical images to a user, means for presenting a preventive measure based on the future medical images, and means for periodically following up the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention aims to resolve the lack of awareness regarding preventive measures for brain atrophy and abnormalities. Specifically, it aims to improve the current situation in which it is difficult for users to visually understand their own future risks and encourage appropriate behavioral changes in the prevention of Alzheimer's disease and dementia. [Means for solving the problem]

[0005] The present invention provides a system for acquiring current medical images and lifestyle data, and generating future medical images based on the acquired data using a generative artificial intelligence model, which specifically includes the following means:

[0006] 1. A means of obtaining current medical imaging and lifestyle data.

[0007] 2. A means of processing the acquired current medical images and lifestyle data and inputting them into a generative artificial intelligence model.

[0008] 3. A means of generating future medical images using generative artificial intelligence models.

[0009] 4. A means to visualize the generated future medical images and provide them to users.

[0010] 5. A means of suggesting preventative measures based on future medical images.

[0011] 6. A means of following up with users on a regular basis.

[0012] This allows users to visually understand their own future brain state, raise their awareness of prevention, and take concrete measures. Furthermore, regular follow-ups can support ongoing health management and behavioral change.

[0013] "Current medical images" are image data showing the current state of the user's brain, and are taken using medical equipment such as an MRI or CT scan.

[0014] "Lifestyle data" is information about the user's daily activities and habits, and includes data such as exercise frequency, diet, sleep duration, and stress level.

[0015] A "generative artificial intelligence model" is a model that implements a machine learning algorithm to generate future medical images using current medical images and lifestyle data as input.

[0016] "Future medical images" are image data generated by predicting future brain conditions based on current medical images and lifestyle data.

[0017] "Visualization" refers to displaying the generated future medical images in a way that is understandable to the user, including annotations, highlighting, etc.

[0018] "Preventive measures" are specific action plans or suggestions presented to improve the user's lifestyle based on the generated future medical images.

[0019] "Follow-up" refers to providing notifications and support to users to periodically check their health status and encourage them to continue taking measures. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention is a system that predicts the future state of the brain based on the user's current medical images and lifestyle data, and provides the results to the user to encourage preventative care and behavioral change. Specific embodiments of the system are described below.

[0042] System Configuration

[0043] 1. User terminal: A device that allows users to input current medical images and lifestyle data. This can be a PC, tablet, or smartphone.

[0044] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs AI models.

[0045] 3. Database: A storage for storing the user's current medical images, lifestyle data, generated future medical images, and preventive measures based on them.

[0046] Program processing

[0047] The system operates through the following major processing steps:

[0048] 1. User registration and data acquisition:

[0049] User: Uploads their current medical images and lifestyle data.

[0050] Terminal: Converts the uploaded data into the appropriate format and sends it to the server.

[0051] 2. Data Processing and Future Medical Imaging:

[0052] Server: Analyzes the received data and performs image preprocessing (such as noise removal and normalization).

[0053] Server: Inputs the prepared data into the AI ​​model to generate future medical images.

[0054] 3. Visualizing and explaining the results:

[0055] Server: Stores the generated future medical images in a database and simultaneously transmits the results to the user's terminal.

[0056] Terminal: Displays future medical images and their analysis results to the user in a visually easy-to-understand format.

[0057] 4. Suggested preventive measures:

[0058] Server: Generates individually customized preventive measures based on user data and analysis results.

[0059] Device: Presents preventive measures to users and encourages specific behavioral changes.

[0060] 5. Regular follow-up:

[0061] Server: Provides periodic follow-up notifications to users, prompting them to upload new medical images and data.

[0062] Terminal: Receives follow-up notifications and displays them to the user.

[0063] Specific examples

[0064] Let us take a 50-year-old male user as an example.

[0065] 1. User: Uploads current brain images taken with an MRI device and lifestyle data indicating low exercise frequency to the device.

[0066] 2. Terminal: Sends data to the server.

[0067] 3. Server: Preprocesses MRI images and analyzes lifestyle data. This data is input into an AI model to predict the state of the brain 10 years from now.

[0068] 4. Server: Sends the generated future MRI images to the terminal and stores the results in a database.

[0069] 5. Terminal: The device displays future MRI images to the user, highlighting the tendency for hippocampal atrophy and explaining that lack of exercise is a risk factor.

[0070] 6. Server: Suggests daily walking and brain training programs as preventive measures based on the user's needs.

[0071] 7. Terminal: Presents specific exercise plans and brain training links to the user.

[0072] 8. Server: Send a follow-up notification after three months to ensure the user has implemented the suggested measures.

[0073] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[0074] This system allows users to visually understand future health risks and encourages them to take proactive preventative measures, which is expected to lead to effective behavioral changes to prevent dementia and Alzheimer's disease.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] User: Logs in to the dedicated website or mobile application, takes and acquires current medical images (MRI images, etc.), enters their lifestyle data (exercise activity, diet, sleep time, etc.) into an input form, and uploads it to the device.

[0078] Step 2:

[0079] Terminal: Receives medical images and lifestyle data uploaded by users, converts them into a specified format (e.g., a standard data format), and sends the data to the server.

[0080] Step 3:

[0081] Server: Preprocesses the received medical images, specifically removing noise from the images, performing necessary filtering and normalization, and preparing them in a format that is easy for the AI ​​model to use.

[0082] Step 4:

[0083] Server: Analyzes lifestyle data, organizes and stores it in a standardized format, and then inputs the medical images and lifestyle data into the AI ​​model.

[0084] Step 5:

[0085] Server: Generates future medical images using an AI model. The AI ​​model uses deep learning algorithms to predict and simulate future brain states based on current images and lifestyle data.

[0086] Step 6:

[0087] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[0088] Step 7:

[0089] Terminal: Displays the received future medical images and their diagnostic results to the user, highlighting and annotating specific areas to make them easier to understand visually.

[0090] Step 8:

[0091] Server: Based on future medical images and diagnostic results, it generates personalized preventative measures for users to improve their lifestyle, including exercise plans, dietary suggestions, and stress management techniques.

[0092] Step 9:

[0093] Device: Presents users with specific action plans and preventative measures, such as daily walking plans, links to brain training programs, and examples of dietary improvements.

[0094] Step 10:

[0095] Server: Schedules and sends periodic follow-up notifications to users, including reminders to re-upload medical images at the appropriate time.

[0096] Step 11:

[0097] Device: Display a follow-up notification to the user, prompting them to enter new medical images and lifestyle data, then return to step 2 and repeat the process.

[0098] The above steps result in a system that allows users to understand their own future brain health status and take effective preventative measures.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] Current medical systems lack the means to predict a user's future health status and provide appropriate preventive measures. As a result, users are not sufficiently motivated to recognize future health risks early and improve their lifestyle habits. Furthermore, there are no systems that suggest specific preventive measures based on future medical images and lifestyle data. This makes it difficult for users to take appropriate measures in response to changes in their health status.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes means for acquiring a user's current medical images and lifestyle data, means for preprocessing the acquired medical images and analyzing the lifestyle data, means for inputting the preprocessed data into a generative AI model, means for creating prompts and generating future medical images using the generative AI model, means for saving the generated future medical images in a database, means for visualizing the generated future medical images and providing them to the user, means for suggesting preventive measures based on the future medical images, and means for periodically following up with the user, thereby enabling the user to visually understand future health risks and take appropriate preventive measures early.

[0104] "User" refers to an individual who uses the system to provide their own medical images and lifestyle data and receive information about future health risks.

[0105] "Current medical images" are images that visually show the user's physical condition, and primarily refer to MRI images and CT scan images.

[0106] "Lifestyle data" is information about the user's daily activities and habits, including, for example, exercise frequency, dietary content, and sleep patterns.

[0107] "Preprocessing" refers to the processing of raw medical image data to make it analyzable, and refers to technical operations such as noise removal, image normalization, and feature extraction.

[0108] A "generative AI model" is a model that uses artificial intelligence to predict future states, and refers to a neural network trained by a learning algorithm.

[0109] A "prompt sentence" is a text-based input used to give specific instructions or queries to a generative AI model, including, for example, content that predicts future brain states.

[0110] "Future medical images" are images that visually show the user's future physical condition, predicted using a generative AI model.

[0111] "Database" refers to a data storage system that stores and efficiently manages all data used by a system.

[0112] "Visualization" refers to the presentation of data or information in a format that users can intuitively understand, and includes methods such as graphs, charts, and image displays.

[0113] "Preventive measures" are specific actions or measures that users should take to reduce future health risks, including exercise plans, dietary advice, stress management techniques, etc.

[0114] "Follow-up" is the process of tracking changes in a user's health condition and lifestyle habits at regular intervals and providing additional advice and guidance as needed.

[0115] This invention is a system that predicts the future state of the brain based on the user's current medical images and lifestyle data, and provides the results to the user to encourage preventative care and behavioral change. This system is implemented using the following hardware and software.

[0116] Hardware Configuration

[0117] 1. User terminal: A device that allows users to input current medical images and lifestyle data. Specifically, this includes PCs, tablets, smartphones, etc.

[0118] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. This is where the entire system is managed and the generative AI model is executed.

[0119] 3. Database: A storage for storing the user's current medical images, lifestyle data, generated future medical images, and preventive measures based on them.

[0120] Software configuration and processing content

[0121] 1. Data Collection:

[0122] User: Enters their current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency, dietary content, etc.).

[0123] Terminal: Converts uploaded data into an appropriate format and sends it to the server. This can be done using Python or JavaScript (registered trademark) libraries.

[0124] 2. Data Preprocessing:

[0125] Server: Preprocesses the received medical images. Preprocessing includes noise reduction, image normalization, feature extraction, etc. Specifically, OpenCV and SciPy are used.

[0126] 3. Data Analysis:

[0127] Server: Also analyzes lifestyle data. This analysis uses libraries such as Scikit-learn.

[0128] 4. Future Predictions:

[0129] Server: The preprocessed data is input into the generative AI model. This model uses tools such as PyTorch or TENSORFLOW (registered trademark). A prompt such as "Please predict the state of the brain in 10 years" is input.

[0130] 5. Save and display results:

[0131] Server: The generated future medical images are stored in a database and simultaneously sent to the user's terminal. MySQL (registered trademark) is used as the database system.

[0132] Terminal: Visually display the results sent. Use a library such as Matplotlib to generate graphs and charts.

[0133] 6. Suggested preventive measures:

[0134] Server: Generates individually customized preventive measures based on the generated future medical images and analysis results. Utilizing appropriate algorithms, the preventive measures are derived and provided to the user.

[0135] 7. Follow-up:

[0136] Server: Sends periodic follow-up notifications to prompt users to re-enter new medical images and lifestyle data. This is scheduled using a Cron job.

[0137] Terminal: Receives follow-up notifications and prompts the user to enter new data.

[0138] Specific examples

[0139] Specific processing will be explained using the case of a 50-year-old male user as an example.

[0140] 1. User: Uploads current brain images taken with an MRI device and lifestyle data indicating low exercise frequency to the device.

[0141] 2. Terminal: Sends data to the server.

[0142] 3. Server: Preprocesses MRI images and analyzes lifestyle data. This data is input into an AI model to predict the state of the brain 10 years from now.

[0143] 4. Server: Generates future MRI images, sends them to the device, and also stores them in the database.

[0144] 5. Terminal: Future MRI images are displayed, highlighting the tendency for hippocampal atrophy in particular. It is also explained that lack of exercise is a risk factor.

[0145] 6. Server: For example, suggest daily walking and a brain training program as preventative measures.

[0146] 7. Terminal: Presents specific exercise plans and brain training links to the user.

[0147] 8. Server: Send a follow-up notice after three months to confirm the continuation of the proposed measures.

[0148] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[0149] Example prompt: "A 50-year-old male user has provided us with MRI brain images and lifestyle data showing that he exercises infrequently. Predict his brain condition 10 years from now and suggest appropriate preventative measures."

[0150] This system allows users to visually understand future health risks and take appropriate preventive measures early on.

[0151] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0152] Step 1:

[0153] User registration and data acquisition

[0154] Users input their current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency, dietary content, etc.) through a dedicated application or web interface.

[0155] Input: Medical image files, lifestyle data

[0156] The terminal receives the data uploaded by the user, standardizes the format, and compresses the data as necessary. Specifically, it converts JPEG images into the standard DICOM format.

[0157] Output: Processed medical image files, lifestyle data

[0158] The terminal transmits the processed data to the server.

[0159] Step 2:

[0160] Data Preprocessing

[0161] The server preprocesses the medical images received from the device, including noise removal, image normalization, and feature extraction, using OpenCV and SciPy libraries.

[0162] Input: Processed medical image files

[0163] Data processing: noise removal, image normalization, feature extraction

[0164] Output: Preprocessed medical image data

[0165] Step 3:

[0166] Lifestyle data analysis

[0167] The server analyzes the lifestyle data. Here, detailed analysis of lifestyle habits is performed and specific parameters are extracted. Specifically, the "Scikit-learn" library is used.

[0168] Input: Lifestyle data

[0169] Data processing: Data normalization, feature extraction

[0170] Output: Analyzed lifestyle data

[0171] Step 4:

[0172] The Future of Medical Imaging

[0173] The server inputs the preprocessed medical image data and analyzed lifestyle data into a generative AI model, which uses PyTorch and TensorFlow.

[0174] Input: Preprocessed medical image data, analyzed lifestyle data

[0175] Prompt: For example, "Predict your brain state in 10 years."

[0176] Data Computation: Predictive Processing with AI Models

[0177] Output: Generated future medical images

[0178] Step 5:

[0179] Saving and displaying results

[0180] The server stores the generated future medical images using a database management system such as MySQL.

[0181] Input: Generated future medical images

[0182] Data processing: Saving to the database

[0183] Output: Future medical image data stored in a database

[0184] The server transmits the generated future medical images to the user terminal.

[0185] The terminal uses the Matplotlib library to generate graphs and charts to visually display the results.

[0186] Input: Generated future medical images

[0187] Output: Visualized medical images, graphs, charts

[0188] Step 6:

[0189] Preventive measures suggested

[0190] The server generates personalized preventative measures based on the generated future medical images and analysis results, using advanced algorithms and behavioral science theories, specifically Scikit-learn.

[0191] Input: Analysis results, future medical images

[0192] Data Computing: Proactive Measures

[0193] Output: Individually customized preventative measures

[0194] The device will provide the user with specific exercise plans, dietary advice, stress management methods, and more.

[0195] Step 7:

[0196] Regular follow-up

[0197] The server will send follow-up notifications periodically (e.g., every 3 months), including automatic scheduling using a "Cron" job.

[0198] Input: Schedule information

[0199] Data Calculation: Generate Follow-Up Notifications

[0200] Output: Follow-up notification

[0201] The device receives a follow-up notification and prompts the user to enter new data. Once the new data is uploaded, the same process begins again, starting with data collection.

[0202] (Application example 1)

[0203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] In today's world, many people face the challenge of fully understanding future health risks and taking appropriate preventative measures. Furthermore, there is a lack of methods for providing users with a visually easy-to-understand view of predicted health risks. The goal of this project is to improve this situation and promote preventative awareness and behavioral change in users.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0206] In this invention, the server includes means for acquiring current medical images and habit data, means for processing the acquired current medical images and habit data and inputting them into a generative artificial intelligence model, means for generating future medical images using the generative artificial intelligence model, means for visualizing the generated future medical images and providing them to the user through a display device, means for presenting individually customized preventive measures and action plans based on the future medical images, means for predicting future health risks and providing health management services through a virtual shop, and means for periodically sending tracking notifications to the user and prompting them to upload new data, thereby enabling the user to visually understand future health risks and take customized preventive measures based on their own lifestyle.

[0207] A "medical image" is an image of the human body taken with medical equipment and used for diagnosis or treatment.

[0208] "Habit data" is information about individual lifestyle habits, including data on diet, exercise, sleep, etc.

[0209] "Generative artificial intelligence models" are algorithms and machine learning models used to predict future states based on existing data.

[0210] A "virtual store" is a virtual store that offers goods and services via the Internet.

[0211] A "display device" is a device for visually presenting images and information to a user, and includes smartphones and head-mounted displays.

[0212] "Preventive measures" are specific actions or efforts to maintain health and prevent disease.

[0213] An "action plan" is a specific set of actions or a timetable for implementing preventative measures.

[0214] "Tracking notifications" are periodic reminders or notifications sent to users to inform them of their progress in taking preventative measures or to provide new data.

[0215] A "health management service" is a service that monitors a user's health status and provides appropriate advice and preventative measures.

[0216] This invention is a system that predicts the user's future brain state based on current medical images and habit data, and provides the results to the user to encourage preventative care and behavioral change. This system is mainly composed of a server, a terminal, and a user. A specific embodiment is described below.

[0217] System Configuration

[0218] 1. User terminal: A device that allows users to input current medical images and habit data. Examples include smartphones and head-mounted displays.

[0219] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs the generative AI model.

[0220] 3. Database: A storage for storing the user's current medical images, habit data, generated future medical images, and preventive measures based on them.

[0221] System Operation

[0222] Data Acquisition

[0223] Users upload their own medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency and dietary habits) to the device, which then converts the data into an appropriate format and sends it to the server.

[0224] Data Processing and Future Medical Imaging

[0225] The server analyzes the received medical images and habit data. First, the medical images undergo preprocessing, such as noise removal and normalization. The preprocessed data is input into a generative AI model, which generates future medical images. For example, based on current brain MRI images and data on lack of exercise, the AI ​​can predict the state of the brain 10 years from now.

[0226] Visualizing the results

[0227] The generated future medical images are stored in a database by the server and sent to the user's device, which displays them to the user in a visually easy-to-understand format. When a head-mounted display is used, the future brain state can be displayed realistically in 3D or AR.

[0228] Precautions

[0229] The server generates personalized preventive measures and action plans based on future medical images and current data. For example, if lack of exercise is a risk factor, daily walking and brain training programs will be suggested. These preventive measures will be provided through a virtual store, allowing users to receive a variety of health management services.

[0230] Regular follow-up

[0231] The server periodically sends tracking notifications to the user, prompting them to upload new data. The user then uploads medical images and lifestyle data again, and the system analyzes the new data, enabling continuous health management.

[0232] Specific examples

[0233] A 50-year-old male user uploads his current MRI scan and his low exercise habits to his device. The data is sent to a server, where a generative AI model predicts his brain state 10 years from now. Future MRI images are displayed in 3D via a head-mounted display, allowing the user to visually confirm his health risks. The server also suggests daily walking and brain training programs as preventative measures, which are offered through a virtual shopping mall.

[0234] Prompt Sentence Examples

[0235] "Enter your current MRI images and lifestyle data. We'll predict your future brain condition and visualize your health risks."

[0236] In this way, users can visually understand future health risks and take customized preventative measures based on their lifestyle.

[0237] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0238] Step 1:

[0239] The user uploads current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency) to the device via a smartphone or head-mounted display. The input data are medical image files and text data related to lifestyle habits. The device converts this data into an appropriate format and sends it to the server. The output is the data transfer to the server.

[0240] Step 2:

[0241] The server analyzes the received data. First, it preprocesses the medical image data. Preprocessing such as noise removal and normalization is performed to enable the AI ​​model to accurately analyze the data. The input is medical images and lifestyle data, and the output is the preprocessed data. Specifically, OpenCV is used to remove noise from the images and perform standardization processing.

[0242] Step 3:

[0243] The preprocessed data is input into a generative AI model to generate future medical images. Specifically, TensorFlow is used to input data into the AI ​​model and perform prediction processing. The input is the preprocessed data, and the output is the generated future medical images. The generative AI model predicts future brain health based on current medical images and lifestyle data.

[0244] Step 4:

[0245] The server stores the generated future medical images in a database and transmits them to the terminal. The input is the generated future medical image, and the output is storage in the database and transmission to the terminal. Specifically, the generated image is stored in an appropriate format and transmitted to the terminal via the Internet.

[0246] Step 5:

[0247] The device visually presents the received future medical images to the user. The input is the received future medical image, and the output is visual information provided to the user. Specifically, 3D and AR displays are performed using the display functions of a smartphone or head-mounted display.

[0248] Step 6:

[0249] The server generates personalized preventive measures and action plans based on future medical images and current data. The input is future medical images and current lifestyle data, and the output is personalized preventive measures. Specifically, it uses an algorithm that evaluates the user's risk factors and suggests optimal measures to address them.

[0250] Step 7:

[0251] The server provides users with preventive measures and action plans through a virtual shop. The input is individually customized preventive measures, and the output is services provided through the virtual shop. Specifically, users can view and purchase the suggested preventive measures through the virtual shop interface.

[0252] Step 8:

[0253] The server periodically sends tracking notifications to users to prompt them to upload new data. The input is the configured follow-up schedule, and the output is the notification message. Specifically, it uses a reminder function to send emails or alerts to prompt users to update their data.

[0254] In this way, a system is created that allows users to visually understand their health risks and implement customized preventative measures.

[0255] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0256] The present invention is a system that combines a user's current medical images and lifestyle data with an emotion engine that recognizes the user's emotional state. This makes it possible to present individually customized preventive measures that take the user's emotional state into consideration and send optimal follow-up notifications. Specific embodiments of the system are described below.

[0257] System Configuration

[0258] 1. User device: A device that allows users to input current medical images, lifestyle data, and emotional state. This can be a PC, tablet, or smartphone.

[0259] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs AI models.

[0260] 3. Emotion engine: An engine that analyzes the user's emotional state and processes data in conjunction with the generative AI model.

[0261] 4. Database: A storage for storing the user's current medical images, lifestyle data, emotional state, generated future medical images, and preventive measures based on them.

[0262] Program processing

[0263] The system operates through the following major processing steps:

[0264] 1. User registration and data acquisition:

[0265] User: Uploads their current medical images and lifestyle data, and inputs their emotional state (e.g., via a questionnaire or facial expression recognition).

[0266] Terminal: Converts the uploaded data into the appropriate format and sends it to the server.

[0267] 2. Data Processing and Future Medical Imaging:

[0268] Server: Analyzes the received data and performs image preprocessing (noise removal, normalization, etc.). It also analyzes lifestyle data and emotional state data, and organizes and stores them in a standardized format.

[0269] Server: Inputs the prepared data into the AI ​​model and emotion engine to generate future medical images.

[0270] 3. Visualizing and explaining the results:

[0271] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[0272] Terminal: Displays future medical images and their diagnostic results to the user in a visually easy-to-understand format.

[0273] 4. Suggested preventive measures:

[0274] Server: Generates individually customized preventive measures based on the user's data and analysis results. The emotion engine adjusts the way the preventive measures are presented based on the user's emotional state.

[0275] Device: Presents users with specific action plans and preventative measures, such as daily walking plans, links to brain training programs, and examples of dietary improvements.

[0276] 5. Emotional state monitoring and follow-up:

[0277] Server: The emotion engine monitors the user's emotional state in real time and sends the data to the server.

[0278] Server: Sends follow-up notifications at optimal times based on the user's emotional state.

[0279] On the device: Follow-up notifications are displayed to the user, prompting them to enter new medical images and lifestyle data.

[0280] Specific examples

[0281] Let us take a 50-year-old female user as an example.

[0282] 1. User: Uploads current brain images taken with an MRI machine, lifestyle data such as low exercise frequency, and recent stress levels to the device.

[0283] 2. Terminal: Sends data to the server.

[0284] 3. Server: Preprocesses MRI images and analyzes lifestyle and emotional state data. This data is input into an AI model to predict the brain state 10 years from now.

[0285] 4. Server: Sends the generated future MRI images to the terminal and stores the results in a database.

[0286] 5. Terminal: Future MRI images are displayed to the user, highlighting the tendency for hippocampal atrophy, while explaining that lack of exercise and high stress are risk factors.

[0287] 6. Server: Considers the user's emotional state and suggests exercise plans and stress management methods to promote relaxation.

[0288] 7. Terminal: Presents the user with specific exercise plans and brain training links for relaxation.

[0289] 8. Server: Send a follow-up notification after three months to ensure the user has implemented the suggested measures.

[0290] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[0291] This system allows users to visually understand future health risks and encourages them to take proactive preventative measures while taking into account their emotional state, which is expected to lead to effective behavioral changes to prevent dementia and Alzheimer's disease.

[0292] The processing flow will be explained below.

[0293] Step 1:

[0294] User: Logs in to a dedicated website or mobile application, takes and acquires current medical images (MRI images, etc.), enters lifestyle data (exercise frequency, diet, sleep time, stress level, etc.) and emotional state into an input form, and uploads the data to the device.

[0295] Step 2:

[0296] Terminal: Receives uploaded medical images, lifestyle data, and emotional state data, converts the data into a unified format, and sends it to the server.

[0297] Step 3:

[0298] Server: Preprocesses the received medical images, specifically removing noise from the images, performing necessary filtering and normalization, and preparing them in a format that is easy for the AI ​​model to use.

[0299] Step 4:

[0300] Server: Analyzes lifestyle data and emotional state data, organizes and stores them in a standardized format, and then inputs the medical images, lifestyle data, and emotional state data into the AI ​​model and emotion engine.

[0301] Step 5:

[0302] Server: Generates future medical images using an AI model. The AI ​​model uses deep learning algorithms to predict and simulate future brain states from current images and data.

[0303] Step 6:

[0304] Server: The emotion engine analyzes the emotional state data and monitors the user's emotional state in real time. This information is used to tailor the presentation of future medical images and preventative measures.

[0305] Step 7:

[0306] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[0307] Step 8:

[0308] Terminal: Displays the received future medical images and their diagnostic results to the user, highlighting and annotating specific areas to make them easier to understand visually.

[0309] Step 9:

[0310] Server: Generates preventive measures customized to the user's individual needs based on future medical images and diagnostic results. The emotion engine selects and adjusts the presentation of preventive measures based on the user's emotional state.

[0311] Step 10:

[0312] Device: Providing specific preventative measures to the user, such as exercise plans to promote relaxation, stress management techniques, links to brain training programs, and examples of dietary improvements.

[0313] Step 11:

[0314] Server: Schedules and sends periodic follow-up notifications to users, including reminders to re-upload new medical images and data when appropriate.

[0315] Step 12:

[0316] On the device: A follow-up notification is displayed to the user, prompting them to enter new medical images and lifestyle data as the next step, then returning to step 1 to repeat the process.

[0317] These steps result in a system that allows users to understand their future brain health status and implement individually customized preventative measures that take into account their emotional state.

[0318] Example 2

[0319] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0320] Previous systems only collected a user's medical images and lifestyle data separately, but were unable to take their emotional state into account. This made it difficult to accurately predict a user's future health risks and provide individually customized preventive measures. Furthermore, it was not possible to monitor a user's emotional state in real time and respond accordingly when proposing preventive measures or providing follow-up notifications to the user.

[0321] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0322] In this invention, the server includes: means for a user to input current medical images, lifestyle data, and emotional state data; means for transmitting the acquired current medical images, lifestyle data, and emotional state data to the server; means for preprocessing and integrating the data received by the server; means for inputting the preprocessed data into a generative AI model and an emotion engine; means for generating future medical images using the generative AI model and the emotion engine; means for visualizing the generated future medical images and providing them to a user terminal; means for presenting individually customized preventive measures based on the future medical images; means for monitoring the user's emotional state in real time and sending follow-up notifications; and means for the user who receives the follow-up notifications to input new data. This enables accurate prediction of future health risks taking into account the user's emotional state and the provision of individually customized preventive measures.

[0323] "User terminal" refers to a device that allows a user to input medical images, lifestyle data, and emotional state data and transmit them to the server. This includes PCs, tablets, smartphones, etc.

[0324] A "server" is a core device that receives data sent from user terminals and processes and analyzes the data.

[0325] "Medical images" are video data that show the internal state of a user's body, such as MRI images and CT scan images.

[0326] "Lifestyle data" refers to data related to the user's daily activities and habits, such as exercise time and dietary details.

[0327] "Emotional state data" refers to data relating to the user's current emotional and mood state, including stress levels and mood states.

[0328] "Preprocessing" refers to the process of converting received data (medical images, lifestyle data, emotional state data) into a format that is easier to analyze. Specifically, this includes noise removal and normalization.

[0329] A "generative AI model" is an artificial intelligence model that generates future medical images based on user data.

[0330] The "emotion engine" is an engine that analyzes the user's emotional state and reflects this in the analysis results of the generative AI model and the way in which preventive measures are presented.

[0331] "Future medical images" are images that show the user's future medical condition, predicted using a generative AI model.

[0332] "Visualization" refers to the visual display of data and analytical results using a computer or display.

[0333] "Preventive measures" are specific instructions or action plans provided to users to avoid future health risks.

[0334] A "follow-up notification" is a notification that confirms whether the user is taking the suggested preventative measures and encourages further action.

[0335] The present invention is a system that combines a user's current medical image, lifestyle data, and emotional state data to generate a future medical image and provide individually customized preventive measures, allowing the user to identify health risks in advance and take appropriate measures.

[0336] System Configuration

[0337] 1. User Device

[0338] User terminal: A device through which a user inputs medical images, lifestyle data, and emotional state data. Specifically, this applies to PCs, tablets, and smartphones.

[0339] Users upload current medical images taken with an MRI machine and enter lifestyle data such as daily exercise time, dietary habits, and stress levels. This data is then formatted and sent from the device to the server.

[0340] 2. Server

[0341] Server: This is the core device that receives, processes, and analyzes data sent from user devices. The server performs preprocessing such as noise reduction and normalization on the received medical image data, and analyzes lifestyle and emotional state data. The preprocessed data is then input into the generative AI model and emotion engine to generate future medical images.

[0342] Specifically, the server uses the following software:

[0343] Preprocess medical images using an image processing library (e.g., OpenCV).

[0344] Use a data analysis library (e.g., Pandas) to standardize and integrate lifestyle data and emotional state data.

[0345] Deep learning frameworks (e.g., TensorFlow, PyTorch) are used to run generative AI models and generate future medical images.

[0346] 3. Database and visualization system

[0347] Database: A storage for storing the user's current medical images, lifestyle data, emotional state data, generated future medical images, and preventive measures based on them.

[0348] Visualization system: This system visually displays the generated future medical images, analysis results, and preventive measures on the user's device. The data is converted into PNG or JPEG format and provided to the user with detailed annotations.

[0349] 4. Proposal of preventive measures

[0350] Based on the generated future medical images and analysis results, the server generates personalized preventative measures, including exercise plans, dietary suggestions, and stress management techniques, all of which reflect the user's emotional state.

[0351] On your device: View detailed prevention measures, including specific action plans and links.

[0352] 5. Follow-up notifications and emotional state monitoring

[0353] The server uses an emotion engine to monitor the user's emotional state in real time and send follow-up notifications at optimal times.

[0354] On your device: Receive follow-up notifications to prompt you to enter new medical images and lifestyle data.

[0355] Specific examples

[0356] Let us take a 50-year-old female user as an example.

[0357] 1. User: The user inputs into the terminal the current brain image taken with an MRI device, lifestyle data such as low exercise frequency and high stress level.

[0358] 2. Terminal: Formats the data and sends it to the server.

[0359] 3. Server: The received MRI images are denoised and normalised, and the lifestyle and emotional state data are analysed. Based on this data, a generative AI model is used to predict the brain state 10 years from now.

[0360] 4. Server: The generated MRI images from 10 years later are stored in a database and sent to the user's terminal.

[0361] 5. Terminal: Displays future MRI images showing hippocampal atrophy and explains that lack of exercise and high stress are risk factors.

[0362] 6. Server: Considers the user's emotional state and suggests exercise plans and stress management methods to promote relaxation.

[0363] 7. Device: Displays specific exercise plans and links to relaxation.

[0364] 8. Server: Send a follow-up notification after 3 months and ask the user to enter the data again.

[0365] Prompt Sentence Examples

[0366] "A 50-year-old female user has provided lifestyle data showing that she exercises infrequently and has a high stress level, and has uploaded current brain images taken with an MRI machine. Please predict her future brain state and suggest an exercise plan to promote relaxation and stress management methods. Also, please include a process for sending a follow-up notification three months later to check whether the suggested measures have been implemented."

[0367] As described above, the present invention realizes prediction of future health risks and provision of individually customized preventive measures while taking into account the user's emotional state.

[0368] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0369] Step 1: User registration and data acquisition

[0370] User: Enters their own medical images (e.g., MRI images), lifestyle data (e.g., exercise frequency and dietary content), and emotional state data (e.g., stress level) using a PC, tablet, or smartphone.

[0371] Input: Medical image data, lifestyle data, emotional state data.

[0372] Output: Formatted data.

[0373] Specific operation: The user launches the application on the device they are using, enters data in the specified format, and presses the send button.

[0374] Step 2: Sending data

[0375] Terminal: Checks the data entered by the user, converts it into the appropriate format, and sends it to the server.

[0376] Input: User-entered medical image data, lifestyle data, and emotional state data.

[0377] Output: The formatted data sent to the server.

[0378] Specific operation: The terminal uses a communication protocol to send input data to the server in JSON format, DICOM format, etc.

[0379] Step 3: Receiving and Preprocessing Data

[0380] Server: Preprocesses the received data. Medical image data is denoised and normalised, while lifestyle data and emotional state data are analysed and standardised.

[0381] Input: Submitted medical image data, lifestyle data, and emotional state data.

[0382] Output: The preprocessed dataset.

[0383] Specific operation: The server denoises medical images and analyzes and standardizes lifestyle data and emotional state data using Pandas.

[0384] Step 4: Integrate the data

[0385] Server: Integrates pre-processed medical image data, lifestyle data, and emotional state data into a single dataset.

[0386] Input: Preprocessed individual data.

[0387] Output: A consolidated dataset.

[0388] Specific operation: The server consolidates all preprocessed data and converts it into a format that can be input into the AI ​​model.

[0389] Step 5: Analysis by AI model and emotion engine

[0390] Server: Inputs the integrated dataset into the generative AI model and emotion engine to generate future medical images.

[0391] Input: Unified dataset.

[0392] Output: Futuristic medical images.

[0393] Specific operation: The AI ​​model is run on the server using TensorFlow or PyTorch, and analysis is performed in conjunction with the emotion engine. Predictions are made of the brain's state 10 years from now.

[0394] Step 6: Save and send the results

[0395] Server: Stores the generated future medical images in a database and sends the analysis results to the user's device.

[0396] Input: futuristic medical images.

[0397] Output: Analysis results sent to the user's device.

[0398] Specific operation: The results are converted into PNG or JPEG format and sent to be displayed in a user-friendly format.

[0399] Step 7: Visualize the results

[0400] Terminal: Visually displays future medical images and diagnostic results to the user.

[0401] Input: Analysis results sent from the server.

[0402] Output: The results that are displayed to the user.

[0403] Specific operation: The device displays the received images and data in a dashboard or slideshow format, informing the user of the latest results.

[0404] Step 8: Generate and propose preventative measures

[0405] Server: Generates personalized preventive measures based on the analysis results, and adjusts the recommendations based on the user's emotional state.

[0406] Input: Analysis results, emotion engine results.

[0407] Output: Customized precautions.

[0408] Specific actions: The server proposes specific exercise plans and stress management techniques based on the generated data.

[0409] Step 9: Displaying preventative measures

[0410] Device: Presents users with specific action plans and preventative measures (e.g., exercise plans, dietary suggestions, stress management links).

[0411] Input: Prevention data sent from the server.

[0412] Output: The precautions presented to the user.

[0413] Specific action: Display preventative measures on the device in the form of a pop-up notification or detailed PDF.

[0414] Step 10: Follow-up notifications and data updates

[0415] Server: Monitors emotional state in real time and sends follow-up notifications periodically.

[0416] Input: Real-time data from the emotion engine, new input data from users.

[0417] Output: Follow-up notification, updated dataset.

[0418] What it does: Sends reminders every three months to prompt users to enter new medical images and lifestyle data.

[0419] Through these steps, the system can predict the user's future health risks and provide appropriate preventative measures that take into account their emotional state.

[0420] (Application example 2)

[0421] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0422] In today's world, maintaining personal health requires a system that can centrally manage medical and lifestyle data and suggest appropriate preventive measures. However, conventional systems rely only on simple lifestyle and medical data without considering the user's emotional state, which is often overlooked, making it difficult to suggest and implement effective preventive measures. Furthermore, they lack specific action plans and support to help users implement the proposed preventive measures, and follow-up is insufficient.

[0423] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0424] In this invention, the server includes: means for acquiring current medical images, lifestyle data, and emotional states; means for processing the acquired current medical images, lifestyle data, and emotional states and inputting them into a generative artificial intelligence model; means for generating future medical images using the generative artificial intelligence model; means for visualizing the generated future medical images and providing them to the user; means for presenting individually customized preventive measures based on the future medical images; means for periodically monitoring the user's emotional state and sending follow-up notifications at optimal times; and means for generating a meal plan based on the user's emotional state and providing the user with specific meal suggestions and delivery links. This makes it possible to support the proposal and implementation of effective, individually customized preventive measures based on lifestyle and medical images while taking the user's emotional state into consideration. Furthermore, providing specific meal suggestions and delivery links makes it easier for the user to take action immediately, thereby increasing the effectiveness of the preventive measures.

[0425] A "medical image" is an image of the internal structures of the human body taken with a medical device.

[0426] "Lifestyle data" refers to data relating to the user's lifestyle, such as exercise status, dietary content, and sleep patterns.

[0427] "Emotional state" is data that indicates the user's psychological state, and includes information such as stress level and happiness.

[0428] A "generative artificial intelligence model" is a machine learning model that predicts future states based on acquired data.

[0429] "Future medical images" are future medical images predicted based on current medical images, lifestyle data, and emotional state.

[0430] "Visualization" refers to visually displaying the generated data so that the user can easily understand it.

[0431] "Preventive measures" are specific action plans or advice to take to prevent future deterioration of a medical condition.

[0432] "Follow-up notifications" are notifications that track the implementation status of preventive measures and provide reminders or additional guidance to users at appropriate times.

[0433] A "meal plan" is a plan that suggests healthy meals based on the user's health and emotional state.

[0434] "Delivery Link" means an internet link that allows for easy ordering of the suggested meal.

[0435] The invention is a system that integrates and analyzes a user's medical images, lifestyle data, and emotional state, thereby enabling personalized preventative measures and dietary recommendations.

[0436] The system consists of the following main components:

[0437] 1. User Device:

[0438] Users use their smartphones to input their medical images, lifestyle data, and emotional state. This data is then sent to a server. For example, users can use a health app to record their daily exercise routine, take photos of their meals, and answer questionnaires about their emotional state.

[0439] 2. Server:

[0440] It analyzes the received data and generates future medical images using AI models and an emotion engine, while also monitoring the user's emotional state and sending follow-up notifications at optimal times.

[0441] 3. Emotion Engine:

[0442] This is an engine for analyzing the user's emotional state. It analyzes the user's emotions based on data such as photos, videos, and questionnaires, and sends the data to a server.

[0443] 4. Database:

[0444] It stores the user's current medical images, lifestyle data, and emotional state, as well as any future medical images and preventative measures that are generated.

[0445] 5. Meal Plan Generator:

[0446] A subsystem for suggesting appropriate meal plans to users based on their emotional state and providing delivery links.

[0447] Program processing explanation

[0448] 1. Hardware and software used:

[0449] User device: Smartphone (health app)

[0450] Server: Data analysis, collaboration with AI models and emotion engines, meal plan generation

[0451] Emotion Engine API: Analyzing the user's emotional state

[0452] Database: storing and managing user data

[0453] 2. Data processing and calculation:

[0454] The server receives data (medical images, lifestyle data, and emotional state) sent from the user's device and performs processes such as data cleansing, noise removal, and standardization. The data is then input into a generative AI model to generate future medical images. An emotion engine is also used to analyze the user's emotional state, and a meal plan is generated based on the results.

[0455] Specific examples

[0456] For example, a user can upload a photo of themselves using a smartphone app and answer a questionnaire about their recent stress level. This data is sent to a server, and the emotion engine analyzes it as a "high stress state." As a result, the server suggests a relaxing meal to the user, such as "chamomile tea, salmon, and avocado," and provides a delivery link for ordering it on the spot.

[0457] Prompt Sentence Examples

[0458] "Analyze the user's emotional state and suggest foods that will help them relax when they are under high stress."

[0459] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0460] Step 1:

[0461] Data entry and submission

[0462] Users use a smartphone app to input their own medical images, lifestyle data, and emotional state, including uploading MRI images, answering questionnaires about exercise frequency and diet, and recording their emotional state through facial recognition.

[0463] This data is converted into an appropriate format and sent to the server.

[0464] Input: medical images, lifestyle data, emotional state

[0465] Output: Data sent to the server

[0466] Step 2:

[0467] Data reception and preprocessing

[0468] The server receives data sent from the user terminal, and the received data is first preprocessed.

[0469] Specifically, this includes denoising and normalising medical images, standardising lifestyle data and analyzing emotional states.

[0470] Input: Data sent from the user's device

[0471] Output: Preprocessed data

[0472] Step 3:

[0473] AI models for generating future medical images

[0474] The server uses the pre-processed data to input the AI ​​model and generate future medical images.

[0475] For example, based on current MRI images and lifestyle data, the state of the brain 10 years from now can be predicted.

[0476] Input: Preprocessed medical images, lifestyle data, emotional states

[0477] Output: Medical images of the future

[0478] Step 4:

[0479] Emotional state analysis using emotion engine

[0480] The server analyzes the user's emotional state using an emotion engine, which determines the user's current emotional state based on questionnaire responses and facial expression recognition data.

[0481] Input: User's emotional state data

[0482] Output: Parsed emotional state

[0483] Step 5:

[0484] Visualizing and presenting results

[0485] The server transmits the generated future medical images and the analysis results of the emotional state to the user's device, which displays them in a visually easy-to-understand format.

[0486] For example, it can display future MRI images and highlight abnormalities, and also provide explanations based on the patient's emotional state.

[0487] Input: Future medical images, emotional state

[0488] Output: The results displayed on the user's terminal

[0489] Step 6:

[0490] Personalized prevention recommendations

[0491] Based on the generated future medical images and emotional state, the server suggests preventative measures, such as exercise plans, dietary modifications, and stress management techniques.

[0492] Specifically, if the user is found to be in a state of high stress, the system will suggest a meal plan that will have a relaxing effect.

[0493] Input: Future medical images, emotional state

[0494] Output: Individually customized preventative measures

[0495] Step 7:

[0496] Meal plans and delivery links provided

[0497] The server generates a meal plan based on the emotional state and provides the user with a delivery link.

[0498] For example, if a user is experiencing high stress, the app will suggest a relaxing meal such as "chamomile tea, salmon, and avocado" and provide a link to easily order it.

[0499] Input: Emotional state

[0500] Output: Meal plan, delivery link

[0501] Step 8:

[0502] Sending follow-up notifications

[0503] The server periodically monitors the user's emotional state and sends optimal follow-up notifications, including reminders to enter new medical images and lifestyle data.

[0504] Input: User's emotional state data

[0505] Output: Follow-up notification

[0506] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0507] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0508] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0509] [Second embodiment]

[0510] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0511] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0512] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0513] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0514] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0515] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0516] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0517] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0518] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0519] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0520] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0521] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0522] The present invention is a system that predicts the future state of the brain based on the user's current medical images and lifestyle data, and provides the results to the user to encourage preventative care and behavioral change. Specific embodiments of the system are described below.

[0523] System Configuration

[0524] 1. User terminal: A device that allows users to input current medical images and lifestyle data. This can be a PC, tablet, or smartphone.

[0525] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs AI models.

[0526] 3. Database: A storage for storing the user's current medical images, lifestyle data, generated future medical images, and preventive measures based on them.

[0527] Program processing

[0528] The system operates through the following major processing steps:

[0529] 1. User registration and data acquisition:

[0530] User: Uploads their current medical images and lifestyle data.

[0531] Terminal: Converts the uploaded data into the appropriate format and sends it to the server.

[0532] 2. Data Processing and Future Medical Imaging:

[0533] Server: Analyzes the received data and performs image preprocessing (such as noise removal and normalization).

[0534] Server: Inputs the prepared data into the AI ​​model to generate future medical images.

[0535] 3. Visualizing and explaining the results:

[0536] Server: Stores the generated future medical images in a database and simultaneously transmits the results to the user's terminal.

[0537] Terminal: Displays future medical images and their analysis results to the user in a visually easy-to-understand format.

[0538] 4. Suggested preventive measures:

[0539] Server: Generates individually customized preventive measures based on user data and analysis results.

[0540] Device: Presents preventive measures to users and encourages specific behavioral changes.

[0541] 5. Regular follow-up:

[0542] Server: Provides periodic follow-up notifications to users, prompting them to upload new medical images and data.

[0543] Terminal: Receives follow-up notifications and displays them to the user.

[0544] Specific examples

[0545] Let us take a 50-year-old male user as an example.

[0546] 1. User: Uploads current brain images taken with an MRI device and lifestyle data indicating low exercise frequency to the device.

[0547] 2. Terminal: Sends data to the server.

[0548] 3. Server: Preprocesses MRI images and analyzes lifestyle data. This data is input into an AI model to predict the state of the brain 10 years from now.

[0549] 4. Server: Sends the generated future MRI images to the terminal and stores the results in a database.

[0550] 5. Terminal: The device displays future MRI images to the user, highlighting the tendency for hippocampal atrophy and explaining that lack of exercise is a risk factor.

[0551] 6. Server: Suggests daily walking and brain training programs as preventive measures based on the user's needs.

[0552] 7. Terminal: Presents specific exercise plans and brain training links to the user.

[0553] 8. Server: Send a follow-up notification after three months to ensure the user has implemented the suggested measures.

[0554] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[0555] This system allows users to visually understand future health risks and encourages them to take proactive preventative measures, which is expected to lead to effective behavioral changes to prevent dementia and Alzheimer's disease.

[0556] The processing flow will be explained below.

[0557] Step 1:

[0558] User: Logs in to the dedicated website or mobile application, takes and acquires current medical images (MRI images, etc.), enters their lifestyle data (exercise activity, diet, sleep time, etc.) into an input form, and uploads it to the device.

[0559] Step 2:

[0560] Terminal: Receives medical images and lifestyle data uploaded by users, converts them into a specified format (e.g., a standard data format), and sends the data to the server.

[0561] Step 3:

[0562] Server: Preprocesses the received medical images, specifically removing noise from the images, performing necessary filtering and normalization, and preparing them in a format that is easy for the AI ​​model to use.

[0563] Step 4:

[0564] Server: Analyzes lifestyle data, organizes and stores it in a standardized format, and then inputs the medical images and lifestyle data into the AI ​​model.

[0565] Step 5:

[0566] Server: Generates future medical images using an AI model. The AI ​​model uses deep learning algorithms to predict and simulate future brain states based on current images and lifestyle data.

[0567] Step 6:

[0568] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[0569] Step 7:

[0570] Terminal: Displays the received future medical images and their diagnostic results to the user, highlighting and annotating specific areas to make them easier to understand visually.

[0571] Step 8:

[0572] Server: Based on future medical images and diagnostic results, it generates personalized preventative measures for users to improve their lifestyle, including exercise plans, dietary suggestions, and stress management techniques.

[0573] Step 9:

[0574] Device: Presents users with specific action plans and preventative measures, such as daily walking plans, links to brain training programs, and examples of dietary improvements.

[0575] Step 10:

[0576] Server: Schedules and sends periodic follow-up notifications to users, including reminders to re-upload medical images at the appropriate time.

[0577] Step 11:

[0578] Device: Display a follow-up notification to the user, prompting them to enter new medical images and lifestyle data, then return to step 2 and repeat the process.

[0579] The above steps result in a system that allows users to understand their own future brain health status and take effective preventative measures.

[0580] Example 1

[0581] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0582] Current medical systems lack the means to predict a user's future health status and provide appropriate preventive measures. As a result, users are not sufficiently motivated to recognize future health risks early and improve their lifestyle habits. Furthermore, there are no systems that suggest specific preventive measures based on future medical images and lifestyle data. This makes it difficult for users to take appropriate measures in response to changes in their health status.

[0583] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0584] In this invention, the server includes means for acquiring a user's current medical images and lifestyle data, means for preprocessing the acquired medical images and analyzing the lifestyle data, means for inputting the preprocessed data into a generative AI model, means for creating prompts and generating future medical images using the generative AI model, means for saving the generated future medical images in a database, means for visualizing the generated future medical images and providing them to the user, means for suggesting preventive measures based on the future medical images, and means for periodically following up with the user, thereby enabling the user to visually understand future health risks and take appropriate preventive measures early.

[0585] "User" refers to an individual who uses the system to provide their own medical images and lifestyle data and receive information about future health risks.

[0586] "Current medical images" are images that visually show the user's physical condition, and primarily refer to MRI images and CT scan images.

[0587] "Lifestyle data" is information about the user's daily activities and habits, including, for example, exercise frequency, dietary content, and sleep patterns.

[0588] "Preprocessing" refers to the processing of raw medical image data to make it analyzable, and refers to technical operations such as noise removal, image normalization, and feature extraction.

[0589] A "generative AI model" is a model that uses artificial intelligence to predict future states, and refers to a neural network trained by a learning algorithm.

[0590] A "prompt sentence" is a text-based input used to give specific instructions or queries to a generative AI model, including, for example, content that predicts future brain states.

[0591] "Future medical images" are images that visually show the user's future physical condition, predicted using a generative AI model.

[0592] "Database" refers to a data storage system that stores and efficiently manages all data used by a system.

[0593] "Visualization" refers to the presentation of data or information in a format that users can intuitively understand, and includes methods such as graphs, charts, and image displays.

[0594] "Preventive measures" are specific actions or measures that users should take to reduce future health risks, including exercise plans, dietary advice, stress management techniques, etc.

[0595] "Follow-up" is the process of tracking changes in a user's health condition and lifestyle habits at regular intervals and providing additional advice and guidance as needed.

[0596] This invention is a system that predicts the future state of the brain based on the user's current medical images and lifestyle data, and provides the results to the user to encourage preventative care and behavioral change. This system is implemented using the following hardware and software.

[0597] Hardware Configuration

[0598] 1. User terminal: A device that allows users to input current medical images and lifestyle data. Specifically, this includes PCs, tablets, smartphones, etc.

[0599] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. This is where the entire system is managed and the generative AI model is executed.

[0600] 3. Database: A storage for storing the user's current medical images, lifestyle data, generated future medical images, and preventive measures based on them.

[0601] Software configuration and processing content

[0602] 1. Data Collection:

[0603] User: Enters their current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency, dietary content, etc.).

[0604] Terminal: Converts uploaded data into a suitable format and sends it to the server. This can be done using Python or JavaScript libraries.

[0605] 2. Data Preprocessing:

[0606] Server: Preprocesses the received medical images. Preprocessing includes noise reduction, image normalization, feature extraction, etc. Specifically, OpenCV and SciPy are used.

[0607] 3. Data Analysis:

[0608] Server: Also analyzes lifestyle data. This analysis uses libraries such as Scikit-learn.

[0609] 4. Future Predictions:

[0610] Server: The preprocessed data is input into a generative AI model. This model uses PyTorch or TensorFlow. A prompt such as "Please predict the state of the brain in 10 years" is input.

[0611] 5. Save and display results:

[0612] Server: The generated future medical images are stored in a database and simultaneously sent to the user's terminal. MySQL is used as the database system.

[0613] Terminal: Visually display the results sent. Use a library such as Matplotlib to generate graphs and charts.

[0614] 6. Suggested preventive measures:

[0615] Server: Generates individually customized preventive measures based on the generated future medical images and analysis results. Utilizing appropriate algorithms, the preventive measures are derived and provided to the user.

[0616] 7. Follow-up:

[0617] Server: Sends periodic follow-up notifications to prompt users to re-enter new medical images and lifestyle data. This is scheduled using a Cron job.

[0618] Terminal: Receives follow-up notifications and prompts the user to enter new data.

[0619] Specific examples

[0620] Specific processing will be explained using the case of a 50-year-old male user as an example.

[0621] 1. User: Uploads current brain images taken with an MRI device and lifestyle data indicating low exercise frequency to the device.

[0622] 2. Terminal: Sends data to the server.

[0623] 3. Server: Preprocesses MRI images and analyzes lifestyle data. This data is input into an AI model to predict the state of the brain 10 years from now.

[0624] 4. Server: Generates future MRI images, sends them to the device, and also stores them in the database.

[0625] 5. Terminal: Future MRI images are displayed, highlighting the tendency for hippocampal atrophy in particular. It is also explained that lack of exercise is a risk factor.

[0626] 6. Server: For example, suggest daily walking and a brain training program as preventative measures.

[0627] 7. Terminal: Presents specific exercise plans and brain training links to the user.

[0628] 8. Server: Send a follow-up notice after three months to confirm the continuation of the proposed measures.

[0629] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[0630] Example prompt: "A 50-year-old male user has provided us with MRI brain images and lifestyle data showing that he exercises infrequently. Predict his brain condition 10 years from now and suggest appropriate preventative measures."

[0631] This system allows users to visually understand future health risks and take appropriate preventive measures early on.

[0632] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0633] Step 1:

[0634] User registration and data acquisition

[0635] Users input their current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency, dietary content, etc.) through a dedicated application or web interface.

[0636] Input: Medical image files, lifestyle data

[0637] The terminal receives the data uploaded by the user, standardizes the format, and compresses the data as necessary. Specifically, it converts JPEG images into the standard DICOM format.

[0638] Output: Processed medical image files, lifestyle data

[0639] The terminal transmits the processed data to the server.

[0640] Step 2:

[0641] Data Preprocessing

[0642] The server preprocesses the medical images received from the device, including noise removal, image normalization, and feature extraction, using OpenCV and SciPy libraries.

[0643] Input: Processed medical image files

[0644] Data processing: noise removal, image normalization, feature extraction

[0645] Output: Preprocessed medical image data

[0646] Step 3:

[0647] Lifestyle data analysis

[0648] The server analyzes the lifestyle data. Here, detailed analysis of lifestyle habits is performed and specific parameters are extracted. Specifically, the "Scikit-learn" library is used.

[0649] Input: Lifestyle data

[0650] Data processing: Data normalization, feature extraction

[0651] Output: Analyzed lifestyle data

[0652] Step 4:

[0653] The Future of Medical Imaging

[0654] The server inputs the preprocessed medical image data and analyzed lifestyle data into a generative AI model, which uses PyTorch and TensorFlow.

[0655] Input: Preprocessed medical image data, analyzed lifestyle data

[0656] Prompt: For example, "Predict your brain state in 10 years."

[0657] Data Computation: Predictive Processing with AI Models

[0658] Output: Generated future medical images

[0659] Step 5:

[0660] Saving and displaying results

[0661] The server stores the generated future medical images using a database management system such as MySQL.

[0662] Input: Generated future medical images

[0663] Data processing: Saving to the database

[0664] Output: Future medical image data stored in a database

[0665] The server transmits the generated future medical images to the user terminal.

[0666] The terminal uses the Matplotlib library to generate graphs and charts to visually display the results.

[0667] Input: Generated future medical images

[0668] Output: Visualized medical images, graphs, charts

[0669] Step 6:

[0670] Preventive measures suggested

[0671] The server generates personalized preventative measures based on the generated future medical images and analysis results, using advanced algorithms and behavioral science theories, specifically Scikit-learn.

[0672] Input: Analysis results, future medical images

[0673] Data Computing: Proactive Measures

[0674] Output: Individually customized preventative measures

[0675] The device will provide the user with specific exercise plans, dietary advice, stress management methods, and more.

[0676] Step 7:

[0677] Regular follow-up

[0678] The server will send follow-up notifications periodically (e.g., every 3 months), including automatic scheduling using a "Cron" job.

[0679] Input: Schedule information

[0680] Data Calculation: Generate Follow-Up Notifications

[0681] Output: Follow-up notification

[0682] The device receives a follow-up notification and prompts the user to enter new data. Once the new data is uploaded, the same process begins again, starting with data collection.

[0683] (Application example 1)

[0684] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0685] In today's world, many people face the challenge of fully understanding future health risks and taking appropriate preventative measures. Furthermore, there is a lack of methods for providing users with a visually easy-to-understand view of predicted health risks. The goal of this project is to improve this situation and promote preventative awareness and behavioral change in users.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0687] In this invention, the server includes means for acquiring current medical images and habit data, means for processing the acquired current medical images and habit data and inputting them into a generative artificial intelligence model, means for generating future medical images using the generative artificial intelligence model, means for visualizing the generated future medical images and providing them to the user through a display device, means for presenting individually customized preventive measures and action plans based on the future medical images, means for predicting future health risks and providing health management services through a virtual shop, and means for periodically sending tracking notifications to the user and prompting them to upload new data, thereby enabling the user to visually understand future health risks and take customized preventive measures based on their own lifestyle.

[0688] A "medical image" is an image of the human body taken with medical equipment and used for diagnosis or treatment.

[0689] "Habit data" is information about individual lifestyle habits, including data on diet, exercise, sleep, etc.

[0690] "Generative artificial intelligence models" are algorithms and machine learning models used to predict future states based on existing data.

[0691] A "virtual store" is a virtual store that offers goods and services via the Internet.

[0692] A "display device" is a device for visually presenting images and information to a user, and includes smartphones and head-mounted displays.

[0693] "Preventive measures" are specific actions or efforts to maintain health and prevent disease.

[0694] An "action plan" is a specific set of actions or a timetable for implementing preventative measures.

[0695] "Tracking notifications" are periodic reminders or notifications sent to users to inform them of their progress in taking preventative measures or to provide new data.

[0696] A "health management service" is a service that monitors a user's health status and provides appropriate advice and preventative measures.

[0697] This invention is a system that predicts the user's future brain state based on current medical images and habit data, and provides the results to the user to encourage preventative care and behavioral change. This system is mainly composed of a server, a terminal, and a user. A specific embodiment is described below.

[0698] System Configuration

[0699] 1. User terminal: A device that allows users to input current medical images and habit data. Examples include smartphones and head-mounted displays.

[0700] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs the generative AI model.

[0701] 3. Database: A storage for storing the user's current medical images, habit data, generated future medical images, and preventive measures based on them.

[0702] System Operation

[0703] Data Acquisition

[0704] Users upload their own medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency and dietary habits) to the device, which then converts the data into an appropriate format and sends it to the server.

[0705] Data Processing and Future Medical Imaging

[0706] The server analyzes the received medical images and habit data. First, the medical images undergo preprocessing, such as noise removal and normalization. The preprocessed data is input into a generative AI model, which generates future medical images. For example, based on current brain MRI images and data on lack of exercise, the AI ​​can predict the state of the brain 10 years from now.

[0707] Visualizing the results

[0708] The generated future medical images are stored in a database by the server and sent to the user's device, which displays them to the user in a visually easy-to-understand format. When a head-mounted display is used, the future brain state can be displayed realistically in 3D or AR.

[0709] Precautions

[0710] The server generates personalized preventive measures and action plans based on future medical images and current data. For example, if lack of exercise is a risk factor, daily walking and brain training programs will be suggested. These preventive measures will be provided through a virtual store, allowing users to receive a variety of health management services.

[0711] Regular follow-up

[0712] The server periodically sends tracking notifications to the user, prompting them to upload new data. The user then uploads medical images and lifestyle data again, and the system analyzes the new data, enabling continuous health management.

[0713] Specific examples

[0714] A 50-year-old male user uploads his current MRI scan and his low exercise habits to his device. The data is sent to a server, where a generative AI model predicts his brain state 10 years from now. Future MRI images are displayed in 3D via a head-mounted display, allowing the user to visually confirm his health risks. The server also suggests daily walking and brain training programs as preventative measures, which are offered through a virtual shopping mall.

[0715] Prompt Sentence Examples

[0716] "Enter your current MRI images and lifestyle data. We'll predict your future brain condition and visualize your health risks."

[0717] In this way, users can visually understand future health risks and take customized preventative measures based on their lifestyle.

[0718] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0719] Step 1:

[0720] The user uploads current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency) to the device via a smartphone or head-mounted display. The input data are medical image files and text data related to lifestyle habits. The device converts this data into an appropriate format and sends it to the server. The output is the data transfer to the server.

[0721] Step 2:

[0722] The server analyzes the received data. First, it preprocesses the medical image data. Preprocessing such as noise removal and normalization is performed to enable the AI ​​model to accurately analyze the data. The input is medical images and lifestyle data, and the output is the preprocessed data. Specifically, OpenCV is used to remove noise from the images and perform standardization processing.

[0723] Step 3:

[0724] The preprocessed data is input into a generative AI model to generate future medical images. Specifically, TensorFlow is used to input data into the AI ​​model and perform prediction processing. The input is the preprocessed data, and the output is the generated future medical images. The generative AI model predicts future brain health based on current medical images and lifestyle data.

[0725] Step 4:

[0726] The server stores the generated future medical images in a database and transmits them to the terminal. The input is the generated future medical image, and the output is storage in the database and transmission to the terminal. Specifically, the generated image is stored in an appropriate format and transmitted to the terminal via the Internet.

[0727] Step 5:

[0728] The device visually presents the received future medical images to the user. The input is the received future medical image, and the output is visual information provided to the user. Specifically, 3D and AR displays are performed using the display functions of a smartphone or head-mounted display.

[0729] Step 6:

[0730] The server generates personalized preventive measures and action plans based on future medical images and current data. The input is future medical images and current lifestyle data, and the output is personalized preventive measures. Specifically, it uses an algorithm that evaluates the user's risk factors and suggests optimal measures to address them.

[0731] Step 7:

[0732] The server provides users with preventive measures and action plans through a virtual shop. The input is individually customized preventive measures, and the output is services provided through the virtual shop. Specifically, users can view and purchase the suggested preventive measures through the virtual shop interface.

[0733] Step 8:

[0734] The server periodically sends tracking notifications to users to prompt them to upload new data. The input is the configured follow-up schedule, and the output is the notification message. Specifically, it uses a reminder function to send emails or alerts to prompt users to update their data.

[0735] In this way, a system is created that allows users to visually understand their health risks and implement customized preventative measures.

[0736] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0737] The present invention is a system that combines a user's current medical images and lifestyle data with an emotion engine that recognizes the user's emotional state. This makes it possible to present individually customized preventive measures that take the user's emotional state into consideration and send optimal follow-up notifications. Specific embodiments of the system are described below.

[0738] System Configuration

[0739] 1. User device: A device that allows users to input current medical images, lifestyle data, and emotional state. This can be a PC, tablet, or smartphone.

[0740] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs AI models.

[0741] 3. Emotion engine: An engine that analyzes the user's emotional state and processes data in conjunction with the generative AI model.

[0742] 4. Database: A storage for storing the user's current medical images, lifestyle data, emotional state, generated future medical images, and preventive measures based on them.

[0743] Program processing

[0744] The system operates through the following major processing steps:

[0745] 1. User registration and data acquisition:

[0746] User: Uploads their current medical images and lifestyle data, and inputs their emotional state (e.g., via a questionnaire or facial expression recognition).

[0747] Terminal: Converts the uploaded data into the appropriate format and sends it to the server.

[0748] 2. Data Processing and Future Medical Imaging:

[0749] Server: Analyzes the received data and performs image preprocessing (noise removal, normalization, etc.). It also analyzes lifestyle data and emotional state data, and organizes and stores them in a standardized format.

[0750] Server: Inputs the prepared data into the AI ​​model and emotion engine to generate future medical images.

[0751] 3. Visualizing and explaining the results:

[0752] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[0753] Terminal: Displays future medical images and their diagnostic results to the user in a visually easy-to-understand format.

[0754] 4. Suggested preventive measures:

[0755] Server: Generates individually customized preventive measures based on the user's data and analysis results. The emotion engine adjusts the way the preventive measures are presented based on the user's emotional state.

[0756] Device: Presents users with specific action plans and preventative measures, such as daily walking plans, links to brain training programs, and examples of dietary improvements.

[0757] 5. Emotional state monitoring and follow-up:

[0758] Server: The emotion engine monitors the user's emotional state in real time and sends the data to the server.

[0759] Server: Sends follow-up notifications at optimal times based on the user's emotional state.

[0760] On the device: Follow-up notifications are displayed to the user, prompting them to enter new medical images and lifestyle data.

[0761] Specific examples

[0762] Let us take a 50-year-old female user as an example.

[0763] 1. User: Uploads current brain images taken with an MRI machine, lifestyle data such as low exercise frequency, and recent stress levels to the device.

[0764] 2. Terminal: Sends data to the server.

[0765] 3. Server: Preprocesses MRI images and analyzes lifestyle and emotional state data. This data is input into an AI model to predict the brain state 10 years from now.

[0766] 4. Server: Sends the generated future MRI images to the terminal and stores the results in a database.

[0767] 5. Terminal: Future MRI images are displayed to the user, highlighting the tendency for hippocampal atrophy, while explaining that lack of exercise and high stress are risk factors.

[0768] 6. Server: Considers the user's emotional state and suggests exercise plans and stress management methods to promote relaxation.

[0769] 7. Terminal: Presents the user with specific exercise plans and brain training links for relaxation.

[0770] 8. Server: Send a follow-up notification after three months to ensure the user has implemented the suggested measures.

[0771] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[0772] This system allows users to visually understand future health risks and encourages them to take proactive preventative measures while taking into account their emotional state, which is expected to lead to effective behavioral changes to prevent dementia and Alzheimer's disease.

[0773] The processing flow will be explained below.

[0774] Step 1:

[0775] User: Logs in to a dedicated website or mobile application, takes and acquires current medical images (MRI images, etc.), enters lifestyle data (exercise frequency, diet, sleep time, stress level, etc.) and emotional state into an input form, and uploads the data to the device.

[0776] Step 2:

[0777] Terminal: Receives uploaded medical images, lifestyle data, and emotional state data, converts the data into a unified format, and sends it to the server.

[0778] Step 3:

[0779] Server: Preprocesses the received medical images, specifically removing noise from the images, performing necessary filtering and normalization, and preparing them in a format that is easy for the AI ​​model to use.

[0780] Step 4:

[0781] Server: Analyzes lifestyle data and emotional state data, organizes and stores them in a standardized format, and then inputs the medical images, lifestyle data, and emotional state data into the AI ​​model and emotion engine.

[0782] Step 5:

[0783] Server: Generates future medical images using an AI model. The AI ​​model uses deep learning algorithms to predict and simulate future brain states from current images and data.

[0784] Step 6:

[0785] Server: The emotion engine analyzes the emotional state data and monitors the user's emotional state in real time. This information is used to tailor the presentation of future medical images and preventative measures.

[0786] Step 7:

[0787] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[0788] Step 8:

[0789] Terminal: Displays the received future medical images and their diagnostic results to the user, highlighting and annotating specific areas to make them easier to understand visually.

[0790] Step 9:

[0791] Server: Generates preventive measures customized to the user's individual needs based on future medical images and diagnostic results. The emotion engine selects and adjusts the presentation of preventive measures based on the user's emotional state.

[0792] Step 10:

[0793] Device: Providing specific preventative measures to the user, such as exercise plans to promote relaxation, stress management techniques, links to brain training programs, and examples of dietary improvements.

[0794] Step 11:

[0795] Server: Schedules and sends periodic follow-up notifications to users, including reminders to re-upload new medical images and data when appropriate.

[0796] Step 12:

[0797] On the device: A follow-up notification is displayed to the user, prompting them to enter new medical images and lifestyle data as the next step, then returning to step 1 to repeat the process.

[0798] These steps result in a system that allows users to understand their future brain health status and implement individually customized preventative measures that take into account their emotional state.

[0799] Example 2

[0800] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0801] Previous systems only collected a user's medical images and lifestyle data separately, but were unable to take their emotional state into account. This made it difficult to accurately predict a user's future health risks and provide individually customized preventive measures. Furthermore, it was not possible to monitor a user's emotional state in real time and respond accordingly when proposing preventive measures or providing follow-up notifications to the user.

[0802] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0803] In this invention, the server includes: means for a user to input current medical images, lifestyle data, and emotional state data; means for transmitting the acquired current medical images, lifestyle data, and emotional state data to the server; means for preprocessing and integrating the data received by the server; means for inputting the preprocessed data into a generative AI model and an emotion engine; means for generating future medical images using the generative AI model and the emotion engine; means for visualizing the generated future medical images and providing them to a user terminal; means for presenting individually customized preventive measures based on the future medical images; means for monitoring the user's emotional state in real time and sending follow-up notifications; and means for the user who receives the follow-up notifications to input new data. This enables accurate prediction of future health risks taking into account the user's emotional state and the provision of individually customized preventive measures.

[0804] "User terminal" refers to a device that allows a user to input medical images, lifestyle data, and emotional state data and transmit them to the server. This includes PCs, tablets, smartphones, etc.

[0805] A "server" is a core device that receives data sent from user terminals and processes and analyzes the data.

[0806] "Medical images" are video data that show the internal state of a user's body, such as MRI images and CT scan images.

[0807] "Lifestyle data" refers to data related to the user's daily activities and habits, such as exercise time and dietary details.

[0808] "Emotional state data" refers to data relating to the user's current emotional and mood state, including stress levels and mood states.

[0809] "Preprocessing" refers to the process of converting received data (medical images, lifestyle data, emotional state data) into a format that is easier to analyze. Specifically, this includes noise removal and normalization.

[0810] A "generative AI model" is an artificial intelligence model that generates future medical images based on user data.

[0811] The "emotion engine" is an engine that analyzes the user's emotional state and reflects this in the analysis results of the generative AI model and the way in which preventive measures are presented.

[0812] "Future medical images" are images that show the user's future medical condition, predicted using a generative AI model.

[0813] "Visualization" refers to the visual display of data and analytical results using a computer or display.

[0814] "Preventive measures" are specific instructions or action plans provided to users to avoid future health risks.

[0815] A "follow-up notification" is a notification that confirms whether the user is taking the suggested preventative measures and encourages further action.

[0816] The present invention is a system that combines a user's current medical image, lifestyle data, and emotional state data to generate a future medical image and provide individually customized preventive measures, allowing the user to identify health risks in advance and take appropriate measures.

[0817] System Configuration

[0818] 1. User Device

[0819] User terminal: A device through which a user inputs medical images, lifestyle data, and emotional state data. Specifically, this applies to PCs, tablets, and smartphones.

[0820] Users upload current medical images taken with an MRI machine and enter lifestyle data such as daily exercise time, dietary habits, and stress levels. This data is then formatted and sent from the device to the server.

[0821] 2. Server

[0822] Server: This is the core device that receives, processes, and analyzes data sent from user devices. The server performs preprocessing such as noise reduction and normalization on the received medical image data, and analyzes lifestyle and emotional state data. The preprocessed data is then input into the generative AI model and emotion engine to generate future medical images.

[0823] Specifically, the server uses the following software:

[0824] Preprocess medical images using an image processing library (e.g., OpenCV).

[0825] Use a data analysis library (e.g., Pandas) to standardize and integrate lifestyle data and emotional state data.

[0826] Deep learning frameworks (e.g., TensorFlow, PyTorch) are used to run generative AI models and generate future medical images.

[0827] 3. Database and visualization system

[0828] Database: A storage for storing the user's current medical images, lifestyle data, emotional state data, generated future medical images, and preventive measures based on them.

[0829] Visualization system: This system visually displays the generated future medical images, analysis results, and preventive measures on the user's device. The data is converted into PNG or JPEG format and provided to the user with detailed annotations.

[0830] 4. Proposal of preventive measures

[0831] Based on the generated future medical images and analysis results, the server generates personalized preventative measures, including exercise plans, dietary suggestions, and stress management techniques, all of which reflect the user's emotional state.

[0832] On your device: View detailed prevention measures, including specific action plans and links.

[0833] 5. Follow-up notifications and emotional state monitoring

[0834] The server uses an emotion engine to monitor the user's emotional state in real time and send follow-up notifications at optimal times.

[0835] On your device: Receive follow-up notifications to prompt you to enter new medical images and lifestyle data.

[0836] Specific examples

[0837] Let us take a 50-year-old female user as an example.

[0838] 1. User: The user inputs into the terminal the current brain image taken with an MRI device, lifestyle data such as low exercise frequency and high stress level.

[0839] 2. Terminal: Formats the data and sends it to the server.

[0840] 3. Server: The received MRI images are denoised and normalised, and the lifestyle and emotional state data are analysed. Based on this data, a generative AI model is used to predict the brain state 10 years from now.

[0841] 4. Server: The generated MRI images from 10 years later are stored in a database and sent to the user's terminal.

[0842] 5. Terminal: Displays future MRI images showing hippocampal atrophy and explains that lack of exercise and high stress are risk factors.

[0843] 6. Server: Considers the user's emotional state and suggests exercise plans and stress management methods to promote relaxation.

[0844] 7. Device: Displays specific exercise plans and links to relaxation.

[0845] 8. Server: Send a follow-up notification after 3 months and ask the user to enter the data again.

[0846] Prompt Sentence Examples

[0847] "A 50-year-old female user has provided lifestyle data showing that she exercises infrequently and has a high stress level, and has uploaded current brain images taken with an MRI machine. Please predict her future brain state and suggest an exercise plan to promote relaxation and stress management methods. Also, please include a process for sending a follow-up notification three months later to check whether the suggested measures have been implemented."

[0848] As described above, the present invention realizes prediction of future health risks and provision of individually customized preventive measures while taking into account the user's emotional state.

[0849] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0850] Step 1: User registration and data acquisition

[0851] User: Enters their own medical images (e.g., MRI images), lifestyle data (e.g., exercise frequency and dietary content), and emotional state data (e.g., stress level) using a PC, tablet, or smartphone.

[0852] Input: Medical image data, lifestyle data, emotional state data.

[0853] Output: Formatted data.

[0854] Specific operation: The user launches the application on the device they are using, enters data in the specified format, and presses the send button.

[0855] Step 2: Sending data

[0856] Terminal: Checks the data entered by the user, converts it into the appropriate format, and sends it to the server.

[0857] Input: User-entered medical image data, lifestyle data, and emotional state data.

[0858] Output: The formatted data sent to the server.

[0859] Specific operation: The terminal uses a communication protocol to send input data to the server in JSON format, DICOM format, etc.

[0860] Step 3: Receiving and Preprocessing Data

[0861] Server: Preprocesses the received data. Medical image data is denoised and normalised, while lifestyle data and emotional state data are analysed and standardised.

[0862] Input: Submitted medical image data, lifestyle data, and emotional state data.

[0863] Output: The preprocessed dataset.

[0864] Specific operation: The server denoises medical images and analyzes and standardizes lifestyle data and emotional state data using Pandas.

[0865] Step 4: Integrate the data

[0866] Server: Integrates pre-processed medical image data, lifestyle data, and emotional state data into a single dataset.

[0867] Input: Preprocessed individual data.

[0868] Output: A consolidated dataset.

[0869] Specific operation: The server consolidates all preprocessed data and converts it into a format that can be input into the AI ​​model.

[0870] Step 5: Analysis by AI model and emotion engine

[0871] Server: Inputs the integrated dataset into the generative AI model and emotion engine to generate future medical images.

[0872] Input: Unified dataset.

[0873] Output: Futuristic medical images.

[0874] Specific operation: The AI ​​model is run on the server using TensorFlow or PyTorch, and analysis is performed in conjunction with the emotion engine. Predictions are made of the brain's state 10 years from now.

[0875] Step 6: Save and send the results

[0876] Server: Stores the generated future medical images in a database and sends the analysis results to the user's device.

[0877] Input: futuristic medical images.

[0878] Output: Analysis results sent to the user's device.

[0879] Specific operation: The results are converted into PNG or JPEG format and sent to be displayed in a user-friendly format.

[0880] Step 7: Visualize the results

[0881] Terminal: Visually displays future medical images and diagnostic results to the user.

[0882] Input: Analysis results sent from the server.

[0883] Output: The results that are displayed to the user.

[0884] Specific operation: The device displays the received images and data in a dashboard or slideshow format, informing the user of the latest results.

[0885] Step 8: Generate and propose preventative measures

[0886] Server: Generates personalized preventive measures based on the analysis results, and adjusts the recommendations based on the user's emotional state.

[0887] Input: Analysis results, emotion engine results.

[0888] Output: Customized precautions.

[0889] Specific actions: The server proposes specific exercise plans and stress management techniques based on the generated data.

[0890] Step 9: Displaying preventative measures

[0891] Device: Presents users with specific action plans and preventative measures (e.g., exercise plans, dietary suggestions, stress management links).

[0892] Input: Prevention data sent from the server.

[0893] Output: The precautions presented to the user.

[0894] Specific action: Display preventative measures on the device in the form of a pop-up notification or detailed PDF.

[0895] Step 10: Follow-up notifications and data updates

[0896] Server: Monitors emotional state in real time and sends follow-up notifications periodically.

[0897] Input: Real-time data from the emotion engine, new input data from users.

[0898] Output: Follow-up notification, updated dataset.

[0899] What it does: Sends reminders every three months to prompt users to enter new medical images and lifestyle data.

[0900] Through these steps, the system can predict the user's future health risks and provide appropriate preventative measures that take into account their emotional state.

[0901] (Application example 2)

[0902] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0903] In today's world, maintaining personal health requires a system that can centrally manage medical and lifestyle data and suggest appropriate preventive measures. However, conventional systems rely only on simple lifestyle and medical data without considering the user's emotional state, which is often overlooked, making it difficult to suggest and implement effective preventive measures. Furthermore, they lack specific action plans and support to help users implement the proposed preventive measures, and follow-up is insufficient.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0905] In this invention, the server includes: means for acquiring current medical images, lifestyle data, and emotional states; means for processing the acquired current medical images, lifestyle data, and emotional states and inputting them into a generative artificial intelligence model; means for generating future medical images using the generative artificial intelligence model; means for visualizing the generated future medical images and providing them to the user; means for presenting individually customized preventive measures based on the future medical images; means for periodically monitoring the user's emotional state and sending follow-up notifications at optimal times; and means for generating a meal plan based on the user's emotional state and providing the user with specific meal suggestions and delivery links. This makes it possible to support the proposal and implementation of effective, individually customized preventive measures based on lifestyle and medical images while taking the user's emotional state into consideration. Furthermore, providing specific meal suggestions and delivery links makes it easier for the user to take action immediately, thereby increasing the effectiveness of the preventive measures.

[0906] A "medical image" is an image of the internal structures of the human body taken with a medical device.

[0907] "Lifestyle data" refers to data relating to the user's lifestyle, such as exercise status, dietary content, and sleep patterns.

[0908] "Emotional state" is data that indicates the user's psychological state, and includes information such as stress level and happiness.

[0909] A "generative artificial intelligence model" is a machine learning model that predicts future states based on acquired data.

[0910] "Future medical images" are future medical images predicted based on current medical images, lifestyle data, and emotional state.

[0911] "Visualization" refers to visually displaying the generated data so that the user can easily understand it.

[0912] "Preventive measures" are specific action plans or advice to take to prevent future deterioration of a medical condition.

[0913] "Follow-up notifications" are notifications that track the implementation status of preventive measures and provide reminders or additional guidance to users at appropriate times.

[0914] A "meal plan" is a plan that suggests healthy meals based on the user's health and emotional state.

[0915] "Delivery Link" means an internet link that allows for easy ordering of the suggested meal.

[0916] The invention is a system that integrates and analyzes a user's medical images, lifestyle data, and emotional state, thereby enabling personalized preventative measures and dietary recommendations.

[0917] The system consists of the following main components:

[0918] 1. User Device:

[0919] Users use their smartphones to input their medical images, lifestyle data, and emotional state. This data is then sent to a server. For example, users can use a health app to record their daily exercise routine, take photos of their meals, and answer questionnaires about their emotional state.

[0920] 2. Server:

[0921] It analyzes the received data and generates future medical images using AI models and an emotion engine, while also monitoring the user's emotional state and sending follow-up notifications at optimal times.

[0922] 3. Emotion Engine:

[0923] This is an engine for analyzing the user's emotional state. It analyzes the user's emotions based on data such as photos, videos, and questionnaires, and sends the data to a server.

[0924] 4. Database:

[0925] It stores the user's current medical images, lifestyle data, and emotional state, as well as any future medical images and preventative measures that are generated.

[0926] 5. Meal Plan Generator:

[0927] A subsystem for suggesting appropriate meal plans to users based on their emotional state and providing delivery links.

[0928] Program processing explanation

[0929] 1. Hardware and software used:

[0930] User device: Smartphone (health app)

[0931] Server: Data analysis, collaboration with AI models and emotion engines, meal plan generation

[0932] Emotion Engine API: Analyzing the user's emotional state

[0933] Database: storing and managing user data

[0934] 2. Data processing and calculation:

[0935] The server receives data (medical images, lifestyle data, and emotional state) sent from the user's device and performs processes such as data cleansing, noise removal, and standardization. The data is then input into a generative AI model to generate future medical images. An emotion engine is also used to analyze the user's emotional state, and a meal plan is generated based on the results.

[0936] Specific examples

[0937] For example, a user can upload a photo of themselves using a smartphone app and answer a questionnaire about their recent stress level. This data is sent to a server, and the emotion engine analyzes it as a "high stress state." As a result, the server suggests a relaxing meal to the user, such as "chamomile tea, salmon, and avocado," and provides a delivery link for ordering it on the spot.

[0938] Prompt Sentence Examples

[0939] "Analyze the user's emotional state and suggest foods that will help them relax when they are under high stress."

[0940] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0941] Step 1:

[0942] Data entry and submission

[0943] Users use a smartphone app to input their own medical images, lifestyle data, and emotional state, including uploading MRI images, answering questionnaires about exercise frequency and diet, and recording their emotional state through facial recognition.

[0944] This data is converted into an appropriate format and sent to the server.

[0945] Input: medical images, lifestyle data, emotional state

[0946] Output: Data sent to the server

[0947] Step 2:

[0948] Data reception and preprocessing

[0949] The server receives data sent from the user terminal, and the received data is first preprocessed.

[0950] Specifically, this includes denoising and normalising medical images, standardising lifestyle data and analyzing emotional states.

[0951] Input: Data sent from the user's device

[0952] Output: Preprocessed data

[0953] Step 3:

[0954] AI models for generating future medical images

[0955] The server uses the pre-processed data to input the AI ​​model and generate future medical images.

[0956] For example, based on current MRI images and lifestyle data, the state of the brain 10 years from now can be predicted.

[0957] Input: Preprocessed medical images, lifestyle data, emotional states

[0958] Output: Medical images of the future

[0959] Step 4:

[0960] Emotional state analysis using emotion engine

[0961] The server analyzes the user's emotional state using an emotion engine, which determines the user's current emotional state based on questionnaire responses and facial expression recognition data.

[0962] Input: User's emotional state data

[0963] Output: Parsed emotional state

[0964] Step 5:

[0965] Visualizing and presenting results

[0966] The server transmits the generated future medical images and the analysis results of the emotional state to the user's device, which displays them in a visually easy-to-understand format.

[0967] For example, it can display future MRI images and highlight abnormalities, and also provide explanations based on the patient's emotional state.

[0968] Input: Future medical images, emotional state

[0969] Output: The results displayed on the user's terminal

[0970] Step 6:

[0971] Personalized prevention recommendations

[0972] Based on the generated future medical images and emotional state, the server suggests preventative measures, such as exercise plans, dietary modifications, and stress management techniques.

[0973] Specifically, if the user is found to be in a state of high stress, the system will suggest a meal plan that will have a relaxing effect.

[0974] Input: Future medical images, emotional state

[0975] Output: Individually customized preventative measures

[0976] Step 7:

[0977] Meal plans and delivery links provided

[0978] The server generates a meal plan based on the emotional state and provides the user with a delivery link.

[0979] For example, if a user is experiencing high stress, the app will suggest a relaxing meal such as "chamomile tea, salmon, and avocado" and provide a link to easily order it.

[0980] Input: Emotional state

[0981] Output: Meal plan, delivery link

[0982] Step 8:

[0983] Sending follow-up notifications

[0984] The server periodically monitors the user's emotional state and sends optimal follow-up notifications, including reminders to enter new medical images and lifestyle data.

[0985] Input: User's emotional state data

[0986] Output: Follow-up notification

[0987] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0988] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0989] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0990] [Third embodiment]

[0991] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0992] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0993] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0994] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0995] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0996] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0997] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0998] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0999] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1000] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1001] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1002] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1003] The present invention is a system that predicts the future state of the brain based on the user's current medical images and lifestyle data, and provides the results to the user to encourage preventative care and behavioral change. Specific embodiments of the system are described below.

[1004] System Configuration

[1005] 1. User terminal: A device that allows users to input current medical images and lifestyle data. This can be a PC, tablet, or smartphone.

[1006] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs AI models.

[1007] 3. Database: A storage for storing the user's current medical images, lifestyle data, generated future medical images, and preventive measures based on them.

[1008] Program processing

[1009] The system operates through the following major processing steps:

[1010] 1. User registration and data acquisition:

[1011] User: Uploads their current medical images and lifestyle data.

[1012] Terminal: Converts the uploaded data into the appropriate format and sends it to the server.

[1013] 2. Data Processing and Future Medical Imaging:

[1014] Server: Analyzes the received data and performs image preprocessing (such as noise removal and normalization).

[1015] Server: Inputs the prepared data into the AI ​​model to generate future medical images.

[1016] 3. Visualizing and explaining the results:

[1017] Server: Stores the generated future medical images in a database and simultaneously transmits the results to the user's terminal.

[1018] Terminal: Displays future medical images and their analysis results to the user in a visually easy-to-understand format.

[1019] 4. Suggested preventive measures:

[1020] Server: Generates individually customized preventive measures based on user data and analysis results.

[1021] Device: Presents preventive measures to users and encourages specific behavioral changes.

[1022] 5. Regular follow-up:

[1023] Server: Provides periodic follow-up notifications to users, prompting them to upload new medical images and data.

[1024] Terminal: Receives follow-up notifications and displays them to the user.

[1025] Specific examples

[1026] Let us take a 50-year-old male user as an example.

[1027] 1. User: Uploads current brain images taken with an MRI device and lifestyle data indicating low exercise frequency to the device.

[1028] 2. Terminal: Sends data to the server.

[1029] 3. Server: Preprocesses MRI images and analyzes lifestyle data. This data is input into an AI model to predict the state of the brain 10 years from now.

[1030] 4. Server: Sends the generated future MRI images to the terminal and stores the results in a database.

[1031] 5. Terminal: The device displays future MRI images to the user, highlighting the tendency for hippocampal atrophy and explaining that lack of exercise is a risk factor.

[1032] 6. Server: Suggests daily walking and brain training programs as preventive measures based on the user's needs.

[1033] 7. Terminal: Presents specific exercise plans and brain training links to the user.

[1034] 8. Server: Send a follow-up notification after three months to ensure the user has implemented the suggested measures.

[1035] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[1036] This system allows users to visually understand future health risks and encourages them to take proactive preventative measures, which is expected to lead to effective behavioral changes to prevent dementia and Alzheimer's disease.

[1037] The processing flow will be explained below.

[1038] Step 1:

[1039] User: Logs in to the dedicated website or mobile application, takes and acquires current medical images (MRI images, etc.), enters their lifestyle data (exercise activity, diet, sleep time, etc.) into an input form, and uploads it to the device.

[1040] Step 2:

[1041] Terminal: Receives medical images and lifestyle data uploaded by users, converts them into a specified format (e.g., a standard data format), and sends the data to the server.

[1042] Step 3:

[1043] Server: Preprocesses the received medical images, specifically removing noise from the images, performing necessary filtering and normalization, and preparing them in a format that is easy for the AI ​​model to use.

[1044] Step 4:

[1045] Server: Analyzes lifestyle data, organizes and stores it in a standardized format, and then inputs the medical images and lifestyle data into the AI ​​model.

[1046] Step 5:

[1047] Server: Generates future medical images using an AI model. The AI ​​model uses deep learning algorithms to predict and simulate future brain states based on current images and lifestyle data.

[1048] Step 6:

[1049] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[1050] Step 7:

[1051] Terminal: Displays the received future medical images and their diagnostic results to the user, highlighting and annotating specific areas to make them easier to understand visually.

[1052] Step 8:

[1053] Server: Based on future medical images and diagnostic results, it generates personalized preventative measures for users to improve their lifestyle, including exercise plans, dietary suggestions, and stress management techniques.

[1054] Step 9:

[1055] Device: Presents users with specific action plans and preventative measures, such as daily walking plans, links to brain training programs, and examples of dietary improvements.

[1056] Step 10:

[1057] Server: Schedules and sends periodic follow-up notifications to users, including reminders to re-upload medical images at the appropriate time.

[1058] Step 11:

[1059] Device: Display a follow-up notification to the user, prompting them to enter new medical images and lifestyle data, then return to step 2 and repeat the process.

[1060] The above steps result in a system that allows users to understand their own future brain health status and take effective preventative measures.

[1061] Example 1

[1062] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1063] Current medical systems lack the means to predict a user's future health status and provide appropriate preventive measures. As a result, users are not sufficiently motivated to recognize future health risks early and improve their lifestyle habits. Furthermore, there are no systems that suggest specific preventive measures based on future medical images and lifestyle data. This makes it difficult for users to take appropriate measures in response to changes in their health status.

[1064] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1065] In this invention, the server includes means for acquiring a user's current medical images and lifestyle data, means for preprocessing the acquired medical images and analyzing the lifestyle data, means for inputting the preprocessed data into a generative AI model, means for creating prompts and generating future medical images using the generative AI model, means for saving the generated future medical images in a database, means for visualizing the generated future medical images and providing them to the user, means for suggesting preventive measures based on the future medical images, and means for periodically following up with the user, thereby enabling the user to visually understand future health risks and take appropriate preventive measures early.

[1066] "User" refers to an individual who uses the system to provide their own medical images and lifestyle data and receive information about future health risks.

[1067] "Current medical images" are images that visually show the user's physical condition, and primarily refer to MRI images and CT scan images.

[1068] "Lifestyle data" is information about the user's daily activities and habits, including, for example, exercise frequency, dietary content, and sleep patterns.

[1069] "Preprocessing" refers to the processing of raw medical image data to make it analyzable, and refers to technical operations such as noise removal, image normalization, and feature extraction.

[1070] A "generative AI model" is a model that uses artificial intelligence to predict future states, and refers to a neural network trained by a learning algorithm.

[1071] A "prompt sentence" is a text-based input used to give specific instructions or queries to a generative AI model, including, for example, content that predicts future brain states.

[1072] "Future medical images" are images that visually show the user's future physical condition, predicted using a generative AI model.

[1073] "Database" refers to a data storage system that stores and efficiently manages all data used by a system.

[1074] "Visualization" refers to the presentation of data or information in a format that users can intuitively understand, and includes methods such as graphs, charts, and image displays.

[1075] "Preventive measures" are specific actions or measures that users should take to reduce future health risks, including exercise plans, dietary advice, stress management techniques, etc.

[1076] "Follow-up" is the process of tracking changes in a user's health condition and lifestyle habits at regular intervals and providing additional advice and guidance as needed.

[1077] This invention is a system that predicts the future state of the brain based on the user's current medical images and lifestyle data, and provides the results to the user to encourage preventative care and behavioral change. This system is implemented using the following hardware and software.

[1078] Hardware Configuration

[1079] 1. User terminal: A device that allows users to input current medical images and lifestyle data. Specifically, this includes PCs, tablets, smartphones, etc.

[1080] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. This is where the entire system is managed and the generative AI model is executed.

[1081] 3. Database: A storage for storing the user's current medical images, lifestyle data, generated future medical images, and preventive measures based on them.

[1082] Software configuration and processing content

[1083] 1. Data Collection:

[1084] User: Enters their current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency, dietary content, etc.).

[1085] Terminal: Converts uploaded data into a suitable format and sends it to the server. This can be done using Python or JavaScript libraries.

[1086] 2. Data Preprocessing:

[1087] Server: Preprocesses the received medical images. Preprocessing includes noise reduction, image normalization, feature extraction, etc. Specifically, OpenCV and SciPy are used.

[1088] 3. Data Analysis:

[1089] Server: Also analyzes lifestyle data. This analysis uses libraries such as Scikit-learn.

[1090] 4. Future Predictions:

[1091] Server: The preprocessed data is input into a generative AI model. This model uses PyTorch or TensorFlow. A prompt such as "Please predict the state of the brain in 10 years" is input.

[1092] 5. Save and display results:

[1093] Server: The generated future medical images are stored in a database and simultaneously sent to the user's terminal. MySQL is used as the database system.

[1094] Terminal: Visually display the results sent. Use a library such as Matplotlib to generate graphs and charts.

[1095] 6. Suggested preventive measures:

[1096] Server: Generates individually customized preventive measures based on the generated future medical images and analysis results. Utilizing appropriate algorithms, the preventive measures are derived and provided to the user.

[1097] 7. Follow-up:

[1098] Server: Sends periodic follow-up notifications to prompt users to re-enter new medical images and lifestyle data. This is scheduled using a Cron job.

[1099] Terminal: Receives follow-up notifications and prompts the user to enter new data.

[1100] Specific examples

[1101] Specific processing will be explained using the case of a 50-year-old male user as an example.

[1102] 1. User: Uploads current brain images taken with an MRI device and lifestyle data indicating low exercise frequency to the device.

[1103] 2. Terminal: Sends data to the server.

[1104] 3. Server: Preprocesses MRI images and analyzes lifestyle data. This data is input into an AI model to predict the state of the brain 10 years from now.

[1105] 4. Server: Generates future MRI images, sends them to the device, and also stores them in the database.

[1106] 5. Terminal: Future MRI images are displayed, highlighting the tendency for hippocampal atrophy in particular. It is also explained that lack of exercise is a risk factor.

[1107] 6. Server: For example, suggest daily walking and a brain training program as preventative measures.

[1108] 7. Terminal: Presents specific exercise plans and brain training links to the user.

[1109] 8. Server: Send a follow-up notice after three months to confirm the continuation of the proposed measures.

[1110] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[1111] Example prompt: "A 50-year-old male user has provided us with MRI brain images and lifestyle data showing that he exercises infrequently. Predict his brain condition 10 years from now and suggest appropriate preventative measures."

[1112] This system allows users to visually understand future health risks and take appropriate preventive measures early on.

[1113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1114] Step 1:

[1115] User registration and data acquisition

[1116] Users input their current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency, dietary content, etc.) through a dedicated application or web interface.

[1117] Input: Medical image files, lifestyle data

[1118] The terminal receives the data uploaded by the user, standardizes the format, and compresses the data as necessary. Specifically, it converts JPEG images into the standard DICOM format.

[1119] Output: Processed medical image files, lifestyle data

[1120] The terminal transmits the processed data to the server.

[1121] Step 2:

[1122] Data Preprocessing

[1123] The server preprocesses the medical images received from the device, including noise removal, image normalization, and feature extraction, using OpenCV and SciPy libraries.

[1124] Input: Processed medical image files

[1125] Data processing: noise removal, image normalization, feature extraction

[1126] Output: Preprocessed medical image data

[1127] Step 3:

[1128] Lifestyle data analysis

[1129] The server analyzes the lifestyle data. Here, detailed analysis of lifestyle habits is performed and specific parameters are extracted. Specifically, the "Scikit-learn" library is used.

[1130] Input: Lifestyle data

[1131] Data processing: Data normalization, feature extraction

[1132] Output: Analyzed lifestyle data

[1133] Step 4:

[1134] The Future of Medical Imaging

[1135] The server inputs the preprocessed medical image data and analyzed lifestyle data into a generative AI model, which uses PyTorch and TensorFlow.

[1136] Input: Preprocessed medical image data, analyzed lifestyle data

[1137] Prompt: For example, "Predict your brain state in 10 years."

[1138] Data Computation: Predictive Processing with AI Models

[1139] Output: Generated future medical images

[1140] Step 5:

[1141] Saving and displaying results

[1142] The server stores the generated future medical images using a database management system such as MySQL.

[1143] Input: Generated future medical images

[1144] Data processing: Saving to the database

[1145] Output: Future medical image data stored in a database

[1146] The server transmits the generated future medical images to the user terminal.

[1147] The terminal uses the Matplotlib library to generate graphs and charts to visually display the results.

[1148] Input: Generated future medical images

[1149] Output: Visualized medical images, graphs, charts

[1150] Step 6:

[1151] Preventive measures suggested

[1152] The server generates personalized preventative measures based on the generated future medical images and analysis results, using advanced algorithms and behavioral science theories, specifically Scikit-learn.

[1153] Input: Analysis results, future medical images

[1154] Data Computing: Proactive Measures

[1155] Output: Individually customized preventative measures

[1156] The device will provide the user with specific exercise plans, dietary advice, stress management methods, and more.

[1157] Step 7:

[1158] Regular follow-up

[1159] The server will send follow-up notifications periodically (e.g., every 3 months), including automatic scheduling using a "Cron" job.

[1160] Input: Schedule information

[1161] Data Calculation: Generate Follow-Up Notifications

[1162] Output: Follow-up notification

[1163] The device receives a follow-up notification and prompts the user to enter new data. Once the new data is uploaded, the same process begins again, starting with data collection.

[1164] (Application example 1)

[1165] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1166] In today's world, many people face the challenge of fully understanding future health risks and taking appropriate preventative measures. Furthermore, there is a lack of methods for providing users with a visually easy-to-understand view of predicted health risks. The goal of this project is to improve this situation and promote preventative awareness and behavioral change in users.

[1167] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1168] In this invention, the server includes means for acquiring current medical images and habit data, means for processing the acquired current medical images and habit data and inputting them into a generative artificial intelligence model, means for generating future medical images using the generative artificial intelligence model, means for visualizing the generated future medical images and providing them to the user through a display device, means for presenting individually customized preventive measures and action plans based on the future medical images, means for predicting future health risks and providing health management services through a virtual shop, and means for periodically sending tracking notifications to the user and prompting them to upload new data, thereby enabling the user to visually understand future health risks and take customized preventive measures based on their own lifestyle.

[1169] A "medical image" is an image of the human body taken with medical equipment and used for diagnosis or treatment.

[1170] "Habit data" is information about individual lifestyle habits, including data on diet, exercise, sleep, etc.

[1171] "Generative artificial intelligence models" are algorithms and machine learning models used to predict future states based on existing data.

[1172] A "virtual store" is a virtual store that offers goods and services via the Internet.

[1173] A "display device" is a device for visually presenting images and information to a user, and includes smartphones and head-mounted displays.

[1174] "Preventive measures" are specific actions or efforts to maintain health and prevent disease.

[1175] An "action plan" is a specific set of actions or a timetable for implementing preventative measures.

[1176] "Tracking notifications" are periodic reminders or notifications sent to users to inform them of their progress in taking preventative measures or to provide new data.

[1177] A "health management service" is a service that monitors a user's health status and provides appropriate advice and preventative measures.

[1178] This invention is a system that predicts the user's future brain state based on current medical images and habit data, and provides the results to the user to encourage preventative care and behavioral change. This system is mainly composed of a server, a terminal, and a user. A specific embodiment is described below.

[1179] System Configuration

[1180] 1. User terminal: A device that allows users to input current medical images and habit data. Examples include smartphones and head-mounted displays.

[1181] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs the generative AI model.

[1182] 3. Database: A storage for storing the user's current medical images, habit data, generated future medical images, and preventive measures based on them.

[1183] System Operation

[1184] Data Acquisition

[1185] Users upload their own medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency and dietary habits) to the device, which then converts the data into an appropriate format and sends it to the server.

[1186] Data Processing and Future Medical Imaging

[1187] The server analyzes the received medical images and habit data. First, the medical images undergo preprocessing, such as noise removal and normalization. The preprocessed data is input into a generative AI model, which generates future medical images. For example, based on current brain MRI images and data on lack of exercise, the AI ​​can predict the state of the brain 10 years from now.

[1188] Visualizing the results

[1189] The generated future medical images are stored in a database by the server and sent to the user's device, which displays them to the user in a visually easy-to-understand format. When a head-mounted display is used, the future brain state can be displayed realistically in 3D or AR.

[1190] Precautions

[1191] The server generates personalized preventive measures and action plans based on future medical images and current data. For example, if lack of exercise is a risk factor, daily walking and brain training programs will be suggested. These preventive measures will be provided through a virtual store, allowing users to receive a variety of health management services.

[1192] Regular follow-up

[1193] The server periodically sends tracking notifications to the user, prompting them to upload new data. The user then uploads medical images and lifestyle data again, and the system analyzes the new data, enabling continuous health management.

[1194] Specific examples

[1195] A 50-year-old male user uploads his current MRI scan and his low exercise habits to his device. The data is sent to a server, where a generative AI model predicts his brain state 10 years from now. Future MRI images are displayed in 3D via a head-mounted display, allowing the user to visually confirm his health risks. The server also suggests daily walking and brain training programs as preventative measures, which are offered through a virtual shopping mall.

[1196] Prompt Sentence Examples

[1197] "Enter your current MRI images and lifestyle data. We'll predict your future brain condition and visualize your health risks."

[1198] In this way, users can visually understand future health risks and take customized preventative measures based on their lifestyle.

[1199] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1200] Step 1:

[1201] The user uploads current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency) to the device via a smartphone or head-mounted display. The input data are medical image files and text data related to lifestyle habits. The device converts this data into an appropriate format and sends it to the server. The output is the data transfer to the server.

[1202] Step 2:

[1203] The server analyzes the received data. First, it preprocesses the medical image data. Preprocessing such as noise removal and normalization is performed to enable the AI ​​model to accurately analyze the data. The input is medical images and lifestyle data, and the output is the preprocessed data. Specifically, OpenCV is used to remove noise from the images and perform standardization processing.

[1204] Step 3:

[1205] The preprocessed data is input into a generative AI model to generate future medical images. Specifically, TensorFlow is used to input data into the AI ​​model and perform prediction processing. The input is the preprocessed data, and the output is the generated future medical images. The generative AI model predicts future brain health based on current medical images and lifestyle data.

[1206] Step 4:

[1207] The server stores the generated future medical images in a database and transmits them to the terminal. The input is the generated future medical image, and the output is storage in the database and transmission to the terminal. Specifically, the generated image is stored in an appropriate format and transmitted to the terminal via the Internet.

[1208] Step 5:

[1209] The device visually presents the received future medical images to the user. The input is the received future medical image, and the output is visual information provided to the user. Specifically, 3D and AR displays are performed using the display functions of a smartphone or head-mounted display.

[1210] Step 6:

[1211] The server generates personalized preventive measures and action plans based on future medical images and current data. The input is future medical images and current lifestyle data, and the output is personalized preventive measures. Specifically, it uses an algorithm that evaluates the user's risk factors and suggests optimal measures to address them.

[1212] Step 7:

[1213] The server provides users with preventive measures and action plans through a virtual shop. The input is individually customized preventive measures, and the output is services provided through the virtual shop. Specifically, users can view and purchase the suggested preventive measures through the virtual shop interface.

[1214] Step 8:

[1215] The server periodically sends tracking notifications to users to prompt them to upload new data. The input is the configured follow-up schedule, and the output is the notification message. Specifically, it uses a reminder function to send emails or alerts to prompt users to update their data.

[1216] In this way, a system is created that allows users to visually understand their health risks and implement customized preventative measures.

[1217] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1218] The present invention is a system that combines a user's current medical images and lifestyle data with an emotion engine that recognizes the user's emotional state. This makes it possible to present individually customized preventive measures that take the user's emotional state into consideration and send optimal follow-up notifications. Specific embodiments of the system are described below.

[1219] System Configuration

[1220] 1. User device: A device that allows users to input current medical images, lifestyle data, and emotional state. This can be a PC, tablet, or smartphone.

[1221] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs AI models.

[1222] 3. Emotion engine: An engine that analyzes the user's emotional state and processes data in conjunction with the generative AI model.

[1223] 4. Database: A storage for storing the user's current medical images, lifestyle data, emotional state, generated future medical images, and preventive measures based on them.

[1224] Program processing

[1225] The system operates through the following major processing steps:

[1226] 1. User registration and data acquisition:

[1227] User: Uploads their current medical images and lifestyle data, and inputs their emotional state (e.g., via a questionnaire or facial expression recognition).

[1228] Terminal: Converts the uploaded data into the appropriate format and sends it to the server.

[1229] 2. Data Processing and Future Medical Imaging:

[1230] Server: Analyzes the received data and performs image preprocessing (noise removal, normalization, etc.). It also analyzes lifestyle data and emotional state data, and organizes and stores them in a standardized format.

[1231] Server: Inputs the prepared data into the AI ​​model and emotion engine to generate future medical images.

[1232] 3. Visualizing and explaining the results:

[1233] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[1234] Terminal: Displays future medical images and their diagnostic results to the user in a visually easy-to-understand format.

[1235] 4. Suggested preventive measures:

[1236] Server: Generates individually customized preventive measures based on the user's data and analysis results. The emotion engine adjusts the way the preventive measures are presented based on the user's emotional state.

[1237] Device: Presents users with specific action plans and preventative measures, such as daily walking plans, links to brain training programs, and examples of dietary improvements.

[1238] 5. Emotional state monitoring and follow-up:

[1239] Server: The emotion engine monitors the user's emotional state in real time and sends the data to the server.

[1240] Server: Sends follow-up notifications at optimal times based on the user's emotional state.

[1241] On the device: Follow-up notifications are displayed to the user, prompting them to enter new medical images and lifestyle data.

[1242] Specific examples

[1243] Let us take a 50-year-old female user as an example.

[1244] 1. User: Uploads current brain images taken with an MRI machine, lifestyle data such as low exercise frequency, and recent stress levels to the device.

[1245] 2. Terminal: Sends data to the server.

[1246] 3. Server: Preprocesses MRI images and analyzes lifestyle and emotional state data. This data is input into an AI model to predict the brain state 10 years from now.

[1247] 4. Server: Sends the generated future MRI images to the terminal and stores the results in a database.

[1248] 5. Terminal: Future MRI images are displayed to the user, highlighting the tendency for hippocampal atrophy, while explaining that lack of exercise and high stress are risk factors.

[1249] 6. Server: Considers the user's emotional state and suggests exercise plans and stress management methods to promote relaxation.

[1250] 7. Terminal: Presents the user with specific exercise plans and brain training links for relaxation.

[1251] 8. Server: Send a follow-up notification after three months to ensure the user has implemented the suggested measures.

[1252] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[1253] This system allows users to visually understand future health risks and encourages them to take proactive preventative measures while taking into account their emotional state, which is expected to lead to effective behavioral changes to prevent dementia and Alzheimer's disease.

[1254] The processing flow will be explained below.

[1255] Step 1:

[1256] User: Logs in to a dedicated website or mobile application, takes and acquires current medical images (MRI images, etc.), enters lifestyle data (exercise frequency, diet, sleep time, stress level, etc.) and emotional state into an input form, and uploads the data to the device.

[1257] Step 2:

[1258] Terminal: Receives uploaded medical images, lifestyle data, and emotional state data, converts the data into a unified format, and sends it to the server.

[1259] Step 3:

[1260] Server: Preprocesses the received medical images, specifically removing noise from the images, performing necessary filtering and normalization, and preparing them in a format that is easy for the AI ​​model to use.

[1261] Step 4:

[1262] Server: Analyzes lifestyle data and emotional state data, organizes and stores them in a standardized format, and then inputs the medical images, lifestyle data, and emotional state data into the AI ​​model and emotion engine.

[1263] Step 5:

[1264] Server: Generates future medical images using an AI model. The AI ​​model uses deep learning algorithms to predict and simulate future brain states from current images and data.

[1265] Step 6:

[1266] Server: The emotion engine analyzes the emotional state data and monitors the user's emotional state in real time. This information is used to tailor the presentation of future medical images and preventative measures.

[1267] Step 7:

[1268] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[1269] Step 8:

[1270] Terminal: Displays the received future medical images and their diagnostic results to the user, highlighting and annotating specific areas to make them easier to understand visually.

[1271] Step 9:

[1272] Server: Generates preventive measures customized to the user's individual needs based on future medical images and diagnostic results. The emotion engine selects and adjusts the presentation of preventive measures based on the user's emotional state.

[1273] Step 10:

[1274] Device: Providing specific preventative measures to the user, such as exercise plans to promote relaxation, stress management techniques, links to brain training programs, and examples of dietary improvements.

[1275] Step 11:

[1276] Server: Schedules and sends periodic follow-up notifications to users, including reminders to re-upload new medical images and data when appropriate.

[1277] Step 12:

[1278] On the device: A follow-up notification is displayed to the user, prompting them to enter new medical images and lifestyle data as the next step, then returning to step 1 to repeat the process.

[1279] These steps result in a system that allows users to understand their future brain health status and implement individually customized preventative measures that take into account their emotional state.

[1280] Example 2

[1281] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1282] Previous systems only collected a user's medical images and lifestyle data separately, but were unable to take their emotional state into account. This made it difficult to accurately predict a user's future health risks and provide individually customized preventive measures. Furthermore, it was not possible to monitor a user's emotional state in real time and respond accordingly when proposing preventive measures or providing follow-up notifications to the user.

[1283] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1284] In this invention, the server includes: means for a user to input current medical images, lifestyle data, and emotional state data; means for transmitting the acquired current medical images, lifestyle data, and emotional state data to the server; means for preprocessing and integrating the data received by the server; means for inputting the preprocessed data into a generative AI model and an emotion engine; means for generating future medical images using the generative AI model and the emotion engine; means for visualizing the generated future medical images and providing them to a user terminal; means for presenting individually customized preventive measures based on the future medical images; means for monitoring the user's emotional state in real time and sending follow-up notifications; and means for the user who receives the follow-up notifications to input new data. This enables accurate prediction of future health risks taking into account the user's emotional state and the provision of individually customized preventive measures.

[1285] "User terminal" refers to a device that allows a user to input medical images, lifestyle data, and emotional state data and transmit them to the server. This includes PCs, tablets, smartphones, etc.

[1286] A "server" is a core device that receives data sent from user terminals and processes and analyzes the data.

[1287] "Medical images" are video data that show the internal state of a user's body, such as MRI images and CT scan images.

[1288] "Lifestyle data" refers to data related to the user's daily activities and habits, such as exercise time and dietary details.

[1289] "Emotional state data" refers to data relating to the user's current emotional and mood state, including stress levels and mood states.

[1290] "Preprocessing" refers to the process of converting received data (medical images, lifestyle data, emotional state data) into a format that is easier to analyze. Specifically, this includes noise removal and normalization.

[1291] A "generative AI model" is an artificial intelligence model that generates future medical images based on user data.

[1292] The "emotion engine" is an engine that analyzes the user's emotional state and reflects this in the analysis results of the generative AI model and the way in which preventive measures are presented.

[1293] "Future medical images" are images that show the user's future medical condition, predicted using a generative AI model.

[1294] "Visualization" refers to the visual display of data and analytical results using a computer or display.

[1295] "Preventive measures" are specific instructions or action plans provided to users to avoid future health risks.

[1296] A "follow-up notification" is a notification that confirms whether the user is taking the suggested preventative measures and encourages further action.

[1297] The present invention is a system that combines a user's current medical image, lifestyle data, and emotional state data to generate a future medical image and provide individually customized preventive measures, allowing the user to identify health risks in advance and take appropriate measures.

[1298] System Configuration

[1299] 1. User Device

[1300] User terminal: A device through which a user inputs medical images, lifestyle data, and emotional state data. Specifically, this applies to PCs, tablets, and smartphones.

[1301] Users upload current medical images taken with an MRI machine and enter lifestyle data such as daily exercise time, dietary habits, and stress levels. This data is then formatted and sent from the device to the server.

[1302] 2. Server

[1303] Server: This is the core device that receives, processes, and analyzes data sent from user devices. The server performs preprocessing such as noise reduction and normalization on the received medical image data, and analyzes lifestyle and emotional state data. The preprocessed data is then input into the generative AI model and emotion engine to generate future medical images.

[1304] Specifically, the server uses the following software:

[1305] Preprocess medical images using an image processing library (e.g., OpenCV).

[1306] Use a data analysis library (e.g., Pandas) to standardize and integrate lifestyle data and emotional state data.

[1307] Deep learning frameworks (e.g., TensorFlow, PyTorch) are used to run generative AI models and generate future medical images.

[1308] 3. Database and visualization system

[1309] Database: A storage for storing the user's current medical images, lifestyle data, emotional state data, generated future medical images, and preventive measures based on them.

[1310] Visualization system: This system visually displays the generated future medical images, analysis results, and preventive measures on the user's device. The data is converted into PNG or JPEG format and provided to the user with detailed annotations.

[1311] 4. Proposal of preventive measures

[1312] Based on the generated future medical images and analysis results, the server generates personalized preventative measures, including exercise plans, dietary suggestions, and stress management techniques, all of which reflect the user's emotional state.

[1313] On your device: View detailed prevention measures, including specific action plans and links.

[1314] 5. Follow-up notifications and emotional state monitoring

[1315] The server uses an emotion engine to monitor the user's emotional state in real time and send follow-up notifications at optimal times.

[1316] On your device: Receive follow-up notifications to prompt you to enter new medical images and lifestyle data.

[1317] Specific examples

[1318] Let us take a 50-year-old female user as an example.

[1319] 1. User: The user inputs into the terminal the current brain image taken with an MRI device, lifestyle data such as low exercise frequency and high stress level.

[1320] 2. Terminal: Formats the data and sends it to the server.

[1321] 3. Server: The received MRI images are denoised and normalised, and the lifestyle and emotional state data are analysed. Based on this data, a generative AI model is used to predict the brain state 10 years from now.

[1322] 4. Server: The generated MRI images from 10 years later are stored in a database and sent to the user's terminal.

[1323] 5. Terminal: Displays future MRI images showing hippocampal atrophy and explains that lack of exercise and high stress are risk factors.

[1324] 6. Server: Considers the user's emotional state and suggests exercise plans and stress management methods to promote relaxation.

[1325] 7. Device: Displays specific exercise plans and links to relaxation.

[1326] 8. Server: Send a follow-up notification after 3 months and ask the user to enter the data again.

[1327] Prompt Sentence Examples

[1328] "A 50-year-old female user has provided lifestyle data showing that she exercises infrequently and has a high stress level, and has uploaded current brain images taken with an MRI machine. Please predict her future brain state and suggest an exercise plan to promote relaxation and stress management methods. Also, please include a process for sending a follow-up notification three months later to check whether the suggested measures have been implemented."

[1329] As described above, the present invention realizes prediction of future health risks and provision of individually customized preventive measures while taking into account the user's emotional state.

[1330] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1331] Step 1: User registration and data acquisition

[1332] User: Enters their own medical images (e.g., MRI images), lifestyle data (e.g., exercise frequency and dietary content), and emotional state data (e.g., stress level) using a PC, tablet, or smartphone.

[1333] Input: Medical image data, lifestyle data, emotional state data.

[1334] Output: Formatted data.

[1335] Specific operation: The user launches the application on the device they are using, enters data in the specified format, and presses the send button.

[1336] Step 2: Sending data

[1337] Terminal: Checks the data entered by the user, converts it into the appropriate format, and sends it to the server.

[1338] Input: User-entered medical image data, lifestyle data, and emotional state data.

[1339] Output: The formatted data sent to the server.

[1340] Specific operation: The terminal uses a communication protocol to send input data to the server in JSON format, DICOM format, etc.

[1341] Step 3: Receiving and Preprocessing Data

[1342] Server: Preprocesses the received data. Medical image data is denoised and normalised, while lifestyle data and emotional state data are analysed and standardised.

[1343] Input: Submitted medical image data, lifestyle data, and emotional state data.

[1344] Output: The preprocessed dataset.

[1345] Specific operation: The server denoises medical images and analyzes and standardizes lifestyle data and emotional state data using Pandas.

[1346] Step 4: Integrate the data

[1347] Server: Integrates pre-processed medical image data, lifestyle data, and emotional state data into a single dataset.

[1348] Input: Preprocessed individual data.

[1349] Output: A consolidated dataset.

[1350] Specific operation: The server consolidates all preprocessed data and converts it into a format that can be input into the AI ​​model.

[1351] Step 5: Analysis by AI model and emotion engine

[1352] Server: Inputs the integrated dataset into the generative AI model and emotion engine to generate future medical images.

[1353] Input: Unified dataset.

[1354] Output: Futuristic medical images.

[1355] Specific operation: The AI ​​model is run on the server using TensorFlow or PyTorch, and analysis is performed in conjunction with the emotion engine. Predictions are made of the brain's state 10 years from now.

[1356] Step 6: Save and send the results

[1357] Server: Stores the generated future medical images in a database and sends the analysis results to the user's device.

[1358] Input: futuristic medical images.

[1359] Output: Analysis results sent to the user's device.

[1360] Specific operation: The results are converted into PNG or JPEG format and sent to be displayed in a user-friendly format.

[1361] Step 7: Visualize the results

[1362] Terminal: Visually displays future medical images and diagnostic results to the user.

[1363] Input: Analysis results sent from the server.

[1364] Output: The results that are displayed to the user.

[1365] Specific operation: The device displays the received images and data in a dashboard or slideshow format, informing the user of the latest results.

[1366] Step 8: Generate and propose preventative measures

[1367] Server: Generates personalized preventive measures based on the analysis results, and adjusts the recommendations based on the user's emotional state.

[1368] Input: Analysis results, emotion engine results.

[1369] Output: Customized precautions.

[1370] Specific actions: The server proposes specific exercise plans and stress management techniques based on the generated data.

[1371] Step 9: Displaying preventative measures

[1372] Device: Presents users with specific action plans and preventative measures (e.g., exercise plans, dietary suggestions, stress management links).

[1373] Input: Prevention data sent from the server.

[1374] Output: The precautions presented to the user.

[1375] Specific action: Display preventative measures on the device in the form of a pop-up notification or detailed PDF.

[1376] Step 10: Follow-up notifications and data updates

[1377] Server: Monitors emotional state in real time and sends follow-up notifications periodically.

[1378] Input: Real-time data from the emotion engine, new input data from users.

[1379] Output: Follow-up notification, updated dataset.

[1380] What it does: Sends reminders every three months to prompt users to enter new medical images and lifestyle data.

[1381] Through these steps, the system can predict the user's future health risks and provide appropriate preventative measures that take into account their emotional state.

[1382] (Application example 2)

[1383] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1384] In today's world, maintaining personal health requires a system that can centrally manage medical and lifestyle data and suggest appropriate preventive measures. However, conventional systems rely only on simple lifestyle and medical data without considering the user's emotional state, which is often overlooked, making it difficult to suggest and implement effective preventive measures. Furthermore, they lack specific action plans and support to help users implement the proposed preventive measures, and follow-up is insufficient.

[1385] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1386] In this invention, the server includes: means for acquiring current medical images, lifestyle data, and emotional states; means for processing the acquired current medical images, lifestyle data, and emotional states and inputting them into a generative artificial intelligence model; means for generating future medical images using the generative artificial intelligence model; means for visualizing the generated future medical images and providing them to the user; means for presenting individually customized preventive measures based on the future medical images; means for periodically monitoring the user's emotional state and sending follow-up notifications at optimal times; and means for generating a meal plan based on the user's emotional state and providing the user with specific meal suggestions and delivery links. This makes it possible to support the proposal and implementation of effective, individually customized preventive measures based on lifestyle and medical images while taking the user's emotional state into consideration. Furthermore, providing specific meal suggestions and delivery links makes it easier for the user to take action immediately, thereby increasing the effectiveness of the preventive measures.

[1387] A "medical image" is an image of the internal structures of the human body taken with a medical device.

[1388] "Lifestyle data" refers to data relating to the user's lifestyle, such as exercise status, dietary content, and sleep patterns.

[1389] "Emotional state" is data that indicates the user's psychological state, and includes information such as stress level and happiness.

[1390] A "generative artificial intelligence model" is a machine learning model that predicts future states based on acquired data.

[1391] "Future medical images" are future medical images predicted based on current medical images, lifestyle data, and emotional state.

[1392] "Visualization" refers to visually displaying the generated data so that the user can easily understand it.

[1393] "Preventive measures" are specific action plans or advice to take to prevent future deterioration of a medical condition.

[1394] "Follow-up notifications" are notifications that track the implementation status of preventive measures and provide reminders or additional guidance to users at appropriate times.

[1395] A "meal plan" is a plan that suggests healthy meals based on the user's health and emotional state.

[1396] "Delivery Link" means an internet link that allows for easy ordering of the suggested meal.

[1397] The invention is a system that integrates and analyzes a user's medical images, lifestyle data, and emotional state, thereby enabling personalized preventative measures and dietary recommendations.

[1398] The system consists of the following main components:

[1399] 1. User Device:

[1400] Users use their smartphones to input their medical images, lifestyle data, and emotional state. This data is then sent to a server. For example, users can use a health app to record their daily exercise routine, take photos of their meals, and answer questionnaires about their emotional state.

[1401] 2. Server:

[1402] It analyzes the received data and generates future medical images using AI models and an emotion engine, while also monitoring the user's emotional state and sending follow-up notifications at optimal times.

[1403] 3. Emotion Engine:

[1404] This is an engine for analyzing the user's emotional state. It analyzes the user's emotions based on data such as photos, videos, and questionnaires, and sends the data to a server.

[1405] 4. Database:

[1406] It stores the user's current medical images, lifestyle data, and emotional state, as well as any future medical images and preventative measures that are generated.

[1407] 5. Meal Plan Generator:

[1408] A subsystem for suggesting appropriate meal plans to users based on their emotional state and providing delivery links.

[1409] Program processing explanation

[1410] 1. Hardware and software used:

[1411] User device: Smartphone (health app)

[1412] Server: Data analysis, collaboration with AI models and emotion engines, meal plan generation

[1413] Emotion Engine API: Analyzing the user's emotional state

[1414] Database: storing and managing user data

[1415] 2. Data processing and calculation:

[1416] The server receives data (medical images, lifestyle data, and emotional state) sent from the user's device and performs processes such as data cleansing, noise removal, and standardization. The data is then input into a generative AI model to generate future medical images. An emotion engine is also used to analyze the user's emotional state, and a meal plan is generated based on the results.

[1417] Specific examples

[1418] For example, a user can upload a photo of themselves using a smartphone app and answer a questionnaire about their recent stress level. This data is sent to a server, and the emotion engine analyzes it as a "high stress state." As a result, the server suggests a relaxing meal to the user, such as "chamomile tea, salmon, and avocado," and provides a delivery link for ordering it on the spot.

[1419] Prompt Sentence Examples

[1420] "Analyze the user's emotional state and suggest foods that will help them relax when they are under high stress."

[1421] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1422] Step 1:

[1423] Data entry and submission

[1424] Users use a smartphone app to input their own medical images, lifestyle data, and emotional state, including uploading MRI images, answering questionnaires about exercise frequency and diet, and recording their emotional state through facial recognition.

[1425] This data is converted into an appropriate format and sent to the server.

[1426] Input: medical images, lifestyle data, emotional state

[1427] Output: Data sent to the server

[1428] Step 2:

[1429] Data reception and preprocessing

[1430] The server receives data sent from the user terminal, and the received data is first preprocessed.

[1431] Specifically, this includes denoising and normalising medical images, standardising lifestyle data and analyzing emotional states.

[1432] Input: Data sent from the user's device

[1433] Output: Preprocessed data

[1434] Step 3:

[1435] AI models for generating future medical images

[1436] The server uses the pre-processed data to input the AI ​​model and generate future medical images.

[1437] For example, based on current MRI images and lifestyle data, the state of the brain 10 years from now can be predicted.

[1438] Input: Preprocessed medical images, lifestyle data, emotional states

[1439] Output: Medical images of the future

[1440] Step 4:

[1441] Emotional state analysis using emotion engine

[1442] The server analyzes the user's emotional state using an emotion engine, which determines the user's current emotional state based on questionnaire responses and facial expression recognition data.

[1443] Input: User's emotional state data

[1444] Output: Parsed emotional state

[1445] Step 5:

[1446] Visualizing and presenting results

[1447] The server transmits the generated future medical images and the analysis results of the emotional state to the user's device, which displays them in a visually easy-to-understand format.

[1448] For example, it can display future MRI images and highlight abnormalities, and also provide explanations based on the patient's emotional state.

[1449] Input: Future medical images, emotional state

[1450] Output: The results displayed on the user's terminal

[1451] Step 6:

[1452] Personalized prevention recommendations

[1453] Based on the generated future medical images and emotional state, the server suggests preventative measures, such as exercise plans, dietary modifications, and stress management techniques.

[1454] Specifically, if the user is found to be in a state of high stress, the system will suggest a meal plan that will have a relaxing effect.

[1455] Input: Future medical images, emotional state

[1456] Output: Individually customized preventative measures

[1457] Step 7:

[1458] Meal plans and delivery links provided

[1459] The server generates a meal plan based on the emotional state and provides the user with a delivery link.

[1460] For example, if a user is experiencing high stress, the app will suggest a relaxing meal such as "chamomile tea, salmon, and avocado" and provide a link to easily order it.

[1461] Input: Emotional state

[1462] Output: Meal plan, delivery link

[1463] Step 8:

[1464] Sending follow-up notifications

[1465] The server periodically monitors the user's emotional state and sends optimal follow-up notifications, including reminders to enter new medical images and lifestyle data.

[1466] Input: User's emotional state data

[1467] Output: Follow-up notification

[1468] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1469] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1470] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1471] [Fourth embodiment]

[1472] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1473] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1474] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1475] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1476] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1477] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1478] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1479] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1480] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1481] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1482] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1483] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1484] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1485] The present invention is a system that predicts the future state of the brain based on the user's current medical images and lifestyle data, and provides the results to the user to encourage preventative care and behavioral change. Specific embodiments of the system are described below.

[1486] System Configuration

[1487] 1. User terminal: A device that allows users to input current medical images and lifestyle data. This can be a PC, tablet, or smartphone.

[1488] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs AI models.

[1489] 3. Database: A storage for storing the user's current medical images, lifestyle data, generated future medical images, and preventive measures based on them.

[1490] Program processing

[1491] The system operates through the following major processing steps:

[1492] 1. User registration and data acquisition:

[1493] User: Uploads their current medical images and lifestyle data.

[1494] Terminal: Converts the uploaded data into the appropriate format and sends it to the server.

[1495] 2. Data Processing and Future Medical Imaging:

[1496] Server: Analyzes the received data and performs image preprocessing (such as noise removal and normalization).

[1497] Server: Inputs the prepared data into the AI ​​model to generate future medical images.

[1498] 3. Visualizing and explaining the results:

[1499] Server: Stores the generated future medical images in a database and simultaneously transmits the results to the user's terminal.

[1500] Terminal: Displays future medical images and their analysis results to the user in a visually easy-to-understand format.

[1501] 4. Suggested preventive measures:

[1502] Server: Generates individually customized preventive measures based on user data and analysis results.

[1503] Device: Presents preventive measures to users and encourages specific behavioral changes.

[1504] 5. Regular follow-up:

[1505] Server: Provides periodic follow-up notifications to users, prompting them to upload new medical images and data.

[1506] Terminal: Receives follow-up notifications and displays them to the user.

[1507] Specific examples

[1508] Let us take a 50-year-old male user as an example.

[1509] 1. User: Uploads current brain images taken with an MRI device and lifestyle data indicating low exercise frequency to the device.

[1510] 2. Terminal: Sends data to the server.

[1511] 3. Server: Preprocesses MRI images and analyzes lifestyle data. This data is input into an AI model to predict the state of the brain 10 years from now.

[1512] 4. Server: Sends the generated future MRI images to the terminal and stores the results in a database.

[1513] 5. Terminal: The device displays future MRI images to the user, highlighting the tendency for hippocampal atrophy and explaining that lack of exercise is a risk factor.

[1514] 6. Server: Suggests daily walking and brain training programs as preventive measures based on the user's needs.

[1515] 7. Terminal: Presents specific exercise plans and brain training links to the user.

[1516] 8. Server: Send a follow-up notification after three months to ensure the user has implemented the suggested measures.

[1517] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[1518] This system allows users to visually understand future health risks and encourages them to take proactive preventative measures, which is expected to lead to effective behavioral changes to prevent dementia and Alzheimer's disease.

[1519] The processing flow will be explained below.

[1520] Step 1:

[1521] User: Logs in to the dedicated website or mobile application, takes and acquires current medical images (MRI images, etc.), enters their lifestyle data (exercise activity, diet, sleep time, etc.) into an input form, and uploads it to the device.

[1522] Step 2:

[1523] Terminal: Receives medical images and lifestyle data uploaded by users, converts them into a specified format (e.g., a standard data format), and sends the data to the server.

[1524] Step 3:

[1525] Server: Preprocesses the received medical images, specifically removing noise from the images, performing necessary filtering and normalization, and preparing them in a format that is easy for the AI ​​model to use.

[1526] Step 4:

[1527] Server: Analyzes lifestyle data, organizes and stores it in a standardized format, and then inputs the medical images and lifestyle data into the AI ​​model.

[1528] Step 5:

[1529] Server: Generates future medical images using an AI model. The AI ​​model uses deep learning algorithms to predict and simulate future brain states based on current images and lifestyle data.

[1530] Step 6:

[1531] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[1532] Step 7:

[1533] Terminal: Displays the received future medical images and their diagnostic results to the user, highlighting and annotating specific areas to make them easier to understand visually.

[1534] Step 8:

[1535] Server: Based on future medical images and diagnostic results, it generates personalized preventative measures for users to improve their lifestyle, including exercise plans, dietary suggestions, and stress management techniques.

[1536] Step 9:

[1537] Device: Presents users with specific action plans and preventative measures, such as daily walking plans, links to brain training programs, and examples of dietary improvements.

[1538] Step 10:

[1539] Server: Schedules and sends periodic follow-up notifications to users, including reminders to re-upload medical images at the appropriate time.

[1540] Step 11:

[1541] Device: Display a follow-up notification to the user, prompting them to enter new medical images and lifestyle data, then return to step 2 and repeat the process.

[1542] The above steps result in a system that allows users to understand their own future brain health status and take effective preventative measures.

[1543] Example 1

[1544] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1545] Current medical systems lack the means to predict a user's future health status and provide appropriate preventive measures. As a result, users are not sufficiently motivated to recognize future health risks early and improve their lifestyle habits. Furthermore, there are no systems that suggest specific preventive measures based on future medical images and lifestyle data. This makes it difficult for users to take appropriate measures in response to changes in their health status.

[1546] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1547] In this invention, the server includes means for acquiring a user's current medical images and lifestyle data, means for preprocessing the acquired medical images and analyzing the lifestyle data, means for inputting the preprocessed data into a generative AI model, means for creating prompts and generating future medical images using the generative AI model, means for saving the generated future medical images in a database, means for visualizing the generated future medical images and providing them to the user, means for suggesting preventive measures based on the future medical images, and means for periodically following up with the user, thereby enabling the user to visually understand future health risks and take appropriate preventive measures early.

[1548] "User" refers to an individual who uses the system to provide their own medical images and lifestyle data and receive information about future health risks.

[1549] "Current medical images" are images that visually show the user's physical condition, and primarily refer to MRI images and CT scan images.

[1550] "Lifestyle data" is information about the user's daily activities and habits, including, for example, exercise frequency, dietary content, and sleep patterns.

[1551] "Preprocessing" refers to the processing of raw medical image data to make it analyzable, and refers to technical operations such as noise removal, image normalization, and feature extraction.

[1552] A "generative AI model" is a model that uses artificial intelligence to predict future states, and refers to a neural network trained by a learning algorithm.

[1553] A "prompt sentence" is a text-based input used to give specific instructions or queries to a generative AI model, including, for example, content that predicts future brain states.

[1554] "Future medical images" are images that visually show the user's future physical condition, predicted using a generative AI model.

[1555] "Database" refers to a data storage system that stores and efficiently manages all data used by a system.

[1556] "Visualization" refers to the presentation of data or information in a format that users can intuitively understand, and includes methods such as graphs, charts, and image displays.

[1557] "Preventive measures" are specific actions or measures that users should take to reduce future health risks, including exercise plans, dietary advice, stress management techniques, etc.

[1558] "Follow-up" is the process of tracking changes in a user's health condition and lifestyle habits at regular intervals and providing additional advice and guidance as needed.

[1559] This invention is a system that predicts the future state of the brain based on the user's current medical images and lifestyle data, and provides the results to the user to encourage preventative care and behavioral change. This system is implemented using the following hardware and software.

[1560] Hardware Configuration

[1561] 1. User terminal: A device that allows users to input current medical images and lifestyle data. Specifically, this includes PCs, tablets, smartphones, etc.

[1562] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. This is where the entire system is managed and the generative AI model is executed.

[1563] 3. Database: A storage for storing the user's current medical images, lifestyle data, generated future medical images, and preventive measures based on them.

[1564] Software configuration and processing content

[1565] 1. Data Collection:

[1566] User: Enters their current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency, dietary content, etc.).

[1567] Terminal: Converts uploaded data into a suitable format and sends it to the server. This can be done using Python or JavaScript libraries.

[1568] 2. Data Preprocessing:

[1569] Server: Preprocesses the received medical images. Preprocessing includes noise reduction, image normalization, feature extraction, etc. Specifically, OpenCV and SciPy are used.

[1570] 3. Data Analysis:

[1571] Server: Also analyzes lifestyle data. This analysis uses libraries such as Scikit-learn.

[1572] 4. Future Predictions:

[1573] Server: The preprocessed data is input into a generative AI model. This model uses PyTorch or TensorFlow. A prompt such as "Please predict the state of the brain in 10 years" is input.

[1574] 5. Save and display results:

[1575] Server: The generated future medical images are stored in a database and simultaneously sent to the user's terminal. MySQL is used as the database system.

[1576] Terminal: Visually display the results sent. Use a library such as Matplotlib to generate graphs and charts.

[1577] 6. Suggested preventive measures:

[1578] Server: Generates individually customized preventive measures based on the generated future medical images and analysis results. Utilizing appropriate algorithms, the preventive measures are derived and provided to the user.

[1579] 7. Follow-up:

[1580] Server: Sends periodic follow-up notifications to prompt users to re-enter new medical images and lifestyle data. This is scheduled using a Cron job.

[1581] Terminal: Receives follow-up notifications and prompts the user to enter new data.

[1582] Specific examples

[1583] Specific processing will be explained using the case of a 50-year-old male user as an example.

[1584] 1. User: Uploads current brain images taken with an MRI device and lifestyle data indicating low exercise frequency to the device.

[1585] 2. Terminal: Sends data to the server.

[1586] 3. Server: Preprocesses MRI images and analyzes lifestyle data. This data is input into an AI model to predict the state of the brain 10 years from now.

[1587] 4. Server: Generates future MRI images, sends them to the device, and also stores them in the database.

[1588] 5. Terminal: Future MRI images are displayed, highlighting the tendency for hippocampal atrophy in particular. It is also explained that lack of exercise is a risk factor.

[1589] 6. Server: For example, suggest daily walking and a brain training program as preventative measures.

[1590] 7. Terminal: Presents specific exercise plans and brain training links to the user.

[1591] 8. Server: Send a follow-up notice after three months to confirm the continuation of the proposed measures.

[1592] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[1593] Example prompt: "A 50-year-old male user has provided us with MRI brain images and lifestyle data showing that he exercises infrequently. Predict his brain condition 10 years from now and suggest appropriate preventative measures."

[1594] This system allows users to visually understand future health risks and take appropriate preventive measures early on.

[1595] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1596] Step 1:

[1597] User registration and data acquisition

[1598] Users input their current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency, dietary content, etc.) through a dedicated application or web interface.

[1599] Input: Medical image files, lifestyle data

[1600] The terminal receives the data uploaded by the user, standardizes the format, and compresses the data as necessary. Specifically, it converts JPEG images into the standard DICOM format.

[1601] Output: Processed medical image files, lifestyle data

[1602] The terminal transmits the processed data to the server.

[1603] Step 2:

[1604] Data Preprocessing

[1605] The server preprocesses the medical images received from the device, including noise removal, image normalization, and feature extraction, using OpenCV and SciPy libraries.

[1606] Input: Processed medical image files

[1607] Data processing: noise removal, image normalization, feature extraction

[1608] Output: Preprocessed medical image data

[1609] Step 3:

[1610] Lifestyle data analysis

[1611] The server analyzes the lifestyle data. Here, detailed analysis of lifestyle habits is performed and specific parameters are extracted. Specifically, the "Scikit-learn" library is used.

[1612] Input: Lifestyle data

[1613] Data processing: Data normalization, feature extraction

[1614] Output: Analyzed lifestyle data

[1615] Step 4:

[1616] The Future of Medical Imaging

[1617] The server inputs the preprocessed medical image data and analyzed lifestyle data into a generative AI model, which uses PyTorch and TensorFlow.

[1618] Input: Preprocessed medical image data, analyzed lifestyle data

[1619] Prompt: For example, "Predict your brain state in 10 years."

[1620] Data Computation: Predictive Processing with AI Models

[1621] Output: Generated future medical images

[1622] Step 5:

[1623] Saving and displaying results

[1624] The server stores the generated future medical images using a database management system such as MySQL.

[1625] Input: Generated future medical images

[1626] Data processing: Saving to the database

[1627] Output: Future medical image data stored in a database

[1628] The server transmits the generated future medical images to the user terminal.

[1629] The terminal uses the Matplotlib library to generate graphs and charts to visually display the results.

[1630] Input: Generated future medical images

[1631] Output: Visualized medical images, graphs, charts

[1632] Step 6:

[1633] Preventive measures suggested

[1634] The server generates personalized preventative measures based on the generated future medical images and analysis results, using advanced algorithms and behavioral science theories, specifically Scikit-learn.

[1635] Input: Analysis results, future medical images

[1636] Data Computing: Proactive Measures

[1637] Output: Individually customized preventative measures

[1638] The device will provide the user with specific exercise plans, dietary advice, stress management methods, and more.

[1639] Step 7:

[1640] Regular follow-up

[1641] The server will send follow-up notifications periodically (e.g., every 3 months), including automatic scheduling using a "Cron" job.

[1642] Input: Schedule information

[1643] Data Calculation: Generate Follow-Up Notifications

[1644] Output: Follow-up notification

[1645] The device receives a follow-up notification and prompts the user to enter new data. Once the new data is uploaded, the same process begins again, starting with data collection.

[1646] (Application example 1)

[1647] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1648] In today's world, many people face the challenge of fully understanding future health risks and taking appropriate preventative measures. Furthermore, there is a lack of methods for providing users with a visually easy-to-understand view of predicted health risks. The goal of this project is to improve this situation and promote preventative awareness and behavioral change in users.

[1649] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1650] In this invention, the server includes means for acquiring current medical images and habit data, means for processing the acquired current medical images and habit data and inputting them into a generative artificial intelligence model, means for generating future medical images using the generative artificial intelligence model, means for visualizing the generated future medical images and providing them to the user through a display device, means for presenting individually customized preventive measures and action plans based on the future medical images, means for predicting future health risks and providing health management services through a virtual shop, and means for periodically sending tracking notifications to the user and prompting them to upload new data, thereby enabling the user to visually understand future health risks and take customized preventive measures based on their own lifestyle.

[1651] A "medical image" is an image of the human body taken with medical equipment and used for diagnosis or treatment.

[1652] "Habit data" is information about individual lifestyle habits, including data on diet, exercise, sleep, etc.

[1653] "Generative artificial intelligence models" are algorithms and machine learning models used to predict future states based on existing data.

[1654] A "virtual store" is a virtual store that offers goods and services via the Internet.

[1655] A "display device" is a device for visually presenting images and information to a user, and includes smartphones and head-mounted displays.

[1656] "Preventive measures" are specific actions or efforts to maintain health and prevent disease.

[1657] An "action plan" is a specific set of actions or a timetable for implementing preventative measures.

[1658] "Tracking notifications" are periodic reminders or notifications sent to users to inform them of their progress in taking preventative measures or to provide new data.

[1659] A "health management service" is a service that monitors a user's health status and provides appropriate advice and preventative measures.

[1660] This invention is a system that predicts the user's future brain state based on current medical images and habit data, and provides the results to the user to encourage preventative care and behavioral change. This system is mainly composed of a server, a terminal, and a user. A specific embodiment is described below.

[1661] System Configuration

[1662] 1. User terminal: A device that allows users to input current medical images and habit data. Examples include smartphones and head-mounted displays.

[1663] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs the generative AI model.

[1664] 3. Database: A storage for storing the user's current medical images, habit data, generated future medical images, and preventive measures based on them.

[1665] System Operation

[1666] Data Acquisition

[1667] Users upload their own medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency and dietary habits) to the device, which then converts the data into an appropriate format and sends it to the server.

[1668] Data Processing and Future Medical Imaging

[1669] The server analyzes the received medical images and habit data. First, the medical images undergo preprocessing, such as noise removal and normalization. The preprocessed data is input into a generative AI model, which generates future medical images. For example, based on current brain MRI images and data on lack of exercise, the AI ​​can predict the state of the brain 10 years from now.

[1670] Visualizing the results

[1671] The generated future medical images are stored in a database by the server and sent to the user's device, which displays them to the user in a visually easy-to-understand format. When a head-mounted display is used, the future brain state can be displayed realistically in 3D or AR.

[1672] Precautions

[1673] The server generates personalized preventive measures and action plans based on future medical images and current data. For example, if lack of exercise is a risk factor, daily walking and brain training programs will be suggested. These preventive measures will be provided through a virtual store, allowing users to receive a variety of health management services.

[1674] Regular follow-up

[1675] The server periodically sends tracking notifications to the user, prompting them to upload new data. The user then uploads medical images and lifestyle data again, and the system analyzes the new data, enabling continuous health management.

[1676] Specific examples

[1677] A 50-year-old male user uploads his current MRI scan and his low exercise habits to his device. The data is sent to a server, where a generative AI model predicts his brain state 10 years from now. Future MRI images are displayed in 3D via a head-mounted display, allowing the user to visually confirm his health risks. The server also suggests daily walking and brain training programs as preventative measures, which are offered through a virtual shopping mall.

[1678] Prompt Sentence Examples

[1679] "Enter your current MRI images and lifestyle data. We'll predict your future brain condition and visualize your health risks."

[1680] In this way, users can visually understand future health risks and take customized preventative measures based on their lifestyle.

[1681] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1682] Step 1:

[1683] The user uploads current medical images (e.g., MRI images) and lifestyle data (e.g., exercise frequency) to the device via a smartphone or head-mounted display. The input data are medical image files and text data related to lifestyle habits. The device converts this data into an appropriate format and sends it to the server. The output is the data transfer to the server.

[1684] Step 2:

[1685] The server analyzes the received data. First, it preprocesses the medical image data. Preprocessing such as noise removal and normalization is performed to enable the AI ​​model to accurately analyze the data. The input is medical images and lifestyle data, and the output is the preprocessed data. Specifically, OpenCV is used to remove noise from the images and perform standardization processing.

[1686] Step 3:

[1687] The preprocessed data is input into a generative AI model to generate future medical images. Specifically, TensorFlow is used to input data into the AI ​​model and perform prediction processing. The input is the preprocessed data, and the output is the generated future medical images. The generative AI model predicts future brain health based on current medical images and lifestyle data.

[1688] Step 4:

[1689] The server stores the generated future medical images in a database and transmits them to the terminal. The input is the generated future medical image, and the output is storage in the database and transmission to the terminal. Specifically, the generated image is stored in an appropriate format and transmitted to the terminal via the Internet.

[1690] Step 5:

[1691] The device visually presents the received future medical images to the user. The input is the received future medical image, and the output is visual information provided to the user. Specifically, 3D and AR displays are performed using the display functions of a smartphone or head-mounted display.

[1692] Step 6:

[1693] The server generates personalized preventive measures and action plans based on future medical images and current data. The input is future medical images and current lifestyle data, and the output is personalized preventive measures. Specifically, it uses an algorithm that evaluates the user's risk factors and suggests optimal measures to address them.

[1694] Step 7:

[1695] The server provides users with preventive measures and action plans through a virtual shop. The input is individually customized preventive measures, and the output is services provided through the virtual shop. Specifically, users can view and purchase the suggested preventive measures through the virtual shop interface.

[1696] Step 8:

[1697] The server periodically sends tracking notifications to users to prompt them to upload new data. The input is the configured follow-up schedule, and the output is the notification message. Specifically, it uses a reminder function to send emails or alerts to prompt users to update their data.

[1698] In this way, a system is created that allows users to visually understand their health risks and implement customized preventative measures.

[1699] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1700] The present invention is a system that combines a user's current medical images and lifestyle data with an emotion engine that recognizes the user's emotional state. This makes it possible to present individually customized preventive measures that take the user's emotional state into consideration and send optimal follow-up notifications. Specific embodiments of the system are described below.

[1701] System Configuration

[1702] 1. User device: A device that allows users to input current medical images, lifestyle data, and emotional state. This can be a PC, tablet, or smartphone.

[1703] 2. Server: This is the core device that receives, processes, and analyzes data sent from user devices. It also manages the entire system and runs AI models.

[1704] 3. Emotion engine: An engine that analyzes the user's emotional state and processes data in conjunction with the generative AI model.

[1705] 4. Database: A storage for storing the user's current medical images, lifestyle data, emotional state, generated future medical images, and preventive measures based on them.

[1706] Program processing

[1707] The system operates through the following major processing steps:

[1708] 1. User registration and data acquisition:

[1709] User: Uploads their current medical images and lifestyle data, and inputs their emotional state (e.g., via a questionnaire or facial expression recognition).

[1710] Terminal: Converts the uploaded data into the appropriate format and sends it to the server.

[1711] 2. Data Processing and Future Medical Imaging:

[1712] Server: Analyzes the received data and performs image preprocessing (noise removal, normalization, etc.). It also analyzes lifestyle data and emotional state data, and organizes and stores them in a standardized format.

[1713] Server: Inputs the prepared data into the AI ​​model and emotion engine to generate future medical images.

[1714] 3. Visualizing and explaining the results:

[1715] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[1716] Terminal: Displays future medical images and their diagnostic results to the user in a visually easy-to-understand format.

[1717] 4. Suggested preventive measures:

[1718] Server: Generates individually customized preventive measures based on the user's data and analysis results. The emotion engine adjusts the way the preventive measures are presented based on the user's emotional state.

[1719] Device: Presents users with specific action plans and preventative measures, such as daily walking plans, links to brain training programs, and examples of dietary improvements.

[1720] 5. Emotional state monitoring and follow-up:

[1721] Server: The emotion engine monitors the user's emotional state in real time and sends the data to the server.

[1722] Server: Sends follow-up notifications at optimal times based on the user's emotional state.

[1723] On the device: Follow-up notifications are displayed to the user, prompting them to enter new medical images and lifestyle data.

[1724] Specific examples

[1725] Let us take a 50-year-old female user as an example.

[1726] 1. User: Uploads current brain images taken with an MRI machine, lifestyle data such as low exercise frequency, and recent stress levels to the device.

[1727] 2. Terminal: Sends data to the server.

[1728] 3. Server: Preprocesses MRI images and analyzes lifestyle and emotional state data. This data is input into an AI model to predict the brain state 10 years from now.

[1729] 4. Server: Sends the generated future MRI images to the terminal and stores the results in a database.

[1730] 5. Terminal: Future MRI images are displayed to the user, highlighting the tendency for hippocampal atrophy, while explaining that lack of exercise and high stress are risk factors.

[1731] 6. Server: Considers the user's emotional state and suggests exercise plans and stress management methods to promote relaxation.

[1732] 7. Terminal: Presents the user with specific exercise plans and brain training links for relaxation.

[1733] 8. Server: Send a follow-up notification after three months to ensure the user has implemented the suggested measures.

[1734] 9. Terminal: Display a follow-up notification and prompt the user to enter new data.

[1735] This system allows users to visually understand future health risks and encourages them to take proactive preventative measures while taking into account their emotional state, which is expected to lead to effective behavioral changes to prevent dementia and Alzheimer's disease.

[1736] The processing flow will be explained below.

[1737] Step 1:

[1738] User: Logs in to a dedicated website or mobile application, takes and acquires current medical images (MRI images, etc.), enters lifestyle data (exercise frequency, diet, sleep time, stress level, etc.) and emotional state into an input form, and uploads the data to the device.

[1739] Step 2:

[1740] Terminal: Receives uploaded medical images, lifestyle data, and emotional state data, converts the data into a unified format, and sends it to the server.

[1741] Step 3:

[1742] Server: Preprocesses the received medical images, specifically removing noise from the images, performing necessary filtering and normalization, and preparing them in a format that is easy for the AI ​​model to use.

[1743] Step 4:

[1744] Server: Analyzes lifestyle data and emotional state data, organizes and stores them in a standardized format, and then inputs the medical images, lifestyle data, and emotional state data into the AI ​​model and emotion engine.

[1745] Step 5:

[1746] Server: Generates future medical images using an AI model. The AI ​​model uses deep learning algorithms to predict and simulate future brain states from current images and data.

[1747] Step 6:

[1748] Server: The emotion engine analyzes the emotional state data and monitors the user's emotional state in real time. This information is used to tailor the presentation of future medical images and preventative measures.

[1749] Step 7:

[1750] Server: Stores the generated future medical images in a database and sends the results to the user's device, along with detailed information on diagnosis results and abnormality detection.

[1751] Step 8:

[1752] Terminal: Displays the received future medical images and their diagnostic results to the user, highlighting and annotating specific areas to make them easier to understand visually.

[1753] Step 9:

[1754] Server: Generates preventive measures customized to the user's individual needs based on future medical images and diagnostic results. The emotion engine selects and adjusts the presentation of preventive measures based on the user's emotional state.

[1755] Step 10:

[1756] Device: Providing specific preventative measures to the user, such as exercise plans to promote relaxation, stress management techniques, links to brain training programs, and examples of dietary improvements.

[1757] Step 11:

[1758] Server: Schedules and sends periodic follow-up notifications to users, including reminders to re-upload new medical images and data when appropriate.

[1759] Step 12:

[1760] On the device: A follow-up notification is displayed to the user, prompting them to enter new medical images and lifestyle data as the next step, then returning to step 1 to repeat the process.

[1761] These steps result in a system that allows users to understand their future brain health status and implement individually customized preventative measures that take into account their emotional state.

[1762] Example 2

[1763] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1764] Previous systems only collected a user's medical images and lifestyle data separately, but were unable to take their emotional state into account. This made it difficult to accurately predict a user's future health risks and provide individually customized preventive measures. Furthermore, it was not possible to monitor a user's emotional state in real time and respond accordingly when proposing preventive measures or providing follow-up notifications to the user.

[1765] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1766] In this invention, the server includes: means for a user to input current medical images, lifestyle data, and emotional state data; means for transmitting the acquired current medical images, lifestyle data, and emotional state data to the server; means for preprocessing and integrating the data received by the server; means for inputting the preprocessed data into a generative AI model and an emotion engine; means for generating future medical images using the generative AI model and the emotion engine; means for visualizing the generated future medical images and providing them to a user terminal; means for presenting individually customized preventive measures based on the future medical images; means for monitoring the user's emotional state in real time and sending follow-up notifications; and means for the user who receives the follow-up notifications to input new data. This enables accurate prediction of future health risks taking into account the user's emotional state and the provision of individually customized preventive measures.

[1767] "User terminal" refers to a device that allows a user to input medical images, lifestyle data, and emotional state data and transmit them to the server. This includes PCs, tablets, smartphones, etc.

[1768] A "server" is a core device that receives data sent from user terminals and processes and analyzes the data.

[1769] "Medical images" are video data that show the internal state of a user's body, such as MRI images and CT scan images.

[1770] "Lifestyle data" refers to data related to the user's daily activities and habits, such as exercise time and dietary details.

[1771] "Emotional state data" refers to data relating to the user's current emotional and mood state, including stress levels and mood states.

[1772] "Preprocessing" refers to the process of converting received data (medical images, lifestyle data, emotional state data) into a format that is easier to analyze. Specifically, this includes noise removal and normalization.

[1773] A "generative AI model" is an artificial intelligence model that generates future medical images based on user data.

[1774] The "emotion engine" is an engine that analyzes the user's emotional state and reflects this in the analysis results of the generative AI model and the way in which preventive measures are presented.

[1775] "Future medical images" are images that show the user's future medical condition, predicted using a generative AI model.

[1776] "Visualization" refers to the visual display of data and analytical results using a computer or display.

[1777] "Preventive measures" are specific instructions or action plans provided to users to avoid future health risks.

[1778] A "follow-up notification" is a notification that confirms whether the user is taking the suggested preventative measures and encourages further action.

[1779] The present invention is a system that combines a user's current medical image, lifestyle data, and emotional state data to generate a future medical image and provide individually customized preventive measures, allowing the user to identify health risks in advance and take appropriate measures.

[1780] System Configuration

[1781] 1. User Device

[1782] User terminal: A device through which a user inputs medical images, lifestyle data, and emotional state data. Specifically, this applies to PCs, tablets, and smartphones.

[1783] Users upload current medical images taken with an MRI machine and enter lifestyle data such as daily exercise time, dietary habits, and stress levels. This data is then formatted and sent from the device to the server.

[1784] 2. Server

[1785] Server: This is the core device that receives, processes, and analyzes data sent from user devices. The server performs preprocessing such as noise reduction and normalization on the received medical image data, and analyzes lifestyle and emotional state data. The preprocessed data is then input into the generative AI model and emotion engine to generate future medical images.

[1786] Specifically, the server uses the following software:

[1787] Preprocess medical images using an image processing library (e.g., OpenCV).

[1788] Use a data analysis library (e.g., Pandas) to standardize and integrate lifestyle data and emotional state data.

[1789] Deep learning frameworks (e.g., TensorFlow, PyTorch) are used to run generative AI models and generate future medical images.

[1790] 3. Database and visualization system

[1791] Database: A storage for storing the user's current medical images, lifestyle data, emotional state data, generated future medical images, and preventive measures based on them.

[1792] Visualization system: This system visually displays the generated future medical images, analysis results, and preventive measures on the user's device. The data is converted into PNG or JPEG format and provided to the user with detailed annotations.

[1793] 4. Proposal of preventive measures

[1794] Based on the generated future medical images and analysis results, the server generates personalized preventative measures, including exercise plans, dietary suggestions, and stress management techniques, all of which reflect the user's emotional state.

[1795] On your device: View detailed prevention measures, including specific action plans and links.

[1796] 5. Follow-up notifications and emotional state monitoring

[1797] The server uses an emotion engine to monitor the user's emotional state in real time and send follow-up notifications at optimal times.

[1798] On your device: Receive follow-up notifications to prompt you to enter new medical images and lifestyle data.

[1799] Specific examples

[1800] Let us take a 50-year-old female user as an example.

[1801] 1. User: The user inputs into the terminal the current brain image taken with an MRI device, lifestyle data such as low exercise frequency and high stress level.

[1802] 2. Terminal: Formats the data and sends it to the server.

[1803] 3. Server: The received MRI images are denoised and normalised, and the lifestyle and emotional state data are analysed. Based on this data, a generative AI model is used to predict the brain state 10 years from now.

[1804] 4. Server: The generated MRI images from 10 years later are stored in a database and sent to the user's terminal.

[1805] 5. Terminal: Displays future MRI images showing hippocampal atrophy and explains that lack of exercise and high stress are risk factors.

[1806] 6. Server: Considers the user's emotional state and suggests exercise plans and stress management methods to promote relaxation.

[1807] 7. Device: Displays specific exercise plans and links to relaxation.

[1808] 8. Server: Send a follow-up notification after 3 months and ask the user to enter the data again.

[1809] Prompt Sentence Examples

[1810] "A 50-year-old female user has provided lifestyle data showing that she exercises infrequently and has a high stress level, and has uploaded current brain images taken with an MRI machine. Please predict her future brain state and suggest an exercise plan to promote relaxation and stress management methods. Also, please include a process for sending a follow-up notification three months later to check whether the suggested measures have been implemented."

[1811] As described above, the present invention realizes prediction of future health risks and provision of individually customized preventive measures while taking into account the user's emotional state.

[1812] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1813] Step 1: User registration and data acquisition

[1814] User: Enters their own medical images (e.g., MRI images), lifestyle data (e.g., exercise frequency and dietary content), and emotional state data (e.g., stress level) using a PC, tablet, or smartphone.

[1815] Input: Medical image data, lifestyle data, emotional state data.

[1816] Output: Formatted data.

[1817] Specific operation: The user launches the application on the device they are using, enters data in the specified format, and presses the send button.

[1818] Step 2: Sending data

[1819] Terminal: Checks the data entered by the user, converts it into the appropriate format, and sends it to the server.

[1820] Input: User-entered medical image data, lifestyle data, and emotional state data.

[1821] Output: The formatted data sent to the server.

[1822] Specific operation: The terminal uses a communication protocol to send input data to the server in JSON format, DICOM format, etc.

[1823] Step 3: Receiving and Preprocessing Data

[1824] Server: Preprocesses the received data. Medical image data is denoised and normalised, while lifestyle data and emotional state data are analysed and standardised.

[1825] Input: Submitted medical image data, lifestyle data, and emotional state data.

[1826] Output: The preprocessed dataset.

[1827] Specific operation: The server denoises medical images and analyzes and standardizes lifestyle data and emotional state data using Pandas.

[1828] Step 4: Integrate the data

[1829] Server: Integrates pre-processed medical image data, lifestyle data, and emotional state data into a single dataset.

[1830] Input: Preprocessed individual data.

[1831] Output: A consolidated dataset.

[1832] Specific operation: The server consolidates all preprocessed data and converts it into a format that can be input into the AI ​​model.

[1833] Step 5: Analysis by AI model and emotion engine

[1834] Server: Inputs the integrated dataset into the generative AI model and emotion engine to generate future medical images.

[1835] Input: Unified dataset.

[1836] Output: Futuristic medical images.

[1837] Specific operation: The AI ​​model is run on the server using TensorFlow or PyTorch, and analysis is performed in conjunction with the emotion engine. Predictions are made of the brain's state 10 years from now.

[1838] Step 6: Save and send the results

[1839] Server: Stores the generated future medical images in a database and sends the analysis results to the user's device.

[1840] Input: futuristic medical images.

[1841] Output: Analysis results sent to the user's device.

[1842] Specific operation: The results are converted into PNG or JPEG format and sent to be displayed in a user-friendly format.

[1843] Step 7: Visualize the results

[1844] Terminal: Visually displays future medical images and diagnostic results to the user.

[1845] Input: Analysis results sent from the server.

[1846] Output: The results that are displayed to the user.

[1847] Specific operation: The device displays the received images and data in a dashboard or slideshow format, informing the user of the latest results.

[1848] Step 8: Generate and propose preventative measures

[1849] Server: Generates personalized preventive measures based on the analysis results, and adjusts the recommendations based on the user's emotional state.

[1850] Input: Analysis results, emotion engine results.

[1851] Output: Customized precautions.

[1852] Specific actions: The server proposes specific exercise plans and stress management techniques based on the generated data.

[1853] Step 9: Displaying preventative measures

[1854] Device: Presents users with specific action plans and preventative measures (e.g., exercise plans, dietary suggestions, stress management links).

[1855] Input: Prevention data sent from the server.

[1856] Output: The precautions presented to the user.

[1857] Specific action: Display preventative measures on the device in the form of a pop-up notification or detailed PDF.

[1858] Step 10: Follow-up notifications and data updates

[1859] Server: Monitors emotional state in real time and sends follow-up notifications periodically.

[1860] Input: Real-time data from the emotion engine, new input data from users.

[1861] Output: Follow-up notification, updated dataset.

[1862] What it does: Sends reminders every three months to prompt users to enter new medical images and lifestyle data.

[1863] Through these steps, the system can predict the user's future health risks and provide appropriate preventative measures that take into account their emotional state.

[1864] (Application example 2)

[1865] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1866] In today's world, maintaining personal health requires a system that can centrally manage medical and lifestyle data and suggest appropriate preventive measures. However, conventional systems rely only on simple lifestyle and medical data without considering the user's emotional state, which is often overlooked, making it difficult to suggest and implement effective preventive measures. Furthermore, they lack specific action plans and support to help users implement the proposed preventive measures, and follow-up is insufficient.

[1867] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1868] In this invention, the server includes: means for acquiring current medical images, lifestyle data, and emotional states; means for processing the acquired current medical images, lifestyle data, and emotional states and inputting them into a generative artificial intelligence model; means for generating future medical images using the generative artificial intelligence model; means for visualizing the generated future medical images and providing them to the user; means for presenting individually customized preventive measures based on the future medical images; means for periodically monitoring the user's emotional state and sending follow-up notifications at optimal times; and means for generating a meal plan based on the user's emotional state and providing the user with specific meal suggestions and delivery links. This makes it possible to support the proposal and implementation of effective, individually customized preventive measures based on lifestyle and medical images while taking the user's emotional state into consideration. Furthermore, providing specific meal suggestions and delivery links makes it easier for the user to take action immediately, thereby increasing the effectiveness of the preventive measures.

[1869] A "medical image" is an image of the internal structures of the human body taken with a medical device.

[1870] "Lifestyle data" refers to data relating to the user's lifestyle, such as exercise status, dietary content, and sleep patterns.

[1871] "Emotional state" is data that indicates the user's psychological state, and includes information such as stress level and happiness.

[1872] A "generative artificial intelligence model" is a machine learning model that predicts future states based on acquired data.

[1873] "Future medical images" are future medical images predicted based on current medical images, lifestyle data, and emotional state.

[1874] "Visualization" refers to visually displaying the generated data so that the user can easily understand it.

[1875] "Preventive measures" are specific action plans or advice to take to prevent future deterioration of a medical condition.

[1876] "Follow-up notifications" are notifications that track the implementation status of preventive measures and provide reminders or additional guidance to users at appropriate times.

[1877] A "meal plan" is a plan that suggests healthy meals based on the user's health and emotional state.

[1878] "Delivery Link" means an internet link that allows for easy ordering of the suggested meal.

[1879] The invention is a system that integrates and analyzes a user's medical images, lifestyle data, and emotional state, thereby enabling personalized preventative measures and dietary recommendations.

[1880] The system consists of the following main components:

[1881] 1. User Device:

[1882] Users use their smartphones to input their medical images, lifestyle data, and emotional state. This data is then sent to a server. For example, users can use a health app to record their daily exercise routine, take photos of their meals, and answer questionnaires about their emotional state.

[1883] 2. Server:

[1884] It analyzes the received data and generates future medical images using AI models and an emotion engine, while also monitoring the user's emotional state and sending follow-up notifications at optimal times.

[1885] 3. Emotion Engine:

[1886] This is an engine for analyzing the user's emotional state. It analyzes the user's emotions based on data such as photos, videos, and questionnaires, and sends the data to a server.

[1887] 4. Database:

[1888] It stores the user's current medical images, lifestyle data, and emotional state, as well as any future medical images and preventative measures that are generated.

[1889] 5. Meal Plan Generator:

[1890] A subsystem for suggesting appropriate meal plans to users based on their emotional state and providing delivery links.

[1891] Program processing explanation

[1892] 1. Hardware and software used:

[1893] User device: Smartphone (health app)

[1894] Server: Data analysis, collaboration with AI models and emotion engines, meal plan generation

[1895] Emotion Engine API: Analyzing the user's emotional state

[1896] Database: storing and managing user data

[1897] 2. Data processing and calculation:

[1898] The server receives data (medical images, lifestyle data, and emotional state) sent from the user's device and performs processes such as data cleansing, noise removal, and standardization. The data is then input into a generative AI model to generate future medical images. An emotion engine is also used to analyze the user's emotional state, and a meal plan is generated based on the results.

[1899] Specific examples

[1900] For example, a user can upload a photo of themselves using a smartphone app and answer a questionnaire about their recent stress level. This data is sent to a server, and the emotion engine analyzes it as a "high stress state." As a result, the server suggests a relaxing meal to the user, such as "chamomile tea, salmon, and avocado," and provides a delivery link for ordering it on the spot.

[1901] Prompt Sentence Examples

[1902] "Analyze the user's emotional state and suggest foods that will help them relax when they are under high stress."

[1903] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1904] Step 1:

[1905] Data entry and submission

[1906] Users use a smartphone app to input their own medical images, lifestyle data, and emotional state, including uploading MRI images, answering questionnaires about exercise frequency and diet, and recording their emotional state through facial recognition.

[1907] This data is converted into an appropriate format and sent to the server.

[1908] Input: medical images, lifestyle data, emotional state

[1909] Output: Data sent to the server

[1910] Step 2:

[1911] Data reception and preprocessing

[1912] The server receives data sent from the user terminal, and the received data is first preprocessed.

[1913] Specifically, this includes denoising and normalising medical images, standardising lifestyle data and analyzing emotional states.

[1914] Input: Data sent from the user's device

[1915] Output: Preprocessed data

[1916] Step 3:

[1917] AI models for generating future medical images

[1918] The server uses the pre-processed data to input the AI ​​model and generate future medical images.

[1919] For example, based on current MRI images and lifestyle data, the state of the brain 10 years from now can be predicted.

[1920] Input: Preprocessed medical images, lifestyle data, emotional states

[1921] Output: Medical images of the future

[1922] Step 4:

[1923] Emotional state analysis using emotion engine

[1924] The server analyzes the user's emotional state using an emotion engine, which determines the user's current emotional state based on questionnaire responses and facial expression recognition data.

[1925] Input: User's emotional state data

[1926] Output: Parsed emotional state

[1927] Step 5:

[1928] Visualizing and presenting results

[1929] The server transmits the generated future medical images and the analysis results of the emotional state to the user's device, which displays them in a visually easy-to-understand format.

[1930] For example, it can display future MRI images and highlight abnormalities, and also provide explanations based on the patient's emotional state.

[1931] Input: Future medical images, emotional state

[1932] Output: The results displayed on the user's terminal

[1933] Step 6:

[1934] Personalized prevention recommendations

[1935] Based on the generated future medical images and emotional state, the server suggests preventative measures, such as exercise plans, dietary modifications, and stress management techniques.

[1936] Specifically, if the user is found to be in a state of high stress, the system will suggest a meal plan that will have a relaxing effect.

[1937] Input: Future medical images, emotional state

[1938] Output: Individually customized preventative measures

[1939] Step 7:

[1940] Meal plans and delivery links provided

[1941] The server generates a meal plan based on the emotional state and provides the user with a delivery link.

[1942] For example, if a user is experiencing high stress, the app will suggest a relaxing meal such as "chamomile tea, salmon, and avocado" and provide a link to easily order it.

[1943] Input: Emotional state

[1944] Output: Meal plan, delivery link

[1945] Step 8:

[1946] Sending follow-up notifications

[1947] The server periodically monitors the user's emotional state and sends optimal follow-up notifications, including reminders to enter new medical images and lifestyle data.

[1948] Input: User's emotional state data

[1949] Output: Follow-up notification

[1950] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1951] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1952] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1953] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1954] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1955] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1956] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1957] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1958] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1959] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1960] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1961] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1962] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1963] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1964] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1965] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1966] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1967] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1968] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1969] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1970] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1971] The following is further disclosed regarding the above embodiment.

[1972] (Claim 1)

[1973] a means of obtaining current medical imaging and lifestyle data;

[1974] means for processing the acquired current medical images and lifestyle data and inputting them into a generative artificial intelligence model;

[1975] a means for generating future medical images using a generative artificial intelligence model;

[1976] A means for visualizing the generated future medical images and providing them to a user;

[1977] A means for suggesting preventive measures based on future medical images;

[1978] A means of following up with users on a regular basis;

[1979] A system including:

[1980] (Claim 2)

[1981] 10. The system of claim 1, wherein the generated future medical images are indicative of the state of the hippocampus of the brain.

[1982] (Claim 3)

[1983] 10. The system of claim 1, wherein the system provides personalized preventative measures based on the acquired lifestyle data.

[1984] "Example 1"

[1985] (Claim 1)

[1986] means for obtaining current medical imaging and lifestyle data of the user;

[1987] means for preprocessing the acquired medical images and analyzing the lifestyle data;

[1988] a means for inputting the preprocessed data into a generative AI model;

[1989] A means for creating prompts and generating future medical images using a generative AI model;

[1990] a means for storing the generated future medical images in a database;

[1991] A means for visualizing the generated future medical images and providing them to a user;

[1992] A means for suggesting preventive measures based on future medical images;

[1993] A means of following up with users on a regular basis;

[1994] A system including:

[1995] (Claim 2)

[1996] 10. The system of claim 1, wherein the generated future medical images are indicative of the state of the hippocampus of the brain.

[1997] (Claim 3)

[1998] 10. The system of claim 1, wherein the system provides personalized preventative measures based on the acquired lifestyle data.

[1999] "Application Example 1"

[2000] (Claim 1)

[2001] a means of acquiring current medical image and habit data;

[2002] means for processing the acquired current medical image and habit data and inputting it into a generative artificial intelligence model;

[2003] a means for generating future medical images using a generative artificial intelligence model;

[2004] A means for visualizing the generated future medical image and providing it to a user through a display device;

[2005] A means to provide personalized preventative measures and action plans based on future medical images;

[2006] A means for predicting future health risks and providing health management services via a virtual store;

[2007] A method for periodically sending tracking notifications to users to prompt them to upload new data;

[2008] A system including:

[2009] (Claim 2)

[2010] 10. The system of claim 1, wherein the generated future medical images are indicative of the state of memory centers of the brain.

[2011] (Claim 3)

[2012] 10. The system of claim 1, wherein the system presents customized preventative measures through a virtual shopping store based on the acquired habit data.

[2013] "Example 2: Combining Emotion Engines"

[2014] (Claim 1)

[2015] means for a user to input current medical images and lifestyle and emotional state data;

[2016] means for transmitting the acquired current medical images, lifestyle data and emotional state data to a server;

[2017] means for preprocessing and integrating the data received by the server;

[2018] a means for inputting the preprocessed data into a generative AI model and an emotion engine;

[2019] A means for generating future medical images using a generative AI model and an emotion engine;

[2020] A means for visualizing the generated future medical image and providing it to a user terminal;

[2021] A means of suggesting personalized preventative measures based on future medical images;

[2022] means for monitoring the user's emotional state in real time and sending follow-up notifications;

[2023] a means for a user who receives a follow-up notification to enter new data;

[2024] A system including:

[2025] (Claim 2)

[2026] 10. The system of claim 1, wherein the generated future medical images are indicative of the state of the hippocampus of the brain.

[2027] (Claim 3)

[2028] 10. The system of claim 1, wherein the system provides personalized preventative measures based on the acquired lifestyle and emotional state data.

[2029] "Application example 2 when combining emotion engines"

[2030] (Claim 1)

[2031] a means of obtaining current medical imaging and lifestyle data, and emotional state;

[2032] means for processing the acquired current medical images, lifestyle data, and emotional state and inputting them into a generative artificial intelligence model;

[2033] a means for generating future medical images using a generative artificial intelligence model;

[2034] A means for visualizing the generated future medical images and providing them to a user;

[2035] A means of suggesting personalized preventative measures based on future medical images;

[2036] a means for periodically monitoring the user's emotional state and sending follow-up notifications at optimal times;

[2037] means for generating a meal plan based on the emotional state and providing the user with specific meal suggestions and delivery links;

[2038] A system including:

[2039] (Claim 2)

[2040] 10. The system of claim 1, wherein the generated future medical images are indicative of the state of the hippocampus of the brain.

[2041] (Claim 3)

[2042] 10. The system of claim 1, wherein the system provides personalized preventative and dietary suggestions based on the acquired lifestyle data and emotional state. [Explanation of symbols]

[2043] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means of obtaining current medical imaging and lifestyle data; means for processing the acquired current medical images and lifestyle data and inputting them into a generative artificial intelligence model; a means for generating future medical images using a generative artificial intelligence model; A means for visualizing the generated future medical images and providing them to a user; A means for suggesting preventive measures based on future medical images; A means of following up with users on a regular basis; A system including:

2. 10. The system of claim 1, wherein the generated future medical images are indicative of the state of the hippocampus of the brain.

3. The system of claim 1 , wherein the system provides individually customized preventive measures based on the acquired lifestyle data.

Images