System

Abstract

A system is provided.SOLUTION: A system comprising: means for a user to input a health consultation by text, voice, or image; means for a device to send the user input information to a server; means for the server to analyze the received information by a AI diagnostic module and make an initial diagnosis; means for the server to generate and send additional questions to the user through the device; means for receiving the user's additional answers and providing a final diagnostic result; means for providing a recommended action or action plan based on the diagnostic result; and means for linking with an external service and making necessary reservations and arrangements.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional medical system, many people tend to overlook minor health problems as a way to prevent the disease from worsening, and only visit the hospital once the illness has progressed. This makes it difficult to detect and treat illness early, resulting in delayed treatment and an increased risk of the disease becoming more severe. Furthermore, there is a lack of a way for users to easily consult about their health and quickly receive an appropriate diagnosis and action. To solve these problems and realize preventive medicine, a new system is needed that allows users to easily consult about their health and provides a prompt and accurate diagnosis and action plan. [Means for solving the problem]

[0005] The present invention provides a means for users to input health-related inquiries via text, voice, or images. The terminal then has a means for transmitting the user-input information to a server, and the server analyzes the received information using an AI diagnostic module to perform an initial diagnosis. The server also has a means for generating follow-up questions and transmitting them to the user via the terminal, and a means for receiving the user's follow-up answers and providing a final diagnosis result. Furthermore, the system includes a means for providing recommended behaviors and action plans based on the diagnosis results, and incorporates a means for linking with external services and making necessary reservations and arrangements, thereby providing a system that allows users to easily implement preventive medical care.

[0006] "User" refers to an individual or user who uses the AI ​​diagnostic system to provide health diagnoses or consultations.

[0007] "Terminal" refers to a device (e.g., smartphone, tablet, computer, etc.) through which a user accesses the AI ​​diagnostic system and inputs and receives health-related data.

[0008] "Server" refers to the central system that receives data sent by users, analyzes it using the AI ​​diagnostic module, and generates diagnostic results and recommended actions.

[0009] "Text" is a form of health consultation content entered by a user, which expresses information in characters.

[0010] "Voice" is a form of health consultation content that is input by the user by speaking, and is treated as voice data.

[0011] "Images" refers to visual health data obtained by camera, scan, etc., including, for example, photographs of skin abnormalities.

[0012] "AI Diagnostic Module" refers to artificial intelligence algorithms and programs that analyze received text, voice, and image data and generate initial health diagnostic results.

[0013] "Early diagnosis" refers to a diagnosis performed by an AI diagnostic module that identifies possible health problems at the first stage.

[0014] "Additional questions" refer to questions that are generated to obtain more detailed information based on the initial diagnosis results.

[0015] "Final diagnosis result" refers to the final health diagnosis result provided by the AI ​​diagnostic module based on the user's additional answers.

[0016] "Recommended actions" refers to recommended actions such as treatments and measures provided to users based on the final diagnosis results.

[0017] An "action plan" is a plan that provides specific guidelines and procedures (such as making a hospital appointment or purchasing medication) in line with recommended actions.

[0018] "External services" refer to external e-commerce sites, help apps, taxi services, etc. that are linked to the AI ​​diagnostic system, and provide the infrastructure for making reservations and arrangements for users. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention relates to a health checkup system using AI, in which users input their health-related inquiries using text, voice, or images, which are analyzed by a server, and diagnostic results and recommended actions are provided. This enables early detection and prevention of diseases, and supports the user's health management.

[0041] System Overview

[0042] 1. User Interface

[0043] Users access the application using a device (e.g., a smartphone or tablet) and enter their health consultation information, which can be entered either as text, voice, or images.

[0044] 2. Data Transmission

[0045] The device transmits the health information entered by the user to the server in real time. This information is sent to the server as text data, voice data, and image data.

[0046] 3. AI-based data analysis

[0047] The server inputs the received data into the AI ​​diagnostic module, which performs natural language processing, voice analysis, and image analysis to make an initial diagnosis. The AI ​​diagnostic module then refers to past data and a medical knowledge base to suggest possible illnesses and treatments.

[0048] 4. Generate and submit follow-up questions

[0049] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[0050] 5. Providing diagnostic results

[0051] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results and recommended measures (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[0052] 6. Implement the action plan

[0053] Based on the diagnosis results, the server provides recommended actions and action plans. This includes linking with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as needed. This information is then sent to the user via their device, allowing them to take action based on the information provided.

[0054] Specific examples

[0055] 1. If the user complains of a cough

[0056] User: Opens the app, types in the text "My cough has been getting worse lately," and submits.

[0057] Terminal: Sends input information to the server.

[0058] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates an additional question such as "When did the coughing symptoms start?" and sends it to the device.

[0059] Terminal: Display the question to the user.

[0060] User: "A week ago."

[0061] Terminal: Sends the answer to the server.

[0062] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids" along with recommended actions, which are then sent to the device.

[0063] On the device: Display diagnostic results and recommended actions to the user.

[0064] 2. If the user complains of skin abnormalities

[0065] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[0066] Device: Sends photos and text information to the server.

[0067] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[0068] Terminal: Display the question to the user.

[0069] User: "A week ago."

[0070] Terminal: Sends the answer to the server.

[0071] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist" along with information about nearby dermatologists, and sends this to the device.

[0072] On the device: Display diagnostic results and recommended actions to the user.

[0073] In this way, the AI ​​diagnostic system of the present invention allows users to easily receive health consultations and receive prompt and accurate diagnoses and recommended actions. It is a system that promotes early detection of diseases and preventive medicine, and supports users' health management.

[0074] The processing flow will be explained below.

[0075] Step 1:

[0076] User: Installs the application, enters personal information (name, age, gender, medical history, etc.) to create an account, and sets authentication information for future logins.

[0077] Step 2:

[0078] Device: Sends personal information and authentication information to a server and requests the creation of a user account.

[0079] Step 3:

[0080] Server: Stores the received personal information in a database, generates an authentication token, and sends it to the device. This allows the user to log in using the authentication token from the next time onwards.

[0081] Step 4:

[0082] User: Launches the application and enters their health consultation via text, voice, or image. Once completed, they press the send button.

[0083] Step 5:

[0084] Terminal: Receives health information entered by the user in real time and sends it to the server as text, audio, or image data.

[0085] Step 6:

[0086] Server: The received health information is input into the AI ​​diagnostic module, which performs text, voice, or image analysis. Based on the analysis results, an initial diagnosis is made and possible illnesses and symptoms are identified.

[0087] Step 7:

[0088] Server: Based on the results of the initial diagnosis, generate additional questions to gather more detailed information, such as "How long has your cough been bad?"

[0089] Step 8:

[0090] Server: Sends the generated follow-up question to the terminal.

[0091] Step 9:

[0092] Terminal: Displays the question received from the server to the user.

[0093] Step 10:

[0094] User: Answer the questions displayed, enter text such as "It's been a week since" and submit.

[0095] Step 11:

[0096] Terminal: Sends the user's additional answers to the server.

[0097] Step 12:

[0098] Server: The additional answers received are input into the AI ​​diagnostic module again, where they are analyzed and a final diagnostic result is generated.

[0099] Step 13:

[0100] Server: Generates and sends the final diagnosis and recommended measures, such as "You may have a cold. Rest and drink plenty of fluids." to the device.

[0101] Step 14:

[0102] On the device: The diagnostic results and recommended actions from the server are displayed to the user.

[0103] Step 15:

[0104] Server: Based on the diagnosis results, generate an action plan to support the user, including, for example, information on the nearest pharmacy, a link to make an appointment with a medical institution, or information on how to arrange a taxi.

[0105] Step 16:

[0106] Device: Notifies the user of the action plan from the server, making it easier for the user to act according to the plan.

[0107] Step 17:

[0108] User: Review the action plan displayed on the device and take specific action as needed using the links and information provided, for example, making a doctor's appointment or ordering a taxi.

[0109] This processing step allows users to easily consult about their health, and the AI ​​system quickly and accurately provides a diagnosis and recommended actions, enabling early diagnosis and preventive medical care.

[0110] Example 1

[0111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] In modern society, it is difficult to obtain prompt and appropriate advice for daily health management or minor health consultations. Receiving expert advice at the appropriate time is especially important for early detection and prevention of illness. However, the time, effort, and cost involved in visiting a medical institution often make people hesitant to do so. Therefore, there is a need for a system that is easy for users to use and can provide accurate diagnoses and effective action plans.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0114] In this invention, the server includes: a means for a user to input health-related inquiries via text, voice, or image; a means for a terminal to transmit the user-input information to the server; a means for the server to analyze the received information using a generative AI model diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis; a means for providing recommended behaviors and action plans based on the diagnosis results; and a means for linking with external services to make necessary reservations and arrangements. This allows users to easily receive health consultations via devices such as smartphones and tablets, and receive prompt and appropriate diagnoses and recommended actions. Linking with external services also makes it easy to make additional reservations and arrangements, and supports access to actual medical institutions.

[0115] A "user" refers to a person who provides health consultation and uses the system via a device such as a smartphone or tablet.

[0116] "Health consultation" refers to providing information in text, audio, or image format about physical or mental abnormalities or concerns that a user is experiencing.

[0117] "Text" refers to information expressed by characters entered by a user.

[0118] "Voice" refers to information expressed by audible input from a user.

[0119] "Images" refers to photographs and image files taken by users using a camera or other device.

[0120] "Device" refers to electronic devices that users can use, such as smartphones, tablets, and personal computers.

[0121] "Server" refers to the central processing unit that manages the entire system and receives and analyzes information from users.

[0122] A "generative AI model diagnostic module" refers to a software module that uses machine learning technology to analyze data from users and diagnose their health condition based on the results.

[0123] "Initial diagnosis" refers to a provisional diagnosis based on initial input from the user.

[0124] "Additional questions" refer to additional information that needs to be asked of the user in order to make a more accurate diagnosis based on the results of the initial diagnosis.

[0125] "Final Diagnosis" refers to the final medical diagnosis based on all input information from the User.

[0126] "Recommended actions and action plans" refer to specific measures and next steps suggested to users based on the diagnostic results.

[0127] "External services" refers to external resources such as medical institutions, reservation systems, and e-commerce sites that are used by systems to work together and increase user convenience.

[0128] "Reservations and arrangements" refers to using external services to make reservations at medical institutions and arrange for necessary items.

[0129] This invention relates to a health checkup system using AI, in which users input their health-related inquiries using text, voice, or images, which are analyzed by a server, and diagnostic results and recommended actions are provided. This enables early detection and prevention of diseases, and supports the user's health management.

[0130] System Overview

[0131] 1. User Interface

[0132] User: Accesses the application using a device such as a smartphone or tablet. Users can input their health-related inquiries using text, voice, or images.

[0133] 2. Data Transmission

[0134] Device: The device sends the health information (text data, voice data, image data) entered by the user to the server in real time. The device also sends the user's identification information and timestamp along with the entered information.

[0135] 3. AI-based data analysis

[0136] Server: Inputs the received data into the AI ​​diagnostic module and begins analysis. The AI ​​diagnostic module analyzes the data using machine learning models such as natural language processing models (e.g., BERT), speech analysis models (e.g., WaveNet), and image analysis models (e.g., Inception V3) to perform an initial diagnosis.

[0137] 4. Generate and submit follow-up questions

[0138] Server: Based on the results of the initial diagnosis, generate additional questions as needed. For example, automatically generate a question such as "How long have you had a cough?" and send this question to the device.

[0139] 5. Providing diagnostic results

[0140] Server: After receiving the user's additional answers, it makes a final diagnosis. For example, it generates a diagnosis result and recommended actions such as "You may have a cold. Rest and drink plenty of fluids." and sends them to the device.

[0141] 6. Implement the action plan

[0142] Server: Provides recommended actions and action plans based on the diagnosis results. These action plans include linking with related external services (e.g., medical institution reservation systems, e-commerce sites). The server automatically makes reservations and arrangements as necessary.

[0143] Specific examples

[0144] 1. If the user complains of a cough

[0145] User: Opens the app, types in the text "My cough has been getting worse lately," and submits.

[0146] Terminal: Sends input information to the server.

[0147] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates an additional question such as "When did the coughing symptoms start?" and sends it to the device.

[0148] Terminal: Display the question to the user.

[0149] User: "A week ago."

[0150] Terminal: Sends the answer to the server.

[0151] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids." along with recommended actions, which are sent to the device.

[0152] Device: Display diagnostic results and recommended actions to the user.

[0153] 2. If the user complains of skin abnormalities

[0154] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[0155] Device: Sends photos and text information to the server.

[0156] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[0157] Terminal: Display the question to the user.

[0158] User: "A week ago."

[0159] Terminal: Sends the answer to the server.

[0160] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist," along with information about nearby dermatologists, and sends this to the device.

[0161] Device: Display diagnostic results and recommended actions to the user.

[0162] Examples of prompt statements

[0163] Prompt 1: "I've had a bad cough lately. Could it be a cold?"

[0164] Prompt 2: "I'm worried about these red spots. Could I be allergic to something?"

[0165] In this way, the AI ​​diagnostic system of the present invention allows users to easily receive health consultations and receive prompt and accurate diagnoses and recommendations for action, promoting early detection of disease and preventive medicine, and supporting users' health management.

[0166] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0167] Step 1:

[0168] Accessing the User Interface

[0169] User: Launches the application on a device such as a smartphone or tablet. The user taps a button such as "Start health consultation" to proceed to the information input screen.

[0170] Input: Tap to launch the app.

[0171] Output: A health consultation input screen will be displayed.

[0172] Step 2:

[0173] Entering information

[0174] User: Enter a health question. For example, type "My cough has been getting worse recently" into the text box and press the send button. Or, tap the voice input button and say "My cough has been getting worse recently" to have the system recognize the voice.

[0175] Input: Health consultation information in text, audio, or image format.

[0176] Output: After pressing the send button, a message will be displayed indicating that the sending is complete. The input data will be prepared.

[0177] Step 3:

[0178] Data transmission

[0179] Terminal: The health information entered by the user is sent to the server. The data sent includes the input information, user identification information, and a timestamp.

[0180] Input: Text, voice, or image data entered by the user.

[0181] Data processing and calculation: For text data, data encoding and formatting. For audio data, audio file generation and compression. For image data, image file compression and format conversion.

[0182] Output: Data packet sent to the server, notification message of successful delivery.

[0183] Step 4:

[0184] Data analysis

[0185] Server: The received data is input into the generation AI model diagnostic module and analysis begins. A natural language processing model (e.g., BERT) analyzes the text and extracts keywords related to symptoms. For audio data, a voice analysis model (e.g., WaveNet) converts it into text and performs the same analysis. For image data, an image analysis model (e.g., Inception V3) analyzes the characteristics of abnormal areas.

[0186] Input: Text, audio, and image data sent by the user.

[0187] Data processing and computation: text analysis using natural language processing, speech recognition and text conversion, and image analysis to identify abnormalities.

[0188] Output: Results of the initial diagnosis and information about any symptoms detected.

[0189] Step 5:

[0190] Generate and submit follow-up questions

[0191] Server: Based on the results of the initial diagnosis, generate additional questions as needed. For example, generate a question such as "How long have you had a cough?" and send it to the device.

[0192] Input: Initial diagnostic results.

[0193] Data processing and calculation: Based on the results of the initial diagnosis, an automatic question generation algorithm is applied to generate follow-up questions.

[0194] Output: Text data of follow-up questions.

[0195] Step 6:

[0196] View additional questions and submit answers

[0197] Terminal: Show the follow-up questions to the user. The user answers the follow-up questions and presses the submit button again to send the answers to the server.

[0198] Input: Text data of the follow-up question.

[0199] Output: The user's answer text, and an indication that the answer was submitted.

[0200] Step 7:

[0201] Providing a final diagnosis and recommended actions

[0202] Server: Analyzes the user's additional answers and generates a final diagnosis, such as "You may have a cold. Rest and drink plenty of fluids.", along with recommended actions, and sends them to the device.

[0203] Input: Additional user response data.

[0204] Data processing and calculations: Analyzing additional response data, applying diagnostic models, and generating diagnostic results and recommended actions.

[0205] Output: Text data of the final diagnosis result and recommended actions.

[0206] Step 8:

[0207] Implementing the action plan

[0208] On the device: The diagnostic results and recommended actions are displayed to the user, along with supplemental information, such as nearby medical facilities and, if necessary, a link to make an appointment.

[0209] Server: Works with external services (such as medical institution reservation systems and e-commerce sites) to automatically make reservations and arrangements as needed.

[0210] Input: Final diagnostic results and recommended actions.

[0211] Data processing and calculation: Automatic processing of reservations and arrangements through integration with external service APIs.

[0212] Output: Notification to the user and confirmation of any related bookings or arrangements.

[0213] (Application example 1)

[0214] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0215] Conventional health checkup systems make it difficult for users to quickly check their health status and receive appropriate diagnostic results and recommended actions. In addition, there are problems with the lack of systems that can be easily used on a daily basis and the lack of integrated AI analysis functions, which makes it difficult for users to effectively manage their health.

[0216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0217] In this invention, the server includes a means for a user to input health-related inquiries by text, voice, or image, a means for a terminal to transmit the user-input information to the server, and a means for the server to analyze the received information using an AI diagnostic module and perform an initial diagnosis. This enables users to check their health condition on a daily basis using their smart devices and receive prompt and appropriate diagnostic results and recommended actions.

[0218] A "user interface" is the means by which a user interacts with a system.

[0219] "Smart devices" refer to portable electronic devices such as smartphones and tablets.

[0220] The "AI diagnostic module" is a component that uses artificial intelligence technology to analyze a user's health information and make a diagnosis.

[0221] "Text data" refers to information entered by a user using characters.

[0222] "Voice data" means information that has been digitally recorded from a user's voice and converted into an analyzable format.

[0223] "Image data" refers to visual information such as photographs and illustrations taken by users.

[0224] "Analysis" is the process of processing input data and converting it into meaningful information.

[0225] An "initial diagnosis" is the first health assessment performed by the AI ​​diagnostic module.

[0226] "Additional questions" are questions generated to obtain more detailed information based on the initial diagnosis results.

[0227] The "final diagnosis" is the final assessment of health status derived based on the results of all data analysis.

[0228] "Recommended actions" are specific actions or measures suggested based on the diagnostic results.

[0229] A "prompt" is a textual instruction or question presented to the user to obtain additional information.

[0230] "External services" are services that exist outside the system and are linked to make reservations and arrangements.

[0231] This invention relates to an AI health checkup system that runs on smart devices. This system provides an application that allows users to easily check their daily health status, and presents AI-based analysis and diagnosis results, as well as specific recommended actions.

[0232] System Configuration

[0233] 1. User Interface

[0234] Users access the application using smart devices such as smartphones and tablets, where they can input their health concerns via text, voice, or images.

[0235] 2. Data Transmission

[0236] The device sends the health information entered by the user to a cloud server in real time. This information is sent to the server as text data, audio data, and image data.

[0237] 3. AI-based data analysis

[0238] The server inputs the received data into the AI ​​diagnostic module, which performs natural language processing (NLP), voice analysis, and image analysis to make an initial diagnosis. The hardware used is a high-performance cloud computing instance. The software used is nltk, sklearn, speech_recognition, keras, and tensorflow.

[0239] 4. Generate and submit follow-up questions

[0240] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[0241] 5. Providing diagnostic results

[0242] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results and recommended measures (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[0243] Specific examples

[0244] 1. If the user complains of a cough

[0245] User: Opens the app, texts "My cough has been getting worse lately," and sends it.

[0246] Terminal: Sends input information to the server.

[0247] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates an additional question such as "When did the coughing symptoms start?" and sends it to the device.

[0248] Terminal: Display the question to the user.

[0249] User: "A week ago."

[0250] Terminal: Sends the answer to the server.

[0251] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids" along with recommended actions, which are then sent to the device.

[0252] Device: Display diagnostic results and recommended actions to the user.

[0253] 2. If the user complains of skin abnormalities

[0254] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[0255] Device: Sends photos and text information to the server.

[0256] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[0257] Terminal: Display the question to the user.

[0258] User: "A week ago."

[0259] Terminal: Sends the answer to the server.

[0260] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist" along with information about nearby dermatologists, and sends this to the device.

[0261] Device: Display diagnostic results and recommended actions to the user.

[0262] An example of a prompt sentence in this example is "Please diagnose the following symptoms: 'My cough has been getting worse recently.'" Through these specific examples, the AI ​​health diagnosis system of the present invention becomes a system that enables users to conduct daily health consultations and receive quick and accurate diagnoses and recommendations for action.

[0263] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0264] Step 1:

[0265] The user inputs their health-related question via text, voice, or image. The user opens the application on their smart device and inputs their health-related question via text, voice recording using a microphone, or taking an image with a camera and inputting it into the application. This generates user-acquired data.

[0266] Step 2:

[0267] The device sends user input information to the server. The device then sends the acquired text data, voice data, or image data to the cloud server in real time. The server then receives the user input data.

[0268] Step 3:

[0269] The server analyzes the received information using an AI diagnostic module and performs an initial diagnosis. Based on the received data, the server performs an initial diagnosis using natural language processing (nltk, sklearn) for text data, speech analysis (speech_recognition) for audio data, and image analysis (keras and tensorflow) for image data. The server outputs the initial diagnosis results based on the generated diagnosis results.

[0270] Step 4:

[0271] The server generates follow-up questions and sends them to the user via the device. Based on the initial diagnosis results, the server generates follow-up questions to obtain more detailed information. The generative AI model is used to create a prompt sentence and send it to the device, which then displays the follow-up questions to the user.

[0272] Step 5:

[0273] Receive the user's additional answers. The user answers the additional questions displayed on the terminal and enters the answer data into the terminal. The terminal again sends this data to the server, which then receives the additional answer data.

[0274] Step 6:

[0275] The server provides a final diagnosis result based on the additional answers. The server re-inputs the additional answer data into the AI ​​diagnosis module for final analysis. Along with the final diagnosis result, it generates recommended actions for the user and sends them to the device. The device then displays the final diagnosis result to the user.

[0276] Step 7:

[0277] Based on the diagnosis results, the server connects with external services to make the necessary reservations and arrangements. Based on the final diagnosis results, the server generates a detailed action plan to support the user's actions, and connects with external services (such as medical institutions or taxi services) to automatically make the necessary reservations and arrangements. This information is sent to the terminal and notified to the user.

[0278] This series of processing steps allows users to receive a quick and accurate diagnosis and appropriate recommended actions.

[0279] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0280] This invention is an AI-based health checkup system in which users input their health-related inquiries via text, voice, or images, and the system provides diagnostic results and recommended actions after analysis by the server. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more precise diagnoses and responses can be achieved. This system enables early detection and prevention of disease, efficiently supporting the user's health management.

[0281] System Overview

[0282] 1. User Interface

[0283] Users access the application using a device (e.g., a smartphone or tablet) and input their health-related concerns using text, voice, or images. The input is analyzed through an emotion engine, which also determines the user's emotional state.

[0284] 2. Data Transmission

[0285] The device transmits the health information entered by the user and the emotional data analyzed by the emotion engine to the server in real time in the form of text data, voice data, image data, and emotional data.

[0286] 3. AI-based data analysis

[0287] The server inputs the received health information and emotional data into the AI ​​diagnostic module, which performs natural language processing, voice analysis, and image analysis to make an initial diagnosis. It also takes into account the emotional data and reflects it in its diagnosis and recommended actions.

[0288] 4. Generate and submit follow-up questions

[0289] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[0290] 5. Providing diagnostic results

[0291] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results along with recommended measures and actions (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[0292] 6. Implement the action plan

[0293] Based on the diagnosis results, the server provides an action plan to support the user's actions. This includes linking with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as necessary. This information is notified to the user via their device, allowing them to take actual action based on the information provided.

[0294] Specific examples

[0295] 1. If the user complains of stress and coughing symptoms

[0296] User: Opens the app, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[0297] Device: Sends input information and emotional data (how stressed the user is) to the server.

[0298] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[0299] Terminal: Display the question to the user.

[0300] User: "A week ago. Work pressure."

[0301] Terminal: Sends the answer to the server.

[0302] Server: Analyzes the responses and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest, drink plenty of fluids, and try relaxation techniques," and sends these to the device.

[0303] On the device: Display diagnostic results and recommended actions to the user.

[0304] 2. When users complain of skin abnormalities and concerns

[0305] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots and they're making me anxious."

[0306] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[0307] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[0308] Terminal: Display the question to the user.

[0309] User: "A week ago. I've been feeling unwell lately and I'm worried."

[0310] Terminal: Sends the answer to the server.

[0311] Server: Based on the answers and emotion data, the server generates a diagnosis and recommended actions, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends these to the device.

[0312] On the device: Display diagnostic results and recommended actions to the user.

[0313] In this way, by analyzing and responding to the user's emotional state, the system can provide more personalized and appropriate diagnoses and recommended actions, enabling users to take prompt and accurate measures to address health issues and improving the quality of their overall health management.

[0314] The processing flow will be explained below.

[0315] Step 1:

[0316] User: Installs the application, enters personal information (name, age, gender, medical history, etc.) to create an account, and sets authentication information for future logins.

[0317] Step 2:

[0318] Device: Sends personal information and authentication information to a server and requests the creation of a user account.

[0319] Step 3:

[0320] Server: Stores the received personal information in a database, generates an authentication token, and sends it to the device. The user can then use this authentication token to log in from the next time onwards.

[0321] Step 4:

[0322] User: Launches the application and enters their health consultation via text, voice, or image. Once completed, they press the send button.

[0323] Step 5:

[0324] Terminal: Receives health information entered by the user in real time and sends it to the server. The information sent can be text data, voice data, or image data. It also uses an emotion engine to analyze emotional data from the user's input and simultaneously sends it to the server.

[0325] Step 6:

[0326] Server: The received health information and emotional data is input into the AI ​​diagnostic module, which performs text, voice, or image analysis. Based on the analysis results, an initial diagnosis is made and possible illnesses and symptoms are identified. Emotional data is also taken into account to generate an appropriate diagnosis and recommended actions.

[0327] Step 7:

[0328] Server: Based on the initial diagnosis and emotion data, it generates additional questions to gather more detailed information. For example, it generates questions such as, "How long has your cough been bad?" It also generates emotion-based questions such as, "Have you been feeling stressed lately?" depending on the emotional state.

[0329] Step 8:

[0330] Server: Sends the generated follow-up question to the terminal.

[0331] Step 9:

[0332] Terminal: Displays the question received from the server to the user.

[0333] Step 10:

[0334] User: Answer the questions displayed, enter text such as "It's been a week since" and submit.

[0335] Step 11:

[0336] Terminal: Sends the user's additional answers to the server.

[0337] Step 12:

[0338] Server: The received additional answers are input again into the AI ​​diagnostic module, analyzed, and a final diagnostic result is generated. The emotional data is also evaluated again and applied to the diagnostic result.

[0339] Step 13:

[0340] Server: Generates the final diagnosis and recommended actions, and sends them to the device, including, for example, "You may have a cold. Rest, drink plenty of fluids, and adopt relaxation techniques."

[0341] Step 14:

[0342] On the device: The diagnostic results and recommended actions from the server are displayed to the user.

[0343] Step 15:

[0344] Server: Based on the diagnostic results, a detailed action plan is generated to support the user's actions. This includes connecting with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as needed.

[0345] Step 16:

[0346] Device: Notifies the user of the action plan from the server, making it easier for the user to act according to the plan.

[0347] Step 17:

[0348] User: Review the action plan displayed on the device and take specific action as needed using the links and information provided, for example, making a doctor's appointment or ordering a taxi.

[0349] This processing step allows users to easily consult about their health, and the AI ​​system and emotion engine provide quick and accurate diagnoses and recommended actions, enabling early diagnosis and preventive medicine.

[0350] Example 2

[0351] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0352] Conventional health checkup systems only provided general diagnostic results based on the health information entered by the user, making it difficult to perform a detailed diagnosis that took into account the emotional state of each individual user. Furthermore, due to a lack of means to properly analyze and respond to emotional data such as stress and anxiety felt by users, more personalized measures were rarely provided. As a result, early disease detection and prevention, as well as the quality of health management, were not sufficiently improved, making it difficult to efficiently support users' health.

[0353] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to input health-related inquiries using text, voice, or images; a means for a terminal to transmit the user-input information and emotional data to the server; a means for the server to analyze the health information and emotional data received by the server using an AI diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and emotional data and providing a final diagnosis result; a means for providing recommended behaviors and an action plan based on the diagnosis result; and a means for coordinating with external services to make necessary reservations and arrangements. This enables a personalized and precise diagnosis that takes the user's emotional state into consideration, thereby providing more appropriate and prompt measures.

[0354] A "user" is an individual who uses the health checkup system and inputs health-related inquiries.

[0355] A "terminal" is an information and communication device that allows users to access the health checkup system and input and display consultation details and emotional data.

[0356] A "server" is a computing device that receives, analyzes, and diagnoses information sent by users.

[0357] The "AI diagnostic module" is an artificial intelligence program installed on the server that analyzes received health information and emotional data and performs an initial diagnosis.

[0358] An "emotion engine" is software that analyzes the emotional state from user input and generates that data.

[0359] "Health Information" means information in text, audio, or image format that a user inputs into the health examination system.

[0360] "Emotion data" is data that indicates the user's emotional state analyzed through the emotion engine.

[0361] An "initial diagnosis" is the first diagnosis that the AI ​​diagnostic module performs based on the health information and emotional data it receives.

[0362] "Additional questions" are questions that the server generates based on the results of the initial diagnosis, to obtain more detailed information from the user.

[0363] The "final diagnosis result" is a definitive diagnosis made by the server based on the additional answers and emotional data from the user.

[0364] "Recommended actions" are specific measures or suggestions for action for the user that are generated by the server based on the final diagnostic results.

[0365] An "action plan" is a specific plan of action that the user should take in accordance with the recommended action.

[0366] "External services" are external businesses or systems that work in conjunction with the health checkup system to make necessary reservations and arrangements.

[0367] This invention relates to a health checkup system using AI. Users input their health-related inquiries via text, voice, or images, and the system provides diagnostic results and recommended actions after analysis by a server. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more precise diagnoses and responses can be achieved. This system enables early detection and prevention of disease, efficiently supporting users' health management.

[0368] First, a user accesses the application using a device such as a smartphone or tablet and inputs their health-related inquiry using text, voice, or images. The input information is passed to an emotion engine by the device, which analyzes the user's emotional state. This emotion engine is software that extracts emotions from the input data (text, voice, images) and generates emotion data.

[0369] The device transmits the health information and emotional data entered by the user to the server in real time. This data is passed to the server in the form of text data, voice data, image data, and emotional data.

[0370] The server inputs the received health information and emotional data into the AI ​​diagnostic module. This AI diagnostic module is an artificial intelligence program that performs natural language processing (NLP), voice analysis, and image analysis, and has the function of conducting an initial diagnosis. Specifically, it uses NLP technology to analyze text data. It also applies each analysis technology to voice data and image data to perform an initial diagnosis of the user's health condition.

[0371] The server generates additional questions based on the initial diagnosis results and sends them to the user via the device, allowing for more detailed information to be obtained and a more precise diagnosis to be performed.The user answers the additional questions via the device, and the answers are sent back to the server.

[0372] The server analyzes the user's additional responses and emotional data to provide a final diagnosis. Along with the diagnosis, it generates recommended measures and actions (e.g., treatment, referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis and recommended actions through the device.

[0373] Furthermore, the server provides an action plan based on the diagnosis results. This plan is designed to support the user's actions and automatically makes necessary reservations and arrangements by connecting with external services (e.g., e-commerce sites, help apps, taxi services). This information is then sent to the user via their device, allowing them to take action accordingly.

[0374] Specific examples

[0375] 1. If the user complains of stress and coughing symptoms

[0376] User: Opens the app, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[0377] Device: Sends input information and emotional data (how stressed the user is) to the server.

[0378] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[0379] Terminal: Display the question to the user.

[0380] User: "A week ago. Work pressure."

[0381] Terminal: Sends the answer to the server.

[0382] Server: Analyzes the responses and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest, drink plenty of fluids, and try relaxation techniques," and sends these to the device.

[0383] On the device: Display diagnostic results and recommended actions to the user.

[0384] 2. When users complain of skin abnormalities and concerns

[0385] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots and they're making me anxious."

[0386] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[0387] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[0388] Terminal: Display the question to the user.

[0389] User: "A week ago. I've been feeling unwell lately and I'm worried."

[0390] Terminal: Sends the answer to the server.

[0391] Server: Based on the answers and emotion data, the server generates a diagnosis and recommended actions, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends these to the device.

[0392] On the device: Display diagnostic results and recommended actions to the user.

[0393] In this way, by analyzing and responding to a user's emotional state, it is possible to provide more personalized and relevant diagnoses and recommended actions, allowing users to take quicker and more accurate measures to address health issues and improve the quality of their overall health management.

[0394] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0395] Step 1:

[0396] The user accesses the application using a device such as a smartphone or tablet. The user inputs a health-related question in the form of text, voice, or image. For example, the user might input "My cough has gotten worse recently, and I'm feeling stressed." This input serves as the initial data for generating the question content and emotion data.

[0397] Step 2:

[0398] The device passes the health information entered by the user to the emotion engine. The emotion engine analyzes the user's emotional state from the input text, voice, or image data. The analysis generates emotion data, such as "the user is feeling stressed." This emotion data and health information become input data for subsequent data processing.

[0399] Step 3:

[0400] The device sends the emotional data and health information generated by the emotion engine to the server. The data sent includes text data, voice data, image data, and emotional data. The server receives this data and analyzes it in the next step.

[0401] Step 4:

[0402] The server inputs the received health information and emotional data into the AI ​​diagnostic module. This AI diagnostic module performs natural language processing (NLP), voice analysis, and image analysis to make an initial diagnosis. For example, it uses NLP technology to analyze text data and generate follow-up questions such as, "When did your coughing symptoms start?" Here, the initial diagnosis result and any necessary follow-up questions are generated.

[0403] Step 5:

[0404] The server sends the generated follow-up questions to the terminal. The terminal displays these follow-up questions to the user in real time. Based on the user's input, the terminal obtains more detailed information and prepares for a more precise diagnosis. For example, the user might be asked, "When did your coughing symptoms begin?"

[0405] Step 6:

[0406] The user answers additional questions through the device. For example, they might say, "It's been a week. It's because of work pressure." The user's answers are sent to the server through the device. These answers become data for further detailed diagnosis.

[0407] Step 7:

[0408] The server then inputs the user's additional responses and emotional data back into the AI ​​diagnostic module to make a final diagnosis. The AI ​​diagnostic module then analyzes the new data it has received and generates a final diagnosis and recommended actions. For example, it generates a diagnosis and recommended actions such as, "You may have a cold. Please rest, drink plenty of fluids, and adopt relaxation techniques."

[0409] Step 8:

[0410] The server sends the final diagnosis results and recommended actions to the device, which then displays them to the user. The user can check the diagnosis results and take necessary measures.

[0411] Step 9:

[0412] The server provides an action plan based on the diagnosis results. The device then connects with external e-commerce sites, help apps, taxi services, etc., and automatically makes the necessary reservations and arrangements. This information is then sent to the user, who can then take action based on the information provided.

[0413] Through the above steps, users can receive detailed health checkups and appropriate measures in real time, thereby improving the quality of their overall health management.

[0414] (Application example 2)

[0415] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0416] In modern society, there are a wide variety of health-related issues, and users need support to respond quickly and appropriately. However, there is a lack of diagnostic systems that take into account the user's emotional state, and systems that efficiently support health consultations in physical stores. This limits the means by which users can access appropriate medical institutions and products. Therefore, there is a need for a system that can respond to health consultations in physical stores while taking into account the user's health state and emotions.

[0417] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input health-related inquiries using text, voice, or images; a means for a terminal to transmit the user-input information and emotional data to the server; a means for the server to analyze the received information and emotional data using an AI diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis result; a means for providing recommended behaviors and an action plan based on the diagnosis result; a means for coordinating with external services to make necessary reservations and arrangements; and a means having a function for supporting the user's behavior in the store. This enables a quick and appropriate diagnosis and recommended actions to be provided taking into account the user's emotional state, thereby efficiently supporting health consultations in physical stores.

[0418] "User" means a person who inputs health-related questions or symptoms via text, voice, or image.

[0419] A "terminal" is a device that transmits information entered by a user to a server.

[0420] "Server" refers to a computer system that analyzes received information and emotion data and performs diagnosis.

[0421] An "AI diagnostic module" is an artificial intelligence program that analyzes received data and generates an initial diagnosis or follow-up questions.

[0422] "Emotional data" refers to information obtained by analyzing a user's emotional state.

[0423] "Additional questions" are questions generated based on the results of the initial diagnosis to obtain more detailed information.

[0424] "Diagnosis Result" refers to the final health assessment and recommended actions generated by the AI ​​diagnostic module.

[0425] "Recommended actions" are actions or measures suggested to users based on the diagnostic results.

[0426] An "action plan" is a specific plan provided to support the user's actions based on the diagnostic results.

[0427] "External services" refers to medical institutions, reservation services, product sales services, etc. that are linked based on the diagnosis results.

[0428] "Functions to support user behavior within the store" refers to functions that enable users to efficiently receive health consultations and purchase recommended products within the store.

[0429] MODE FOR CARRYING OUT THE INVENTION

[0430] The present invention is a system that provides prompt and appropriate responses when users seek health consultations at physical stores by combining AI diagnosis and emotion analysis. Detailed embodiments of this system are described below.

[0431] System Configuration

[0432] The system includes a means for users to input health-related inquiries via text, voice, or image; a means for the terminal to send the user's input information to a server; a means for the server to analyze the received information and emotional data; a means for the server to generate a diagnosis result and additional questions and send them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis result; a means for providing recommended behaviors and action plans based on the diagnosis result; a means for connecting with external services to make necessary reservations and arrangements; and a function for supporting the user's behavior within the store.

[0433] What the program does

[0434] The program of this system performs the following processing.

[0435] 1. Data capture:

[0436] The device captures health-related information (text, voice, and images) entered by the user. The hardware used can be a smartphone, smart glasses, or a head-mounted display. Voice analysis software is used for voice recognition, and OpenCV is used for image recognition.

[0437] 2. Emotion analysis:

[0438] Extracts emotion data from user input data using a pre-trained EmotionRecognizer model for emotion analysis.

[0439] 3. Data transmission:

[0440] The device transmits the captured data and emotion data to a server, which allows for real-time data analysis.

[0441] 4. AI diagnosis:

[0442] The server inputs the received data into the AI ​​diagnostic module (HealthDiagnosisAI) to perform an initial diagnosis, generates diagnostic results and follow-up questions, and sends them to the user.

[0443] 5. Additional Questions and Answers:

[0444] The server receives the user's answers and analyzes them again, generating additional questions as needed to refine the diagnosis.

[0445] 6. Final diagnosis findings and recommended actions:

[0446] The server generates a final diagnosis and provides the user with a recommended course of action based on the diagnosis, which may include purchasing a product or referring the patient to a specialist.

[0447] 7. Integration with external services:

[0448] Based on the diagnosis results, the server will connect with external medical institutions and services and automatically make the necessary reservations and arrangements.

[0449] 8. In-store support:

[0450] The terminal guides and supports users so that they can efficiently receive health consultations and purchase recommended products in the store.

[0451] Specific examples

[0452] Example 1: User complains of stress and coughing

[0453] User: Opens the app on their smartphone, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[0454] Terminal: Sends input information and emotional data (how stressed the user is) to the server.

[0455] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[0456] Terminal: Display the question to the user.

[0457] User: "A week ago. Work pressure."

[0458] Terminal: Sends the answer to the server.

[0459] Server: Analyzes the answers and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest and drink plenty of fluids. Also, try relaxation techniques," and sends these to the device.

[0460] Device: Display diagnostic results and recommended actions to the user.

[0461] Example 2: When a user complains of skin abnormalities and concerns

[0462] User: Takes a photo of a skin abnormality and types, "These red spots bother me and make me anxious."

[0463] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[0464] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[0465] Terminal: Display the question to the user.

[0466] User: "It's been a week. I've been feeling unwell lately and I'm worried."

[0467] Terminal: Sends the answer to the server.

[0468] Server: Based on the response and emotional data, the server generates a diagnosis and recommended action, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends this to the device.

[0469] Device: Display diagnostic results and recommended actions to the user.

[0470] Prompt Sentence Examples

[0471] "I've been having a bad cough lately. What product would you recommend?"

[0472] "Please give me some advice about this skin condition."

[0473] In this way, the system of the present invention provides prompt and appropriate support for health consultations in physical stores while taking into account the user's emotional state.

[0474] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0475] Step 1:

[0476] The user uses a device (such as a smartphone, smart glasses, or head-mounted display) to input health-related questions and symptoms. The input can be in the form of text, voice, or images, and this data is captured on the device. The device then stores this data as initial data.

[0477] Input: Text, voice, and image data related to the user's health.

[0478] Output: Initial data file on the terminal

[0479] Step 2:

[0480] The initial data captured by the device is collected and emotion analysis is performed. The EmotionRecognizer model is used to determine the user's emotional state (e.g., stress, anxiety, etc.) from the text and voice data. The analysis results are stored as emotion data.

[0481] Input: Initial data file

[0482] Output: Emotion data

[0483] Step 3:

[0484] The device transmits the health information and emotion data entered by the user to the server in real time, and the data format includes text data, voice data, image data, and emotion data.

[0485] Input: Health information, emotional data

[0486] Output: Data sent to the server

[0487] Step 4:

[0488] The server inputs the received data into the AI ​​diagnostic module (HealthDiagnosisAI) for an initial diagnosis. The AI ​​diagnostic module analyzes health information through natural language processing, voice analysis, and image analysis to generate an initial diagnosis.

[0489] Input: Health information, emotional data

[0490] Output: Initial diagnosis result

[0491] Step 5:

[0492] The server generates additional questions based on the initial diagnosis results. The additional questions are intended to obtain additional data necessary to improve the accuracy of the diagnosis. The generated questions are then sent to the device.

[0493] Input: Initial diagnosis result

[0494] Output: Additional questions

[0495] Step 6:

[0496] The device displays the received follow-up questions to the user and collects follow-up answers from the user. The user inputs the answers through the device, and the answers are again saved on the device.

[0497] Input: Additional Question

[0498] Output: User's additional answers

[0499] Step 7:

[0500] The device sends the additional answers it collects to the server, which then analyzes the data again using the AI ​​diagnostic module, resulting in a more accurate final diagnosis.

[0501] Input: Additional Answer

[0502] Output: Final diagnosis result

[0503] Step 8:

[0504] The server generates a recommended course of action or action plan based on the final diagnosis results. The diagnosis results may include, for example, referrals to nearby medical institutions or a list of recommended products. The generated action plan is then sent to the device.

[0505] Input: Final diagnosis result

[0506] Output: Recommended actions, action plan

[0507] Step 9:

[0508] The device will display the recommended actions and action plans received to the user, providing the user with information to take specific actions.

[0509] Input: Recommended Action, Action Plan

[0510] Output: What is displayed to the user

[0511] Step 10:

[0512] Based on the diagnosis results, the server connects with external services and automatically makes necessary reservations and arrangements, such as making appointments at medical institutions or ordering products. This information is then sent to the user via their device.

[0513] Input: Diagnostic results, recommended actions

[0514] Output: Linkage with external services, notification of reservation / arrangement completion

[0515] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0516] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0517] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0518] [Second embodiment]

[0519] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0520] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0521] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0522] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0523] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0524] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0525] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0526] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0527] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0528] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0529] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0530] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0531] This invention relates to a health checkup system using AI, in which users input their health-related inquiries using text, voice, or images, which are analyzed by a server, and diagnostic results and recommended actions are provided. This enables early detection and prevention of diseases, and supports the user's health management.

[0532] System Overview

[0533] 1. User Interface

[0534] Users access the application using a device (e.g., a smartphone or tablet) and enter their health consultation information, which can be entered either as text, voice, or images.

[0535] 2. Data Transmission

[0536] The device transmits the health information entered by the user to the server in real time. This information is sent to the server as text data, voice data, and image data.

[0537] 3. AI-based data analysis

[0538] The server inputs the received data into the AI ​​diagnostic module, which performs natural language processing, voice analysis, and image analysis to make an initial diagnosis. The AI ​​diagnostic module then refers to past data and a medical knowledge base to suggest possible illnesses and treatments.

[0539] 4. Generate and submit follow-up questions

[0540] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[0541] 5. Providing diagnostic results

[0542] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results and recommended measures (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[0543] 6. Implement the action plan

[0544] Based on the diagnosis results, the server provides recommended actions and action plans. This includes linking with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as needed. This information is then sent to the user via their device, allowing them to take action based on the information provided.

[0545] Specific examples

[0546] 1. If the user complains of a cough

[0547] User: Opens the app, types in the text "My cough has been getting worse lately," and submits.

[0548] Terminal: Sends input information to the server.

[0549] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates an additional question such as "When did the coughing symptoms start?" and sends it to the device.

[0550] Terminal: Display the question to the user.

[0551] User: "A week ago."

[0552] Terminal: Sends the answer to the server.

[0553] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids" along with recommended actions, which are then sent to the device.

[0554] On the device: Display diagnostic results and recommended actions to the user.

[0555] 2. If the user complains of skin abnormalities

[0556] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[0557] Device: Sends photos and text information to the server.

[0558] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[0559] Terminal: Display the question to the user.

[0560] User: "A week ago."

[0561] Terminal: Sends the answer to the server.

[0562] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist" along with information about nearby dermatologists, and sends this to the device.

[0563] On the device: Display diagnostic results and recommended actions to the user.

[0564] In this way, the AI ​​diagnostic system of the present invention allows users to easily receive health consultations and receive prompt and accurate diagnoses and recommended actions. It is a system that promotes early detection of diseases and preventive medicine, and supports users' health management.

[0565] The processing flow will be explained below.

[0566] Step 1:

[0567] User: Installs the application, enters personal information (name, age, gender, medical history, etc.) to create an account, and sets authentication information for future logins.

[0568] Step 2:

[0569] Device: Sends personal information and authentication information to a server and requests the creation of a user account.

[0570] Step 3:

[0571] Server: Stores the received personal information in a database, generates an authentication token, and sends it to the device. This allows the user to log in using the authentication token from the next time onwards.

[0572] Step 4:

[0573] User: Launches the application and enters their health consultation via text, voice, or image. Once completed, they press the send button.

[0574] Step 5:

[0575] Terminal: Receives health information entered by the user in real time and sends it to the server as text, audio, or image data.

[0576] Step 6:

[0577] Server: The received health information is input into the AI ​​diagnostic module, which performs text, voice, or image analysis. Based on the analysis results, an initial diagnosis is made and possible illnesses and symptoms are identified.

[0578] Step 7:

[0579] Server: Based on the results of the initial diagnosis, generate additional questions to gather more detailed information, such as "How long has your cough been bad?"

[0580] Step 8:

[0581] Server: Sends the generated follow-up question to the terminal.

[0582] Step 9:

[0583] Terminal: Displays the question received from the server to the user.

[0584] Step 10:

[0585] User: Answer the questions displayed, enter text such as "It's been a week since" and submit.

[0586] Step 11:

[0587] Terminal: Sends the user's additional answers to the server.

[0588] Step 12:

[0589] Server: The additional answers received are input into the AI ​​diagnostic module again, where they are analyzed and a final diagnostic result is generated.

[0590] Step 13:

[0591] Server: Generates and sends the final diagnosis and recommended measures, such as "You may have a cold. Rest and drink plenty of fluids." to the device.

[0592] Step 14:

[0593] On the device: The diagnostic results and recommended actions from the server are displayed to the user.

[0594] Step 15:

[0595] Server: Based on the diagnosis results, generate an action plan to support the user, including, for example, information on the nearest pharmacy, a link to make an appointment with a medical institution, or information on how to arrange a taxi.

[0596] Step 16:

[0597] Device: Notifies the user of the action plan from the server, making it easier for the user to act according to the plan.

[0598] Step 17:

[0599] User: Review the action plan displayed on the device and take specific action as needed using the links and information provided, for example, making a doctor's appointment or ordering a taxi.

[0600] This processing step allows users to easily consult about their health, and the AI ​​system quickly and accurately provides a diagnosis and recommended actions, enabling early diagnosis and preventive medical care.

[0601] Example 1

[0602] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0603] In modern society, it is difficult to obtain prompt and appropriate advice for daily health management or minor health consultations. Receiving expert advice at the appropriate time is especially important for early detection and prevention of illness. However, the time, effort, and cost involved in visiting a medical institution often make people hesitant to do so. Therefore, there is a need for a system that is easy for users to use and can provide accurate diagnoses and effective action plans.

[0604] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0605] In this invention, the server includes: a means for a user to input health-related inquiries via text, voice, or image; a means for a terminal to transmit the user-input information to the server; a means for the server to analyze the received information using a generative AI model diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis; a means for providing recommended behaviors and action plans based on the diagnosis results; and a means for linking with external services to make necessary reservations and arrangements. This allows users to easily receive health consultations via devices such as smartphones and tablets, and receive prompt and appropriate diagnoses and recommended actions. Linking with external services also makes it easy to make additional reservations and arrangements, and supports access to actual medical institutions.

[0606] A "user" refers to a person who provides health consultation and uses the system via a device such as a smartphone or tablet.

[0607] "Health consultation" refers to providing information in text, audio, or image format about physical or mental abnormalities or concerns that a user is experiencing.

[0608] "Text" refers to information expressed by characters entered by a user.

[0609] "Voice" refers to information expressed by audible input from a user.

[0610] "Images" refers to photographs and image files taken by users using a camera or other device.

[0611] "Device" refers to electronic devices that users can use, such as smartphones, tablets, and personal computers.

[0612] "Server" refers to the central processing unit that manages the entire system and receives and analyzes information from users.

[0613] A "generative AI model diagnostic module" refers to a software module that uses machine learning technology to analyze data from users and diagnose their health condition based on the results.

[0614] "Initial diagnosis" refers to a provisional diagnosis based on initial input from the user.

[0615] "Additional questions" refer to additional information that needs to be asked of the user in order to make a more accurate diagnosis based on the results of the initial diagnosis.

[0616] "Final Diagnosis" refers to the final medical diagnosis based on all input information from the User.

[0617] "Recommended actions and action plans" refer to specific measures and next steps suggested to users based on the diagnostic results.

[0618] "External services" refers to external resources such as medical institutions, reservation systems, and e-commerce sites that are used by systems to work together and increase user convenience.

[0619] "Reservations and arrangements" refers to using external services to make reservations at medical institutions and arrange for necessary items.

[0620] This invention relates to a health checkup system using AI, in which users input their health-related inquiries using text, voice, or images, which are analyzed by a server, and diagnostic results and recommended actions are provided. This enables early detection and prevention of diseases, and supports the user's health management.

[0621] System Overview

[0622] 1. User Interface

[0623] User: Accesses the application using a device such as a smartphone or tablet. Users can input their health-related inquiries using text, voice, or images.

[0624] 2. Data Transmission

[0625] Device: The device sends the health information (text data, voice data, image data) entered by the user to the server in real time. The device also sends the user's identification information and timestamp along with the entered information.

[0626] 3. AI-based data analysis

[0627] Server: Inputs the received data into the AI ​​diagnostic module and begins analysis. The AI ​​diagnostic module analyzes the data using machine learning models such as natural language processing models (e.g., BERT), speech analysis models (e.g., WaveNet), and image analysis models (e.g., Inception V3) to perform an initial diagnosis.

[0628] 4. Generate and submit follow-up questions

[0629] Server: Based on the results of the initial diagnosis, generate additional questions as needed. For example, automatically generate a question such as "How long have you had a cough?" and send this question to the device.

[0630] 5. Providing diagnostic results

[0631] Server: After receiving the user's additional answers, it makes a final diagnosis. For example, it generates a diagnosis result and recommended actions such as "You may have a cold. Rest and drink plenty of fluids." and sends them to the device.

[0632] 6. Implement the action plan

[0633] Server: Provides recommended actions and action plans based on the diagnosis results. These action plans include linking with related external services (e.g., medical institution reservation systems, e-commerce sites). The server automatically makes reservations and arrangements as necessary.

[0634] Specific examples

[0635] 1. If the user complains of a cough

[0636] User: Opens the app, types in the text "My cough has been getting worse lately," and submits.

[0637] Terminal: Sends input information to the server.

[0638] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates an additional question such as "When did the coughing symptoms start?" and sends it to the device.

[0639] Terminal: Display the question to the user.

[0640] User: "A week ago."

[0641] Terminal: Sends the answer to the server.

[0642] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids." along with recommended actions, which are sent to the device.

[0643] Device: Display diagnostic results and recommended actions to the user.

[0644] 2. If the user complains of skin abnormalities

[0645] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[0646] Device: Sends photos and text information to the server.

[0647] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[0648] Terminal: Display the question to the user.

[0649] User: "A week ago."

[0650] Terminal: Sends the answer to the server.

[0651] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist," along with information about nearby dermatologists, and sends this to the device.

[0652] Device: Display diagnostic results and recommended actions to the user.

[0653] Examples of prompt statements

[0654] Prompt 1: "I've had a bad cough lately. Could it be a cold?"

[0655] Prompt 2: "I'm worried about these red spots. Could I be allergic to something?"

[0656] In this way, the AI ​​diagnostic system of the present invention allows users to easily receive health consultations and receive prompt and accurate diagnoses and recommendations for action, promoting early detection of disease and preventive medicine, and supporting users' health management.

[0657] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0658] Step 1:

[0659] Accessing the User Interface

[0660] User: Launches the application on a device such as a smartphone or tablet. The user taps a button such as "Start health consultation" to proceed to the information input screen.

[0661] Input: Tap to launch the app.

[0662] Output: A health consultation input screen will be displayed.

[0663] Step 2:

[0664] Entering information

[0665] User: Enter a health question. For example, type "My cough has been getting worse recently" into the text box and press the send button. Or, tap the voice input button and say "My cough has been getting worse recently" to have the system recognize the voice.

[0666] Input: Health consultation information in text, audio, or image format.

[0667] Output: After pressing the send button, a message will be displayed indicating that the sending is complete. The input data will be prepared.

[0668] Step 3:

[0669] Data transmission

[0670] Terminal: The health information entered by the user is sent to the server. The data sent includes the input information, user identification information, and a timestamp.

[0671] Input: Text, voice, or image data entered by the user.

[0672] Data processing and calculation: For text data, data encoding and formatting. For audio data, audio file generation and compression. For image data, image file compression and format conversion.

[0673] Output: Data packet sent to the server, notification message of successful delivery.

[0674] Step 4:

[0675] Data analysis

[0676] Server: The received data is input into the generation AI model diagnostic module and analysis begins. A natural language processing model (e.g., BERT) analyzes the text and extracts keywords related to symptoms. For audio data, a voice analysis model (e.g., WaveNet) converts it into text and performs the same analysis. For image data, an image analysis model (e.g., Inception V3) analyzes the characteristics of abnormal areas.

[0677] Input: Text, audio, and image data sent by the user.

[0678] Data processing and computation: text analysis using natural language processing, speech recognition and text conversion, and image analysis to identify abnormalities.

[0679] Output: Results of the initial diagnosis and information about any symptoms detected.

[0680] Step 5:

[0681] Generate and submit follow-up questions

[0682] Server: Based on the results of the initial diagnosis, generate additional questions as needed. For example, generate a question such as "How long have you had a cough?" and send it to the device.

[0683] Input: Initial diagnostic results.

[0684] Data processing and calculation: Based on the results of the initial diagnosis, an automatic question generation algorithm is applied to generate follow-up questions.

[0685] Output: Text data of follow-up questions.

[0686] Step 6:

[0687] View additional questions and submit answers

[0688] Terminal: Show the follow-up questions to the user. The user answers the follow-up questions and presses the submit button again to send the answers to the server.

[0689] Input: Text data of the follow-up question.

[0690] Output: The user's answer text, and an indication that the answer was submitted.

[0691] Step 7:

[0692] Providing a final diagnosis and recommended actions

[0693] Server: Analyzes the user's additional answers and generates a final diagnosis, such as "You may have a cold. Rest and drink plenty of fluids.", along with recommended actions, and sends them to the device.

[0694] Input: Additional user response data.

[0695] Data processing and calculations: Analyzing additional response data, applying diagnostic models, and generating diagnostic results and recommended actions.

[0696] Output: Text data of the final diagnosis result and recommended actions.

[0697] Step 8:

[0698] Implementing the action plan

[0699] On the device: The diagnostic results and recommended actions are displayed to the user, along with supplemental information, such as nearby medical facilities and, if necessary, a link to make an appointment.

[0700] Server: Works with external services (such as medical institution reservation systems and e-commerce sites) to automatically make reservations and arrangements as needed.

[0701] Input: Final diagnostic results and recommended actions.

[0702] Data processing and calculation: Automatic processing of reservations and arrangements through integration with external service APIs.

[0703] Output: Notification to the user and confirmation of any related bookings or arrangements.

[0704] (Application example 1)

[0705] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0706] Conventional health checkup systems make it difficult for users to quickly check their health status and receive appropriate diagnostic results and recommended actions. In addition, there are problems with the lack of systems that can be easily used on a daily basis and the lack of integrated AI analysis functions, which makes it difficult for users to effectively manage their health.

[0707] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0708] In this invention, the server includes a means for a user to input health-related inquiries by text, voice, or image, a means for a terminal to transmit the user-input information to the server, and a means for the server to analyze the received information using an AI diagnostic module and perform an initial diagnosis. This enables users to check their health condition on a daily basis using their smart devices and receive prompt and appropriate diagnostic results and recommended actions.

[0709] A "user interface" is the means by which a user interacts with a system.

[0710] "Smart devices" refer to portable electronic devices such as smartphones and tablets.

[0711] The "AI diagnostic module" is a component that uses artificial intelligence technology to analyze a user's health information and make a diagnosis.

[0712] "Text data" refers to information entered by a user using characters.

[0713] "Voice data" means information that has been digitally recorded from a user's voice and converted into an analyzable format.

[0714] "Image data" refers to visual information such as photographs and illustrations taken by users.

[0715] "Analysis" is the process of processing input data and converting it into meaningful information.

[0716] An "initial diagnosis" is the first health assessment performed by the AI ​​diagnostic module.

[0717] "Additional questions" are questions generated to obtain more detailed information based on the initial diagnosis results.

[0718] The "final diagnosis" is the final assessment of health status derived based on the results of all data analysis.

[0719] "Recommended actions" are specific actions or measures suggested based on the diagnostic results.

[0720] A "prompt" is a textual instruction or question presented to the user to obtain additional information.

[0721] "External services" are services that exist outside the system and are linked to make reservations and arrangements.

[0722] This invention relates to an AI health checkup system that runs on smart devices. This system provides an application that allows users to easily check their daily health status, and presents AI-based analysis and diagnosis results, as well as specific recommended actions.

[0723] System Configuration

[0724] 1. User Interface

[0725] Users access the application using smart devices such as smartphones and tablets, where they can input their health concerns via text, voice, or images.

[0726] 2. Data Transmission

[0727] The device sends the health information entered by the user to a cloud server in real time. This information is sent to the server as text data, audio data, and image data.

[0728] 3. AI-based data analysis

[0729] The server inputs the received data into the AI ​​diagnostic module, which performs natural language processing (NLP), voice analysis, and image analysis to make an initial diagnosis. The hardware used is a high-performance cloud computing instance. The software used is nltk, sklearn, speech_recognition, keras, and tensorflow.

[0730] 4. Generate and submit follow-up questions

[0731] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[0732] 5. Providing diagnostic results

[0733] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results and recommended measures (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[0734] Specific examples

[0735] 1. If the user complains of a cough

[0736] User: Opens the app, texts "My cough has been getting worse lately," and sends it.

[0737] Terminal: Sends input information to the server.

[0738] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates an additional question such as "When did the coughing symptoms start?" and sends it to the device.

[0739] Terminal: Display the question to the user.

[0740] User: "A week ago."

[0741] Terminal: Sends the answer to the server.

[0742] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids" along with recommended actions, which are then sent to the device.

[0743] Device: Display diagnostic results and recommended actions to the user.

[0744] 2. If the user complains of skin abnormalities

[0745] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[0746] Device: Sends photos and text information to the server.

[0747] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[0748] Terminal: Display the question to the user.

[0749] User: "A week ago."

[0750] Terminal: Sends the answer to the server.

[0751] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist" along with information about nearby dermatologists, and sends this to the device.

[0752] Device: Display diagnostic results and recommended actions to the user.

[0753] An example of a prompt sentence in this example is "Please diagnose the following symptoms: 'My cough has been getting worse recently.'" Through these specific examples, the AI ​​health diagnosis system of the present invention becomes a system that enables users to conduct daily health consultations and receive quick and accurate diagnoses and recommendations for action.

[0754] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0755] Step 1:

[0756] The user inputs their health-related question via text, voice, or image. The user opens the application on their smart device and inputs their health-related question via text, voice recording using a microphone, or taking an image with a camera and inputting it into the application. This generates user-acquired data.

[0757] Step 2:

[0758] The device sends user input information to the server. The device then sends the acquired text data, voice data, or image data to the cloud server in real time. The server then receives the user input data.

[0759] Step 3:

[0760] The server analyzes the received information using an AI diagnostic module and performs an initial diagnosis. Based on the received data, the server performs an initial diagnosis using natural language processing (nltk, sklearn) for text data, speech analysis (speech_recognition) for audio data, and image analysis (keras and tensorflow) for image data. The server outputs the initial diagnosis results based on the generated diagnosis results.

[0761] Step 4:

[0762] The server generates follow-up questions and sends them to the user via the device. Based on the initial diagnosis results, the server generates follow-up questions to obtain more detailed information. The generative AI model is used to create a prompt sentence and send it to the device, which then displays the follow-up questions to the user.

[0763] Step 5:

[0764] Receive the user's additional answers. The user answers the additional questions displayed on the terminal and enters the answer data into the terminal. The terminal again sends this data to the server, which then receives the additional answer data.

[0765] Step 6:

[0766] The server provides a final diagnosis result based on the additional answers. The server re-inputs the additional answer data into the AI ​​diagnosis module for final analysis. Along with the final diagnosis result, it generates recommended actions for the user and sends them to the device. The device then displays the final diagnosis result to the user.

[0767] Step 7:

[0768] Based on the diagnosis results, the server connects with external services to make the necessary reservations and arrangements. Based on the final diagnosis results, the server generates a detailed action plan to support the user's actions, and connects with external services (such as medical institutions or taxi services) to automatically make the necessary reservations and arrangements. This information is sent to the terminal and notified to the user.

[0769] This series of processing steps allows users to receive a quick and accurate diagnosis and appropriate recommended actions.

[0770] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0771] This invention is an AI-based health checkup system in which users input their health-related inquiries via text, voice, or images, and the system provides diagnostic results and recommended actions after analysis by the server. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more precise diagnoses and responses can be achieved. This system enables early detection and prevention of disease, efficiently supporting the user's health management.

[0772] System Overview

[0773] 1. User Interface

[0774] Users access the application using a device (e.g., a smartphone or tablet) and input their health-related concerns using text, voice, or images. The input is analyzed through an emotion engine, which also determines the user's emotional state.

[0775] 2. Data Transmission

[0776] The device transmits the health information entered by the user and the emotional data analyzed by the emotion engine to the server in real time in the form of text data, voice data, image data, and emotional data.

[0777] 3. AI-based data analysis

[0778] The server inputs the received health information and emotional data into the AI ​​diagnostic module, which performs natural language processing, voice analysis, and image analysis to make an initial diagnosis. It also takes into account the emotional data and reflects it in its diagnosis and recommended actions.

[0779] 4. Generate and submit follow-up questions

[0780] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[0781] 5. Providing diagnostic results

[0782] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results along with recommended measures and actions (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[0783] 6. Implement the action plan

[0784] Based on the diagnosis results, the server provides an action plan to support the user's actions. This includes linking with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as necessary. This information is notified to the user via their device, allowing them to take actual action based on the information provided.

[0785] Specific examples

[0786] 1. If the user complains of stress and coughing symptoms

[0787] User: Opens the app, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[0788] Device: Sends input information and emotional data (how stressed the user is) to the server.

[0789] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[0790] Terminal: Display the question to the user.

[0791] User: "A week ago. Work pressure."

[0792] Terminal: Sends the answer to the server.

[0793] Server: Analyzes the responses and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest, drink plenty of fluids, and try relaxation techniques," and sends these to the device.

[0794] On the device: Display diagnostic results and recommended actions to the user.

[0795] 2. When users complain of skin abnormalities and concerns

[0796] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots and they're making me anxious."

[0797] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[0798] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[0799] Terminal: Display the question to the user.

[0800] User: "A week ago. I've been feeling unwell lately and I'm worried."

[0801] Terminal: Sends the answer to the server.

[0802] Server: Based on the answers and emotion data, the server generates a diagnosis and recommended actions, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends these to the device.

[0803] On the device: Display diagnostic results and recommended actions to the user.

[0804] In this way, by analyzing and responding to the user's emotional state, the system can provide more personalized and appropriate diagnoses and recommended actions, enabling users to take prompt and accurate measures to address health issues and improving the quality of their overall health management.

[0805] The processing flow will be explained below.

[0806] Step 1:

[0807] User: Installs the application, enters personal information (name, age, gender, medical history, etc.) to create an account, and sets authentication information for future logins.

[0808] Step 2:

[0809] Device: Sends personal information and authentication information to a server and requests the creation of a user account.

[0810] Step 3:

[0811] Server: Stores the received personal information in a database, generates an authentication token, and sends it to the device. The user can then use this authentication token to log in from the next time onwards.

[0812] Step 4:

[0813] User: Launches the application and enters their health consultation via text, voice, or image. Once completed, they press the send button.

[0814] Step 5:

[0815] Terminal: Receives health information entered by the user in real time and sends it to the server. The information sent can be text data, voice data, or image data. It also uses an emotion engine to analyze emotional data from the user's input and simultaneously sends it to the server.

[0816] Step 6:

[0817] Server: The received health information and emotional data is input into the AI ​​diagnostic module, which performs text, voice, or image analysis. Based on the analysis results, an initial diagnosis is made and possible illnesses and symptoms are identified. Emotional data is also taken into account to generate an appropriate diagnosis and recommended actions.

[0818] Step 7:

[0819] Server: Based on the initial diagnosis and emotion data, it generates additional questions to gather more detailed information. For example, it generates questions such as, "How long has your cough been bad?" It also generates emotion-based questions such as, "Have you been feeling stressed lately?" depending on the emotional state.

[0820] Step 8:

[0821] Server: Sends the generated follow-up question to the terminal.

[0822] Step 9:

[0823] Terminal: Displays the question received from the server to the user.

[0824] Step 10:

[0825] User: Answer the questions displayed, enter text such as "It's been a week since" and submit.

[0826] Step 11:

[0827] Terminal: Sends the user's additional answers to the server.

[0828] Step 12:

[0829] Server: The received additional answers are input again into the AI ​​diagnostic module, analyzed, and a final diagnostic result is generated. The emotional data is also evaluated again and applied to the diagnostic result.

[0830] Step 13:

[0831] Server: Generates the final diagnosis and recommended actions, and sends them to the device, including, for example, "You may have a cold. Rest, drink plenty of fluids, and adopt relaxation techniques."

[0832] Step 14:

[0833] On the device: The diagnostic results and recommended actions from the server are displayed to the user.

[0834] Step 15:

[0835] Server: Based on the diagnostic results, a detailed action plan is generated to support the user's actions. This includes connecting with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as needed.

[0836] Step 16:

[0837] Device: Notifies the user of the action plan from the server, making it easier for the user to act according to the plan.

[0838] Step 17:

[0839] User: Review the action plan displayed on the device and take specific action as needed using the links and information provided, for example, making a doctor's appointment or ordering a taxi.

[0840] This processing step allows users to easily consult about their health, and the AI ​​system and emotion engine provide quick and accurate diagnoses and recommended actions, enabling early diagnosis and preventive medicine.

[0841] Example 2

[0842] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0843] Conventional health checkup systems only provided general diagnostic results based on the health information entered by the user, making it difficult to perform a detailed diagnosis that took into account the emotional state of each individual user. Furthermore, due to a lack of means to properly analyze and respond to emotional data such as stress and anxiety felt by users, more personalized measures were rarely provided. As a result, early disease detection and prevention, as well as the quality of health management, were not sufficiently improved, making it difficult to efficiently support users' health.

[0844] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to input health-related inquiries using text, voice, or images; a means for a terminal to transmit the user-input information and emotional data to the server; a means for the server to analyze the health information and emotional data received by the server using an AI diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and emotional data and providing a final diagnosis result; a means for providing recommended behaviors and an action plan based on the diagnosis result; and a means for coordinating with external services to make necessary reservations and arrangements. This enables a personalized and precise diagnosis that takes the user's emotional state into consideration, thereby providing more appropriate and prompt measures.

[0845] A "user" is an individual who uses the health checkup system and inputs health-related inquiries.

[0846] A "terminal" is an information and communication device that allows users to access the health checkup system and input and display consultation details and emotional data.

[0847] A "server" is a computing device that receives, analyzes, and diagnoses information sent by users.

[0848] The "AI diagnostic module" is an artificial intelligence program installed on the server that analyzes received health information and emotional data and performs an initial diagnosis.

[0849] An "emotion engine" is software that analyzes the emotional state from user input and generates that data.

[0850] "Health Information" means information in text, audio, or image format that a user inputs into the health examination system.

[0851] "Emotion data" is data that indicates the user's emotional state analyzed through the emotion engine.

[0852] An "initial diagnosis" is the first diagnosis that the AI ​​diagnostic module performs based on the health information and emotional data it receives.

[0853] "Additional questions" are questions that the server generates based on the results of the initial diagnosis, to obtain more detailed information from the user.

[0854] The "final diagnosis result" is a definitive diagnosis made by the server based on the additional answers and emotional data from the user.

[0855] "Recommended actions" are specific measures or suggestions for action for the user that are generated by the server based on the final diagnostic results.

[0856] An "action plan" is a specific plan of action that the user should take in accordance with the recommended action.

[0857] "External services" are external businesses or systems that work in conjunction with the health checkup system to make necessary reservations and arrangements.

[0858] This invention relates to a health checkup system using AI. Users input their health-related inquiries via text, voice, or images, and the system provides diagnostic results and recommended actions after analysis by a server. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more precise diagnoses and responses can be achieved. This system enables early detection and prevention of disease, efficiently supporting users' health management.

[0859] First, a user accesses the application using a device such as a smartphone or tablet and inputs their health-related inquiry using text, voice, or images. The input information is passed to an emotion engine by the device, which analyzes the user's emotional state. This emotion engine is software that extracts emotions from the input data (text, voice, images) and generates emotion data.

[0860] The device transmits the health information and emotional data entered by the user to the server in real time. This data is passed to the server in the form of text data, voice data, image data, and emotional data.

[0861] The server inputs the received health information and emotional data into the AI ​​diagnostic module. This AI diagnostic module is an artificial intelligence program that performs natural language processing (NLP), voice analysis, and image analysis, and has the function of conducting an initial diagnosis. Specifically, it uses NLP technology to analyze text data. It also applies each analysis technology to voice data and image data to perform an initial diagnosis of the user's health condition.

[0862] The server generates additional questions based on the initial diagnosis results and sends them to the user via the device, allowing for more detailed information to be obtained and a more precise diagnosis to be performed.The user answers the additional questions via the device, and the answers are sent back to the server.

[0863] The server analyzes the user's additional responses and emotional data to provide a final diagnosis. Along with the diagnosis, it generates recommended measures and actions (e.g., treatment, referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis and recommended actions through the device.

[0864] Furthermore, the server provides an action plan based on the diagnosis results. This plan is designed to support the user's actions and automatically makes necessary reservations and arrangements by connecting with external services (e.g., e-commerce sites, help apps, taxi services). This information is then sent to the user via their device, allowing them to take action accordingly.

[0865] Specific examples

[0866] 1. If the user complains of stress and coughing symptoms

[0867] User: Opens the app, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[0868] Device: Sends input information and emotional data (how stressed the user is) to the server.

[0869] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[0870] Terminal: Display the question to the user.

[0871] User: "A week ago. Work pressure."

[0872] Terminal: Sends the answer to the server.

[0873] Server: Analyzes the responses and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest, drink plenty of fluids, and try relaxation techniques," and sends these to the device.

[0874] On the device: Display diagnostic results and recommended actions to the user.

[0875] 2. When users complain of skin abnormalities and concerns

[0876] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots and they're making me anxious."

[0877] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[0878] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[0879] Terminal: Display the question to the user.

[0880] User: "A week ago. I've been feeling unwell lately and I'm worried."

[0881] Terminal: Sends the answer to the server.

[0882] Server: Based on the answers and emotion data, the server generates a diagnosis and recommended actions, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends these to the device.

[0883] On the device: Display diagnostic results and recommended actions to the user.

[0884] In this way, by analyzing and responding to a user's emotional state, it is possible to provide more personalized and relevant diagnoses and recommended actions, allowing users to take quicker and more accurate measures to address health issues and improve the quality of their overall health management.

[0885] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0886] Step 1:

[0887] The user accesses the application using a device such as a smartphone or tablet. The user inputs a health-related question in the form of text, voice, or image. For example, the user might input "My cough has gotten worse recently, and I'm feeling stressed." This input serves as the initial data for generating the question content and emotion data.

[0888] Step 2:

[0889] The device passes the health information entered by the user to the emotion engine. The emotion engine analyzes the user's emotional state from the input text, voice, or image data. The analysis generates emotion data, such as "the user is feeling stressed." This emotion data and health information become input data for subsequent data processing.

[0890] Step 3:

[0891] The device sends the emotional data and health information generated by the emotion engine to the server. The data sent includes text data, voice data, image data, and emotional data. The server receives this data and analyzes it in the next step.

[0892] Step 4:

[0893] The server inputs the received health information and emotional data into the AI ​​diagnostic module. This AI diagnostic module performs natural language processing (NLP), voice analysis, and image analysis to make an initial diagnosis. For example, it uses NLP technology to analyze text data and generate follow-up questions such as, "When did your coughing symptoms start?" Here, the initial diagnosis result and any necessary follow-up questions are generated.

[0894] Step 5:

[0895] The server sends the generated follow-up questions to the terminal. The terminal displays these follow-up questions to the user in real time. Based on the user's input, the terminal obtains more detailed information and prepares for a more precise diagnosis. For example, the user might be asked, "When did your coughing symptoms begin?"

[0896] Step 6:

[0897] The user answers additional questions through the device. For example, they might say, "It's been a week. It's because of work pressure." The user's answers are sent to the server through the device. These answers become data for further detailed diagnosis.

[0898] Step 7:

[0899] The server then inputs the user's additional responses and emotional data back into the AI ​​diagnostic module to make a final diagnosis. The AI ​​diagnostic module then analyzes the new data it has received and generates a final diagnosis and recommended actions. For example, it generates a diagnosis and recommended actions such as, "You may have a cold. Please rest, drink plenty of fluids, and adopt relaxation techniques."

[0900] Step 8:

[0901] The server sends the final diagnosis results and recommended actions to the device, which then displays them to the user. The user can check the diagnosis results and take necessary measures.

[0902] Step 9:

[0903] The server provides an action plan based on the diagnosis results. The device then connects with external e-commerce sites, help apps, taxi services, etc., and automatically makes the necessary reservations and arrangements. This information is then sent to the user, who can then take action based on the information provided.

[0904] Through the above steps, users can receive detailed health checkups and appropriate measures in real time, thereby improving the quality of their overall health management.

[0905] (Application example 2)

[0906] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0907] In modern society, there are a wide variety of health-related issues, and users need support to respond quickly and appropriately. However, there is a lack of diagnostic systems that take into account the user's emotional state, and systems that efficiently support health consultations in physical stores. This limits the means by which users can access appropriate medical institutions and products. Therefore, there is a need for a system that can respond to health consultations in physical stores while taking into account the user's health state and emotions.

[0908] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input health-related inquiries using text, voice, or images; a means for a terminal to transmit the user-input information and emotional data to the server; a means for the server to analyze the received information and emotional data using an AI diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis result; a means for providing recommended behaviors and an action plan based on the diagnosis result; a means for coordinating with external services to make necessary reservations and arrangements; and a means having a function for supporting the user's behavior in the store. This enables a quick and appropriate diagnosis and recommended actions to be provided taking into account the user's emotional state, thereby efficiently supporting health consultations in physical stores.

[0909] "User" means a person who inputs health-related questions or symptoms via text, voice, or image.

[0910] A "terminal" is a device that transmits information entered by a user to a server.

[0911] "Server" refers to a computer system that analyzes received information and emotion data and performs diagnosis.

[0912] An "AI diagnostic module" is an artificial intelligence program that analyzes received data and generates an initial diagnosis or follow-up questions.

[0913] "Emotional data" refers to information obtained by analyzing a user's emotional state.

[0914] "Additional questions" are questions generated based on the results of the initial diagnosis to obtain more detailed information.

[0915] "Diagnosis Result" refers to the final health assessment and recommended actions generated by the AI ​​diagnostic module.

[0916] "Recommended actions" are actions or measures suggested to users based on the diagnostic results.

[0917] An "action plan" is a specific plan provided to support the user's actions based on the diagnostic results.

[0918] "External services" refers to medical institutions, reservation services, product sales services, etc. that are linked based on the diagnosis results.

[0919] "Functions to support user behavior within the store" refers to functions that enable users to efficiently receive health consultations and purchase recommended products within the store.

[0920] MODE FOR CARRYING OUT THE INVENTION

[0921] The present invention is a system that provides prompt and appropriate responses when users seek health consultations at physical stores by combining AI diagnosis and emotion analysis. Detailed embodiments of this system are described below.

[0922] System Configuration

[0923] The system includes a means for users to input health-related inquiries via text, voice, or image; a means for the terminal to send the user's input information to a server; a means for the server to analyze the received information and emotional data; a means for the server to generate a diagnosis result and additional questions and send them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis result; a means for providing recommended behaviors and action plans based on the diagnosis result; a means for connecting with external services to make necessary reservations and arrangements; and a function for supporting the user's behavior within the store.

[0924] What the program does

[0925] The program of this system performs the following processing.

[0926] 1. Data capture:

[0927] The device captures health-related information (text, voice, and images) entered by the user. The hardware used can be a smartphone, smart glasses, or a head-mounted display. Voice analysis software is used for voice recognition, and OpenCV is used for image recognition.

[0928] 2. Emotion analysis:

[0929] Extracts emotion data from user input data using a pre-trained EmotionRecognizer model for emotion analysis.

[0930] 3. Data transmission:

[0931] The device transmits the captured data and emotion data to a server, which allows for real-time data analysis.

[0932] 4. AI diagnosis:

[0933] The server inputs the received data into the AI ​​diagnostic module (HealthDiagnosisAI) to perform an initial diagnosis, generates diagnostic results and follow-up questions, and sends them to the user.

[0934] 5. Additional Questions and Answers:

[0935] The server receives the user's answers and analyzes them again, generating additional questions as needed to refine the diagnosis.

[0936] 6. Final diagnosis findings and recommended actions:

[0937] The server generates a final diagnosis and provides the user with a recommended course of action based on the diagnosis, which may include purchasing a product or referring the patient to a specialist.

[0938] 7. Integration with external services:

[0939] Based on the diagnosis results, the server will connect with external medical institutions and services and automatically make the necessary reservations and arrangements.

[0940] 8. In-store support:

[0941] The terminal guides and supports users so that they can efficiently receive health consultations and purchase recommended products in the store.

[0942] Specific examples

[0943] Example 1: User complains of stress and coughing

[0944] User: Opens the app on their smartphone, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[0945] Terminal: Sends input information and emotional data (how stressed the user is) to the server.

[0946] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[0947] Terminal: Display the question to the user.

[0948] User: "A week ago. Work pressure."

[0949] Terminal: Sends the answer to the server.

[0950] Server: Analyzes the answers and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest and drink plenty of fluids. Also, try relaxation techniques," and sends these to the device.

[0951] Device: Display diagnostic results and recommended actions to the user.

[0952] Example 2: When a user complains of skin abnormalities and concerns

[0953] User: Takes a photo of a skin abnormality and types, "These red spots bother me and make me anxious."

[0954] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[0955] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[0956] Terminal: Display the question to the user.

[0957] User: "It's been a week. I've been feeling unwell lately and I'm worried."

[0958] Terminal: Sends the answer to the server.

[0959] Server: Based on the response and emotional data, the server generates a diagnosis and recommended action, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends this to the device.

[0960] Device: Display diagnostic results and recommended actions to the user.

[0961] Prompt Sentence Examples

[0962] "I've been having a bad cough lately. What product would you recommend?"

[0963] "Please give me some advice about this skin condition."

[0964] In this way, the system of the present invention provides prompt and appropriate support for health consultations in physical stores while taking into account the user's emotional state.

[0965] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0966] Step 1:

[0967] The user uses a device (such as a smartphone, smart glasses, or head-mounted display) to input health-related questions and symptoms. The input can be in the form of text, voice, or images, and this data is captured on the device. The device then stores this data as initial data.

[0968] Input: Text, voice, and image data related to the user's health.

[0969] Output: Initial data file on the terminal

[0970] Step 2:

[0971] The initial data captured by the device is collected and emotion analysis is performed. The EmotionRecognizer model is used to determine the user's emotional state (e.g., stress, anxiety, etc.) from the text and voice data. The analysis results are stored as emotion data.

[0972] Input: Initial data file

[0973] Output: Emotion data

[0974] Step 3:

[0975] The device transmits the health information and emotion data entered by the user to the server in real time, and the data format includes text data, voice data, image data, and emotion data.

[0976] Input: Health information, emotional data

[0977] Output: Data sent to the server

[0978] Step 4:

[0979] The server inputs the received data into the AI ​​diagnostic module (HealthDiagnosisAI) for an initial diagnosis. The AI ​​diagnostic module analyzes health information through natural language processing, voice analysis, and image analysis to generate an initial diagnosis.

[0980] Input: Health information, emotional data

[0981] Output: Initial diagnosis result

[0982] Step 5:

[0983] The server generates additional questions based on the initial diagnosis results. The additional questions are intended to obtain additional data necessary to improve the accuracy of the diagnosis. The generated questions are then sent to the device.

[0984] Input: Initial diagnosis result

[0985] Output: Additional questions

[0986] Step 6:

[0987] The device displays the received follow-up questions to the user and collects follow-up answers from the user. The user inputs the answers through the device, and the answers are again saved on the device.

[0988] Input: Additional Question

[0989] Output: User's additional answers

[0990] Step 7:

[0991] The device sends the additional answers it collects to the server, which then analyzes the data again using the AI ​​diagnostic module, resulting in a more accurate final diagnosis.

[0992] Input: Additional Answer

[0993] Output: Final diagnosis result

[0994] Step 8:

[0995] The server generates a recommended course of action or action plan based on the final diagnosis results. The diagnosis results may include, for example, referrals to nearby medical institutions or a list of recommended products. The generated action plan is then sent to the device.

[0996] Input: Final diagnosis result

[0997] Output: Recommended actions, action plan

[0998] Step 9:

[0999] The device will display the recommended actions and action plans received to the user, providing the user with information to take specific actions.

[1000] Input: Recommended Action, Action Plan

[1001] Output: What is displayed to the user

[1002] Step 10:

[1003] Based on the diagnosis results, the server connects with external services and automatically makes necessary reservations and arrangements, such as making appointments at medical institutions or ordering products. This information is then sent to the user via their device.

[1004] Input: Diagnostic results, recommended actions

[1005] Output: Linkage with external services, notification of reservation / arrangement completion

[1006] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1007] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1008] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1009] [Third embodiment]

[1010] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1011] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1012] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1013] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1014] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1015] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1016] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1017] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1018] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1019] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1020] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1021] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1022] This invention relates to a health checkup system using AI, in which users input their health-related inquiries using text, voice, or images, which are analyzed by a server, and diagnostic results and recommended actions are provided. This enables early detection and prevention of diseases, and supports the user's health management.

[1023] System Overview

[1024] 1. User Interface

[1025] Users access the application using a device (e.g., a smartphone or tablet) and enter their health consultation information, which can be entered either as text, voice, or images.

[1026] 2. Data Transmission

[1027] The device transmits the health information entered by the user to the server in real time. This information is sent to the server as text data, voice data, and image data.

[1028] 3. AI-based data analysis

[1029] The server inputs the received data into the AI ​​diagnostic module, which performs natural language processing, voice analysis, and image analysis to make an initial diagnosis. The AI ​​diagnostic module then refers to past data and a medical knowledge base to suggest possible illnesses and treatments.

[1030] 4. Generate and submit follow-up questions

[1031] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[1032] 5. Providing diagnostic results

[1033] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results and recommended actions (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[1034] 6. Implementing the action plan

[1035] Based on the diagnosis results, the server provides recommended actions and an action plan. This includes linking with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as needed. This information is then sent to the user via their device, allowing them to take action based on the information provided.

[1036] Specific examples

[1037] 1. If the user complains of a cough

[1038] User: Opens the app, types in the text "My cough has been getting worse lately," and submits.

[1039] Terminal: Sends input information to the server.

[1040] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start?" and sends them to the device.

[1041] Terminal: Display the question to the user.

[1042] User: "A week ago."

[1043] Terminal: Sends the answer to the server.

[1044] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids" along with recommended actions, which are then sent to the device.

[1045] On the device: Display diagnostic results and recommended actions to the user.

[1046] 2. If the user complains of skin abnormalities

[1047] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[1048] Device: Sends photos and text information to the server.

[1049] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[1050] Terminal: Display the question to the user.

[1051] User: "A week ago."

[1052] Terminal: Sends the answer to the server.

[1053] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist" along with information about nearby dermatologists, and sends this to the device.

[1054] On the device: Display diagnostic results and recommended actions to the user.

[1055] In this way, the AI ​​diagnostic system of the present invention allows users to easily receive health consultations and receive prompt and accurate diagnoses and recommended actions. It is a system that promotes early detection of diseases and preventive medicine, and supports users' health management.

[1056] The processing flow will be explained below.

[1057] Step 1:

[1058] User: Installs the application, enters personal information (name, age, gender, medical history, etc.) to create an account, and sets authentication information for future logins.

[1059] Step 2:

[1060] Device: Sends personal information and authentication information to a server and requests the creation of a user account.

[1061] Step 3:

[1062] Server: Stores the received personal information in a database, generates an authentication token, and sends it to the device. This allows the user to log in using the authentication token from the next time onwards.

[1063] Step 4:

[1064] User: Launches the application and enters their health consultation via text, voice, or image. Once completed, they press the send button.

[1065] Step 5:

[1066] Terminal: Receives health information entered by the user in real time and sends it to the server as text data, audio data, or image data.

[1067] Step 6:

[1068] Server: The received health information is input into the AI ​​diagnostic module, which performs text, voice, or image analysis. Based on the analysis results, an initial diagnosis is made and possible illnesses and symptoms are identified.

[1069] Step 7:

[1070] Server: Based on the results of the initial diagnosis, generate additional questions to gather more detailed information, such as "How long has your cough been bad?"

[1071] Step 8:

[1072] Server: Sends the generated follow-up question to the terminal.

[1073] Step 9:

[1074] Terminal: Displays the question received from the server to the user.

[1075] Step 10:

[1076] User: Answer the questions displayed, enter text such as "It's been a week since" and submit.

[1077] Step 11:

[1078] Terminal: Sends the user's additional answers to the server.

[1079] Step 12:

[1080] Server: The additional answers received are input into the AI ​​diagnostic module again, where they are analyzed and a final diagnostic result is generated.

[1081] Step 13:

[1082] Server: Generates and sends the final diagnosis and recommended measures, such as "You may have a cold. Rest and drink plenty of fluids." to the device.

[1083] Step 14:

[1084] On the device: The diagnostic results and recommended actions from the server are displayed to the user.

[1085] Step 15:

[1086] Server: Based on the diagnosis results, generate an action plan to support the user's actions, including, for example, information on the nearest pharmacy, a link to make an appointment with a medical institution, or information on how to arrange a taxi.

[1087] Step 16:

[1088] Device: Notifies the user of the action plan from the server, making it easier for the user to act according to the plan.

[1089] Step 17:

[1090] User: Review the action plan displayed on the device and take specific action as needed using the links and information provided, for example, making a doctor's appointment or ordering a taxi.

[1091] This processing step allows users to easily consult about their health, and the AI ​​system quickly and accurately provides a diagnosis and recommended actions, enabling early diagnosis and preventive medicine.

[1092] Example 1

[1093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1094] In modern society, it is difficult to obtain prompt and appropriate advice for daily health management or minor health consultations. Receiving expert advice at the appropriate time is especially important for early detection and prevention of illness. However, the time, effort, and cost involved in visiting a medical institution often make people hesitant to do so. Therefore, there is a need for a system that is easy for users to use and can provide accurate diagnoses and effective action plans.

[1095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1096] In this invention, the server includes: a means for a user to input health-related inquiries via text, voice, or image; a means for a terminal to send the user-input information to the server; a means for the server to analyze the received information using a generative AI model diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and send them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis; a means for providing recommended behaviors and action plans based on the diagnosis results; and a means for linking with external services to make necessary reservations and arrangements. This allows users to easily receive health consultations via devices such as smartphones and tablets, and receive prompt and appropriate diagnoses and recommended actions. Linking with external services also makes it easy to make additional reservations and arrangements, facilitating access to actual medical institutions.

[1097] A "user" refers to a person who provides health consultation and uses the system via a device such as a smartphone or tablet.

[1098] "Health consultation" refers to providing information in text, audio, or image format about physical or mental abnormalities or concerns that a user is experiencing.

[1099] "Text" refers to information expressed by characters entered by a user.

[1100] "Voice" refers to information expressed by audible input from a user.

[1101] "Images" refers to photographs and image files taken by users using a camera or other device.

[1102] "Device" refers to electronic devices that users can use, such as smartphones, tablets, and personal computers.

[1103] "Server" refers to the central processing unit that manages the entire system and receives and analyzes information from users.

[1104] A "generative AI model diagnostic module" refers to a software module that uses machine learning technology to analyze data from users and diagnose their health condition based on the results.

[1105] "Initial diagnosis" refers to a provisional diagnosis based on initial input from the user.

[1106] "Additional questions" refer to additional information that needs to be asked of the user in order to make a more accurate diagnosis based on the results of the initial diagnosis.

[1107] "Final Diagnosis" refers to the final medical diagnosis based on all input information from the User.

[1108] "Recommended actions and action plans" refer to specific measures and next steps suggested to users based on the diagnostic results.

[1109] "External services" refers to external resources such as medical institutions, reservation systems, and e-commerce sites that are used by systems to work together and increase user convenience.

[1110] "Reservations and arrangements" refers to using external services to make reservations at medical institutions and arrange for necessary items.

[1111] This invention relates to a health checkup system using AI, in which users input their health-related inquiries using text, voice, or images, which are analyzed by a server, and diagnostic results and recommended actions are provided. This enables early detection and prevention of diseases, and supports the user's health management.

[1112] System Overview

[1113] 1. User Interface

[1114] User: Accesses the application using a device such as a smartphone or tablet. Users can input their health-related inquiries using text, voice, or images.

[1115] 2. Data Transmission

[1116] Device: The device sends the health information (text data, voice data, image data) entered by the user to the server in real time. The device also sends the user's identification information and timestamp along with the entered information.

[1117] 3. AI-based data analysis

[1118] Server: Inputs the received data into the AI ​​diagnostic module and begins analysis. The AI ​​diagnostic module analyzes the data using machine learning models such as natural language processing models (e.g., BERT), speech analysis models (e.g., WaveNet), and image analysis models (e.g., Inception V3) to perform an initial diagnosis.

[1119] 4. Generate and submit follow-up questions

[1120] Server: Based on the results of the initial diagnosis, generate additional questions as needed. For example, automatically generate a question such as "How long have you had a cough?" and send this question to the device.

[1121] 5. Providing diagnostic results

[1122] Server: After receiving the user's additional answers, it makes a final diagnosis. For example, it generates a diagnosis result and recommended actions such as "You may have a cold. Rest and drink plenty of fluids." and sends them to the device.

[1123] 6. Implementing the action plan

[1124] Server: Provides recommended actions and action plans based on the diagnosis results. These action plans include linking with related external services (e.g., medical institution reservation systems, e-commerce sites). The server automatically makes reservations and arrangements as necessary.

[1125] Specific examples

[1126] 1. If the user complains of a cough

[1127] User: Opens the app, types in the text "My cough has been getting worse lately," and submits.

[1128] Terminal: Sends input information to the server.

[1129] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start?" and sends them to the device.

[1130] Terminal: Display the question to the user.

[1131] User: "A week ago."

[1132] Terminal: Sends the answer to the server.

[1133] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids." along with recommended actions, which are sent to the device.

[1134] On the device: Display diagnostic results and recommended actions to the user.

[1135] 2. If the user complains of skin abnormalities

[1136] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[1137] Device: Sends photos and text information to the server.

[1138] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[1139] Terminal: Display the question to the user.

[1140] User: "A week ago."

[1141] Terminal: Sends the answer to the server.

[1142] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist," along with information about nearby dermatologists, and sends this to the device.

[1143] On the device: Display diagnostic results and recommended actions to the user.

[1144] Examples of prompt statements

[1145] Prompt 1: "I've had a bad cough lately. Could it be a cold?"

[1146] Prompt 2: "I'm worried about these red spots. Could I be allergic to something?"

[1147] In this way, the AI ​​diagnostic system of the present invention allows users to easily receive health consultations and receive prompt and accurate diagnoses and recommendations for action, promoting early detection of disease and preventive medicine, and supporting users' health management.

[1148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1149] Step 1:

[1150] Accessing the User Interface

[1151] User: Launches the application on a device such as a smartphone or tablet. The user taps a button such as "Start health consultation" to proceed to the information input screen.

[1152] Input: Tap to launch the app.

[1153] Output: A health consultation input screen will be displayed.

[1154] Step 2:

[1155] Entering information

[1156] User: Enter a health question. For example, type "My cough has been getting worse recently" into the text box and press the send button. Or, tap the voice input button and say "My cough has been getting worse recently" to have the system recognize the voice.

[1157] Input: Health consultation information in text, audio, or image format.

[1158] Output: After pressing the send button, a message will be displayed indicating that the sending is complete. The input data will be prepared.

[1159] Step 3:

[1160] Data transmission

[1161] Terminal: The health information entered by the user is sent to the server. The data sent includes the input information, user identification information, and a timestamp.

[1162] Input: Text, voice, or image data entered by the user.

[1163] Data processing and calculation: For text data, data encoding and formatting. For audio data, audio file generation and compression. For image data, image file compression and format conversion.

[1164] Output: Data packet sent to the server, notification message of successful delivery.

[1165] Step 4:

[1166] Data analysis

[1167] Server: The received data is input into the generation AI model diagnostic module and analysis begins. A natural language processing model (e.g., BERT) analyzes the text and extracts keywords related to symptoms. For audio data, a voice analysis model (e.g., WaveNet) converts it into text and performs the same analysis. For image data, an image analysis model (e.g., Inception V3) analyzes the characteristics of abnormal areas.

[1168] Input: Text, audio, and image data sent by the user.

[1169] Data processing and computation: text analysis using natural language processing, speech recognition and text conversion, and image analysis to identify abnormalities.

[1170] Output: Results of the initial diagnosis and information about any symptoms detected.

[1171] Step 5:

[1172] Generate and submit follow-up questions

[1173] Server: Based on the results of the initial diagnosis, generate additional questions as needed. For example, generate a question such as "How long have you had a cough?" and send it to the device.

[1174] Input: Initial diagnostic results.

[1175] Data processing and calculation: Based on the results of the initial diagnosis, an automatic question generation algorithm is applied to generate follow-up questions.

[1176] Output: Text data of follow-up questions.

[1177] Step 6:

[1178] View additional questions and submit answers

[1179] Terminal: Show the follow-up questions to the user. The user answers the follow-up questions and presses the submit button again to send the answers to the server.

[1180] Input: Text data of the follow-up question.

[1181] Output: The user's answer text, and an indication that the answer was submitted.

[1182] Step 7:

[1183] Providing a final diagnosis and recommended actions

[1184] Server: Analyzes the user's additional answers and generates a final diagnosis, such as "You may have a cold. Rest and drink plenty of fluids.", along with recommended actions, and sends them to the device.

[1185] Input: Additional user response data.

[1186] Data processing and calculations: Analyzing additional response data, applying diagnostic models, and generating diagnostic results and recommended actions.

[1187] Output: Text data of the final diagnosis result and recommended actions.

[1188] Step 8:

[1189] Implementing the action plan

[1190] On the device: The diagnostic results and recommended actions are displayed to the user, along with supplemental information, such as nearby medical facilities and, if necessary, a link to make an appointment.

[1191] Server: Works with external services (such as medical institution reservation systems and e-commerce sites) to automatically make reservations and arrangements as needed.

[1192] Input: Final diagnostic results and recommended actions.

[1193] Data processing and calculation: Automatic processing of reservations and arrangements through integration with external service APIs.

[1194] Output: Notification to the user and confirmation of any related bookings or arrangements.

[1195] (Application example 1)

[1196] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1197] Conventional health checkup systems make it difficult for users to quickly check their health status and receive appropriate diagnostic results and recommended actions. In addition, there are problems with the lack of systems that can be easily used on a daily basis and the lack of integrated AI analysis functions, which makes it difficult for users to effectively manage their health.

[1198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1199] In this invention, the server includes a means for a user to input health-related inquiries by text, voice, or image, a means for a terminal to transmit the user-input information to the server, and a means for the server to analyze the received information using an AI diagnostic module and perform an initial diagnosis. This enables users to check their health condition on a daily basis using their smart devices and receive prompt and appropriate diagnostic results and recommended actions.

[1200] A "user interface" is the means by which a user interacts with a system.

[1201] "Smart devices" refer to portable electronic devices such as smartphones and tablets.

[1202] The "AI diagnostic module" is a component that uses artificial intelligence technology to analyze a user's health information and make a diagnosis.

[1203] "Text data" refers to information entered by a user using characters.

[1204] "Voice data" means information that has been digitally recorded from a user's voice and converted into an analyzable format.

[1205] "Image data" refers to visual information such as photographs and illustrations taken by users.

[1206] "Analysis" is the process of processing input data and converting it into meaningful information.

[1207] An "initial diagnosis" is the first health assessment performed by the AI ​​diagnostic module.

[1208] "Additional questions" are questions generated to obtain more detailed information based on the initial diagnosis results.

[1209] The "final diagnosis" is the final assessment of health status derived based on the results of all data analysis.

[1210] "Recommended actions" are specific actions or measures suggested based on the diagnostic results.

[1211] A "prompt" is a textual instruction or question presented to the user to obtain additional information.

[1212] "External services" are services that exist outside the system and are linked to make reservations and arrangements.

[1213] This invention relates to an AI health checkup system that runs on smart devices. This system provides an application that allows users to easily check their daily health status, and presents AI-based analysis and diagnosis results, as well as specific recommended actions.

[1214] System Configuration

[1215] 1. User Interface

[1216] Users access the application using smart devices such as smartphones and tablets, where they can input their health concerns via text, voice, or images.

[1217] 2. Data Transmission

[1218] The device sends the health information entered by the user to a cloud server in real time. This information is sent to the server as text data, audio data, and image data.

[1219] 3. AI-based data analysis

[1220] The server inputs the received data into the AI ​​diagnostic module, which performs natural language processing (NLP), voice analysis, and image analysis to make an initial diagnosis. The hardware used is a high-performance cloud computing instance. The software used is nltk, sklearn, speech_recognition, keras, and tensorflow.

[1221] 4. Generate and submit follow-up questions

[1222] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[1223] 5. Providing diagnostic results

[1224] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results and recommended actions (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[1225] Specific examples

[1226] 1. If the user complains of a cough

[1227] User: Opens the app, texts "My cough has been getting worse lately," and sends it.

[1228] Terminal: Sends input information to the server.

[1229] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start?" and sends them to the device.

[1230] Terminal: Display the question to the user.

[1231] User: "A week ago."

[1232] Terminal: Sends the answer to the server.

[1233] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids" along with recommended actions, which are then sent to the device.

[1234] On the device: Display diagnostic results and recommended actions to the user.

[1235] 2. If the user complains of skin abnormalities

[1236] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[1237] Device: Sends photos and text information to the server.

[1238] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[1239] Terminal: Display the question to the user.

[1240] User: "A week ago."

[1241] Terminal: Sends the answer to the server.

[1242] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist" along with information about nearby dermatologists, and sends this to the device.

[1243] On the device: Display diagnostic results and recommended actions to the user.

[1244] An example of a prompt sentence in this example is "Please diagnose the following symptoms: 'My cough has been getting worse recently.'" Through these specific examples, the AI ​​health diagnosis system of the present invention becomes a system that enables users to conduct daily health consultations and receive quick and accurate diagnoses and recommendations for action.

[1245] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1246] Step 1:

[1247] The user inputs their health-related question via text, voice, or image. The user opens the application on their smart device and inputs their health-related question via text, voice recording using a microphone, or taking an image with a camera and inputting it into the application. This generates user-acquired data.

[1248] Step 2:

[1249] The device sends user input information to the server. The device then sends the acquired text data, voice data, or image data to the cloud server in real time. The server then receives the user input data.

[1250] Step 3:

[1251] The server analyzes the information it receives using an AI diagnostic module and performs an initial diagnosis. Based on the received data, the server performs an initial diagnosis using natural language processing (nltk, sklearn) for text data, speech analysis (speech_recognition) for audio data, and image analysis (keras and tensorflow) for image data. The server then outputs the initial diagnosis results based on the generated diagnostic results.

[1252] Step 4:

[1253] The server generates follow-up questions and sends them to the user via the device. Based on the initial diagnosis results, the server generates follow-up questions to obtain more detailed information. The generative AI model is used to create a prompt sentence and send it to the device, which then displays the follow-up questions to the user.

[1254] Step 5:

[1255] Receive the user's additional answers. The user answers the additional questions displayed on the terminal and enters the answer data into the terminal. The terminal again sends this data to the server, which then receives the additional answer data.

[1256] Step 6:

[1257] The server provides a final diagnosis result based on the additional answers. The server re-inputs the additional answer data into the AI ​​diagnosis module for final analysis. Along with the final diagnosis result, it generates recommended actions for the user and sends them to the device. The device then displays the final diagnosis result to the user.

[1258] Step 7:

[1259] Based on the diagnosis results, the server connects with external services to make the necessary reservations and arrangements. Based on the final diagnosis results, the server generates a detailed action plan to support the user's actions, and connects with external services (such as medical institutions or taxi services) to automatically make the necessary reservations and arrangements. This information is sent to the terminal and notified to the user.

[1260] This series of processing steps allows users to receive a quick and accurate diagnosis and appropriate recommended actions.

[1261] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1262] This invention is an AI-based health checkup system in which users input their health-related inquiries via text, voice, or images, and the system provides diagnostic results and recommended actions after analysis by the server. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more precise diagnoses and responses can be achieved. This system enables early detection and prevention of disease, efficiently supporting the user's health management.

[1263] System Overview

[1264] 1. User Interface

[1265] Users access the application using a device (e.g., a smartphone or tablet) and input their health-related concerns using text, voice, or images. The input is analyzed through an emotion engine, which also determines the user's emotional state.

[1266] 2. Data Transmission

[1267] The device transmits the health information entered by the user and the emotional data analyzed by the emotion engine to the server in real time in the form of text data, voice data, image data, and emotional data.

[1268] 3. AI-based data analysis

[1269] The server inputs the received health information and emotional data into the AI ​​diagnostic module, which performs natural language processing, voice analysis, and image analysis to make an initial diagnosis. It also takes into account the emotional data and reflects it in its diagnosis and recommended actions.

[1270] 4. Generate and submit follow-up questions

[1271] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[1272] 5. Providing diagnostic results

[1273] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results along with recommended measures and actions (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[1274] 6. Implementing the action plan

[1275] Based on the diagnosis results, the server provides an action plan to support the user's actions. This includes linking with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as necessary. This information is notified to the user via their device, allowing them to take actual action based on the information provided.

[1276] Specific examples

[1277] 1. If the user complains of stress and coughing symptoms

[1278] User: Opens the app, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[1279] Device: Sends input information and emotional data (how stressed the user is) to the server.

[1280] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[1281] Terminal: Display the question to the user.

[1282] User: "A week ago. Work pressure."

[1283] Terminal: Sends the answer to the server.

[1284] Server: Analyzes the responses and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest, drink plenty of fluids, and try relaxation techniques," and sends these to the device.

[1285] On the device: Display diagnostic results and recommended actions to the user.

[1286] 2. When users complain of skin abnormalities and concerns

[1287] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots and they're making me anxious."

[1288] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[1289] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[1290] Terminal: Display the question to the user.

[1291] User: "A week ago. I've been feeling unwell lately and I'm worried."

[1292] Terminal: Sends the answer to the server.

[1293] Server: Based on the answers and emotion data, the server generates a diagnosis and recommended actions, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends these to the device.

[1294] On the device: Display diagnostic results and recommended actions to the user.

[1295] In this way, by analyzing and responding to the user's emotional state, the system can provide more personalized and appropriate diagnoses and recommended actions, enabling users to take prompt and accurate measures to address health issues and improving the quality of their overall health management.

[1296] The processing flow will be explained below.

[1297] Step 1:

[1298] User: Installs the application, enters personal information (name, age, gender, medical history, etc.) to create an account, and sets authentication information for future logins.

[1299] Step 2:

[1300] Device: Sends personal information and authentication information to a server and requests the creation of a user account.

[1301] Step 3:

[1302] Server: Stores the received personal information in a database, generates an authentication token, and sends it to the device. The user can then use this authentication token to log in from the next time onwards.

[1303] Step 4:

[1304] User: Launches the application and enters their health consultation via text, voice, or image. Once completed, they press the send button.

[1305] Step 5:

[1306] Terminal: Receives health information entered by the user in real time and sends it to the server. The information sent can be text data, voice data, or image data. It also uses an emotion engine to analyze emotional data from the user's input and simultaneously sends it to the server.

[1307] Step 6:

[1308] Server: The received health information and emotional data is input into the AI ​​diagnostic module, which performs text, voice, or image analysis. Based on the analysis results, an initial diagnosis is made, identifying possible illnesses and symptoms. Emotional data is also taken into account to generate an appropriate diagnosis and recommended actions.

[1309] Step 7:

[1310] Server: Based on the initial diagnosis and emotion data, it generates additional questions to gather more detailed information. For example, it generates questions such as, "How long has your cough been bad?" It also generates emotion-based questions such as, "Have you been feeling stressed lately?" depending on the emotional state.

[1311] Step 8:

[1312] Server: Sends the generated follow-up question to the terminal.

[1313] Step 9:

[1314] Terminal: Displays the question received from the server to the user.

[1315] Step 10:

[1316] User: Answer the questions displayed, enter text such as "It's been a week since" and submit.

[1317] Step 11:

[1318] Terminal: Sends the user's additional answers to the server.

[1319] Step 12:

[1320] Server: The received additional answers are input again into the AI ​​diagnostic module, analyzed, and a final diagnostic result is generated. The emotional data is also evaluated again and applied to the diagnostic result.

[1321] Step 13:

[1322] Server: Generates the final diagnosis and recommended actions, and sends them to the device, including, for example, "You may have a cold. Rest, drink plenty of fluids, and adopt relaxation techniques."

[1323] Step 14:

[1324] On the device: The diagnostic results and recommended actions from the server are displayed to the user.

[1325] Step 15:

[1326] Server: Based on the diagnostic results, a detailed action plan is generated to support the user's actions. This includes connecting with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as needed.

[1327] Step 16:

[1328] Device: Notifies the user of the action plan from the server, making it easier for the user to act according to the plan.

[1329] Step 17:

[1330] User: Review the action plan displayed on the device and take specific action as needed using the links and information provided, for example, making a doctor's appointment or ordering a taxi.

[1331] This processing step allows users to easily consult about their health, and the AI ​​system and emotion engine provide quick and accurate diagnoses and recommended actions, enabling early diagnosis and preventive medicine.

[1332] Example 2

[1333] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1334] Conventional health checkup systems only provided general diagnostic results based on the health information entered by the user, making it difficult to perform a detailed diagnosis that took into account the emotional state of each individual user. Furthermore, due to a lack of means to properly analyze and respond to emotional data such as stress and anxiety felt by users, more personalized measures were rarely provided. As a result, early disease detection and prevention, as well as the quality of health management, were not sufficiently improved, making it difficult to efficiently support users' health.

[1335] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to input health-related inquiries using text, voice, or images; a means for a terminal to transmit the user-input information and emotional data to the server; a means for the server to analyze the health information and emotional data received by the server using an AI diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and emotional data and providing a final diagnosis result; a means for providing recommended behaviors and an action plan based on the diagnosis result; and a means for coordinating with external services to make necessary reservations and arrangements. This enables a personalized and precise diagnosis that takes the user's emotional state into consideration, thereby providing more appropriate and prompt measures.

[1336] A "user" is an individual who uses the health checkup system and inputs health-related inquiries.

[1337] A "terminal" is an information and communication device that allows users to access the health checkup system and input and display consultation details and emotional data.

[1338] A "server" is a computing device that receives, analyzes, and diagnoses information sent by users.

[1339] The "AI diagnostic module" is an artificial intelligence program installed on the server that analyzes received health information and emotional data and performs an initial diagnosis.

[1340] An "emotion engine" is software that analyzes the emotional state from user input and generates that data.

[1341] "Health Information" means information in text, audio, or image format that a user inputs into the health examination system.

[1342] "Emotion data" is data that indicates the user's emotional state analyzed through the emotion engine.

[1343] An "initial diagnosis" is the first diagnosis that the AI ​​diagnostic module performs based on the health information and emotional data it receives.

[1344] "Additional questions" are questions that the server generates based on the results of the initial diagnosis, to obtain more detailed information from the user.

[1345] The "final diagnosis result" is a definitive diagnosis made by the server based on the additional answers and emotional data from the user.

[1346] "Recommended actions" are specific measures or suggestions for action for the user that are generated by the server based on the final diagnostic results.

[1347] An "action plan" is a specific plan of action that the user should take in accordance with the recommended action.

[1348] "External services" are external businesses or systems that work in conjunction with the health checkup system to make necessary reservations and arrangements.

[1349] This invention relates to a health checkup system using AI. Users input their health-related inquiries via text, voice, or images, and the system provides diagnostic results and recommended actions after analysis by a server. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more precise diagnoses and responses can be achieved. This system enables early detection and prevention of disease, efficiently supporting users' health management.

[1350] First, a user accesses the application using a device such as a smartphone or tablet and inputs their health-related inquiry using text, voice, or images. The input information is passed to an emotion engine by the device, which analyzes the user's emotional state. This emotion engine is software that extracts emotions from the input data (text, voice, images) and generates emotion data.

[1351] The device transmits the health information and emotional data entered by the user to the server in real time. This data is passed to the server in the form of text data, voice data, image data, and emotional data.

[1352] The server inputs the received health information and emotional data into the AI ​​diagnostic module. This AI diagnostic module is an artificial intelligence program that performs natural language processing (NLP), voice analysis, and image analysis, and has the function of conducting an initial diagnosis. Specifically, it uses NLP technology to analyze text data. It also applies each analysis technology to voice data and image data to perform an initial diagnosis of the user's health condition.

[1353] The server generates additional questions based on the initial diagnosis results and sends them to the user via the device, allowing for more detailed information to be obtained and a more precise diagnosis to be performed.The user answers the additional questions via the device, and the answers are sent back to the server.

[1354] The server analyzes the user's additional responses and emotional data to provide a final diagnosis. Along with the diagnosis, it generates recommended measures and actions (e.g., treatment, referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis and recommended actions through the device.

[1355] Furthermore, the server provides an action plan based on the diagnosis results. This plan is designed to support the user's actions and automatically makes necessary reservations and arrangements by connecting with external services (e.g., e-commerce sites, help apps, taxi services). This information is then sent to the user via their device, allowing them to take action accordingly.

[1356] Specific examples

[1357] 1. If the user complains of stress and coughing symptoms

[1358] User: Opens the app, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[1359] Device: Sends input information and emotional data (how stressed the user is) to the server.

[1360] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[1361] Terminal: Display the question to the user.

[1362] User: "A week ago. Work pressure."

[1363] Terminal: Sends the answer to the server.

[1364] Server: Analyzes the responses and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest, drink plenty of fluids, and try relaxation techniques," and sends these to the device.

[1365] On the device: Display diagnostic results and recommended actions to the user.

[1366] 2. When users complain of skin abnormalities and concerns

[1367] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots and they're making me anxious."

[1368] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[1369] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[1370] Terminal: Display the question to the user.

[1371] User: "A week ago. I've been feeling unwell lately and I'm worried."

[1372] Terminal: Sends the answer to the server.

[1373] Server: Based on the answers and emotion data, the server generates a diagnosis and recommended actions, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends these to the device.

[1374] On the device: Display diagnostic results and recommended actions to the user.

[1375] In this way, by analyzing and responding to a user's emotional state, it is possible to provide more personalized and relevant diagnoses and recommended actions, allowing users to take quicker and more accurate measures to address health issues and improve the quality of their overall health management.

[1376] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1377] Step 1:

[1378] The user accesses the application using a device such as a smartphone or tablet. The user inputs a health-related question in the form of text, voice, or image. For example, the user might input "My cough has gotten worse recently, and I'm feeling stressed." This input serves as the initial data for generating the question content and emotion data.

[1379] Step 2:

[1380] The device passes the health information entered by the user to the emotion engine. The emotion engine analyzes the user's emotional state from the input text, voice, or image data. The analysis generates emotion data, such as "the user is feeling stressed." This emotion data and health information become input data for subsequent data processing.

[1381] Step 3:

[1382] The device sends the emotional data and health information generated by the emotion engine to the server. The data sent includes text data, voice data, image data, and emotional data. The server receives this data and analyzes it in the next step.

[1383] Step 4:

[1384] The server inputs the received health information and emotional data into the AI ​​diagnostic module. This AI diagnostic module performs natural language processing (NLP), voice analysis, and image analysis to make an initial diagnosis. For example, it uses NLP technology to analyze text data and generate follow-up questions such as, "When did your coughing symptoms start?" Here, the initial diagnosis result and any necessary follow-up questions are generated.

[1385] Step 5:

[1386] The server sends the generated follow-up questions to the terminal. The terminal displays these follow-up questions to the user in real time. Based on the user's input, the terminal obtains more detailed information and prepares for a more precise diagnosis. For example, the user might be asked, "When did your coughing symptoms begin?"

[1387] Step 6:

[1388] The user answers additional questions through the device. For example, they might say, "It's been a week. It's because of work pressure." The user's answers are sent to the server through the device. These answers become data for further detailed diagnosis.

[1389] Step 7:

[1390] The server then inputs the user's additional responses and emotional data back into the AI ​​diagnostic module to make a final diagnosis. The AI ​​diagnostic module then analyzes the new data it has received and generates a final diagnosis and recommended actions. For example, it generates a diagnosis and recommended actions such as, "You may have a cold. Please rest, drink plenty of fluids, and adopt relaxation techniques."

[1391] Step 8:

[1392] The server sends the final diagnosis results and recommended actions to the device, which then displays them to the user. The user can check the diagnosis results and take necessary measures.

[1393] Step 9:

[1394] The server provides an action plan based on the diagnosis results. The device then connects with external e-commerce sites, help apps, taxi services, etc., and automatically makes the necessary reservations and arrangements. This information is then sent to the user, who can then take action based on the information provided.

[1395] Through the above steps, users can receive detailed health checkups and appropriate measures in real time, thereby improving the quality of their overall health management.

[1396] (Application example 2)

[1397] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1398] In modern society, there are a wide variety of health-related issues, and users need support to respond quickly and appropriately. However, there is a lack of diagnostic systems that take into account the user's emotional state, and systems that efficiently support health consultations in physical stores. This limits the means by which users can access appropriate medical institutions and products. Therefore, there is a need for a system that can respond to health consultations in physical stores while taking into account the user's health state and emotions.

[1399] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input health-related inquiries using text, voice, or images; a means for a terminal to transmit the user-input information and emotional data to the server; a means for the server to analyze the received information and emotional data using an AI diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis result; a means for providing recommended behaviors and an action plan based on the diagnosis result; a means for coordinating with external services to make necessary reservations and arrangements; and a means having a function for supporting the user's behavior in the store. This enables a quick and appropriate diagnosis and recommended actions to be provided taking into account the user's emotional state, thereby efficiently supporting health consultations in physical stores.

[1400] "User" means a person who inputs health-related questions or symptoms via text, voice, or image.

[1401] A "terminal" is a device that transmits information entered by a user to a server.

[1402] "Server" refers to a computer system that analyzes received information and emotion data and performs diagnosis.

[1403] An "AI diagnostic module" is an artificial intelligence program that analyzes received data and generates an initial diagnosis or follow-up questions.

[1404] "Emotional data" refers to information obtained by analyzing a user's emotional state.

[1405] "Additional questions" are questions generated based on the results of the initial diagnosis to obtain more detailed information.

[1406] "Diagnosis Result" refers to the final health assessment and recommended actions generated by the AI ​​diagnostic module.

[1407] "Recommended actions" are actions or measures suggested to users based on the diagnostic results.

[1408] An "action plan" is a specific plan provided to support the user's actions based on the diagnostic results.

[1409] "External services" refers to medical institutions, reservation services, product sales services, etc. that are linked based on the diagnosis results.

[1410] "Functions to support user behavior within the store" refers to functions that enable users to efficiently receive health consultations and purchase recommended products within the store.

[1411] MODE FOR CARRYING OUT THE INVENTION

[1412] The present invention is a system that provides prompt and appropriate responses when users seek health consultations at physical stores by combining AI diagnosis and emotion analysis. Detailed embodiments of this system are described below.

[1413] System Configuration

[1414] The system includes a means for users to input health-related inquiries via text, voice, or image; a means for the terminal to send the user's input information to a server; a means for the server to analyze the received information and emotional data; a means for the server to generate a diagnosis result and additional questions and send them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis result; a means for providing recommended behaviors and action plans based on the diagnosis result; a means for connecting with external services to make necessary reservations and arrangements; and a function for supporting the user's behavior within the store.

[1415] What the program does

[1416] The program of this system performs the following processing.

[1417] 1. Data capture:

[1418] The device captures health-related information (text, voice, and images) entered by the user. The hardware used can be a smartphone, smart glasses, or a head-mounted display. Voice analysis software is used for voice recognition, and OpenCV is used for image recognition.

[1419] 2. Emotion analysis:

[1420] Extracts emotion data from user input data using a pre-trained EmotionRecognizer model for emotion analysis.

[1421] 3. Data transmission:

[1422] The device transmits the captured data and emotion data to a server, which allows for real-time data analysis.

[1423] 4. AI diagnosis:

[1424] The server inputs the received data into the AI ​​diagnostic module (HealthDiagnosisAI) to perform an initial diagnosis, generates diagnostic results and follow-up questions, and sends them to the user.

[1425] 5. Additional Questions and Answers:

[1426] The server receives the user's answers and analyzes them again, generating additional questions as needed to refine the diagnosis.

[1427] 6. Final diagnosis findings and recommended actions:

[1428] The server generates a final diagnosis and provides the user with a recommended course of action based on the diagnosis, which may include purchasing a product or referring the patient to a specialist.

[1429] 7. Integration with external services:

[1430] Based on the diagnosis results, the server will connect with external medical institutions and services and automatically make the necessary reservations and arrangements.

[1431] 8. In-store support:

[1432] The terminal guides and supports users so that they can efficiently receive health consultations and purchase recommended products in the store.

[1433] Specific examples

[1434] Example 1: User complains of stress and coughing

[1435] User: Opens the app on their smartphone, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[1436] Terminal: Sends input information and emotional data (how stressed the user is) to the server.

[1437] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[1438] Terminal: Display the question to the user.

[1439] User: "A week ago. Work pressure."

[1440] Terminal: Sends the answer to the server.

[1441] Server: Analyzes the answers and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest and drink plenty of fluids. Also, try relaxation techniques," and sends these to the device.

[1442] Device: Display diagnostic results and recommended actions to the user.

[1443] Example 2: When a user complains of skin abnormalities and concerns

[1444] User: Takes a photo of a skin abnormality and types, "These red spots bother me and make me anxious."

[1445] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[1446] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[1447] Terminal: Display the question to the user.

[1448] User: "It's been a week. I've been feeling unwell lately and I'm worried."

[1449] Terminal: Sends the answer to the server.

[1450] Server: Based on the response and emotional data, the server generates a diagnosis and recommended action, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends this to the device.

[1451] Device: Display diagnostic results and recommended actions to the user.

[1452] Prompt Sentence Examples

[1453] "I've been having a bad cough lately. What product would you recommend?"

[1454] "Please give me some advice about this skin condition."

[1455] In this way, the system of the present invention provides prompt and appropriate support for health consultations in physical stores while taking into account the user's emotional state.

[1456] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1457] Step 1:

[1458] The user uses a device (such as a smartphone, smart glasses, or head-mounted display) to input health-related questions and symptoms. The input can be in the form of text, voice, or images, and this data is captured on the device. The device then stores this data as initial data.

[1459] Input: Text, voice, and image data related to the user's health.

[1460] Output: Initial data file on the terminal

[1461] Step 2:

[1462] The initial data captured by the device is collected and emotion analysis is performed. The EmotionRecognizer model is used to determine the user's emotional state (e.g., stress, anxiety, etc.) from the text and voice data. The analysis results are stored as emotion data.

[1463] Input: Initial data file

[1464] Output: Emotion data

[1465] Step 3:

[1466] The device transmits the health information and emotion data entered by the user to the server in real time, and the data format includes text data, voice data, image data, and emotion data.

[1467] Input: Health information, emotional data

[1468] Output: Data sent to the server

[1469] Step 4:

[1470] The server inputs the received data into the AI ​​diagnostic module (HealthDiagnosisAI) for an initial diagnosis. The AI ​​diagnostic module analyzes health information through natural language processing, voice analysis, and image analysis to generate an initial diagnosis.

[1471] Input: Health information, emotional data

[1472] Output: Initial diagnosis result

[1473] Step 5:

[1474] The server generates additional questions based on the initial diagnosis results. The additional questions are intended to obtain additional data necessary to improve the accuracy of the diagnosis. The generated questions are then sent to the device.

[1475] Input: Initial diagnosis result

[1476] Output: Additional questions

[1477] Step 6:

[1478] The device displays the received follow-up questions to the user and collects follow-up answers from the user. The user inputs the answers through the device, and the answers are again saved on the device.

[1479] Input: Additional Question

[1480] Output: User's additional answers

[1481] Step 7:

[1482] The device sends the additional answers it collects to the server, which then analyzes the data again using the AI ​​diagnostic module, resulting in a more accurate final diagnosis.

[1483] Input: Additional Answer

[1484] Output: Final diagnosis result

[1485] Step 8:

[1486] The server generates a recommended course of action or action plan based on the final diagnosis results. The diagnosis results may include, for example, referrals to nearby medical institutions or a list of recommended products. The generated action plan is then sent to the device.

[1487] Input: Final diagnosis result

[1488] Output: Recommended actions, action plan

[1489] Step 9:

[1490] The device will display the recommended actions and action plans received to the user, providing the user with information to take specific actions.

[1491] Input: Recommended Action, Action Plan

[1492] Output: What is displayed to the user

[1493] Step 10:

[1494] Based on the diagnosis results, the server connects with external services and automatically makes necessary reservations and arrangements, such as making appointments at medical institutions or ordering products. This information is then sent to the user via their device.

[1495] Input: Diagnostic results, recommended actions

[1496] Output: Linkage with external services, notification of reservation / arrangement completion

[1497] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1498] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1499] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1500] [Fourth embodiment]

[1501] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1502] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1503] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1504] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1505] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1506] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1507] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1508] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1509] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1510] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1511] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1512] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1513] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1514] This invention relates to a health checkup system using AI, in which users input their health-related inquiries using text, voice, or images, which are analyzed by a server, and diagnostic results and recommended actions are provided. This enables early detection and prevention of diseases, and supports the user's health management.

[1515] System Overview

[1516] 1. User Interface

[1517] Users access the application using a device (e.g., a smartphone or tablet) and enter their health consultation information, which can be entered either as text, voice, or images.

[1518] 2. Data Transmission

[1519] The device transmits the health information entered by the user to the server in real time. This information is sent to the server as text data, voice data, and image data.

[1520] 3. AI-based data analysis

[1521] The server inputs the received data into the AI ​​diagnostic module, which performs natural language processing, voice analysis, and image analysis to make an initial diagnosis. The AI ​​diagnostic module then refers to past data and a medical knowledge base to suggest possible illnesses and treatments.

[1522] 4. Generate and submit follow-up questions

[1523] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[1524] 5. Providing diagnostic results

[1525] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results and recommended actions (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[1526] 6. Implementing the action plan

[1527] Based on the diagnosis results, the server provides recommended actions and an action plan. This includes linking with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as needed. This information is then sent to the user via their device, allowing them to take action based on the information provided.

[1528] Specific examples

[1529] 1. If the user complains of a cough

[1530] User: Opens the app, types in the text "My cough has been getting worse lately," and submits.

[1531] Terminal: Sends input information to the server.

[1532] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start?" and sends them to the device.

[1533] Terminal: Display the question to the user.

[1534] User: "A week ago."

[1535] Terminal: Sends the answer to the server.

[1536] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids" along with recommended actions, which are then sent to the device.

[1537] On the device: Display diagnostic results and recommended actions to the user.

[1538] 2. If the user complains of skin abnormalities

[1539] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[1540] Device: Sends photos and text information to the server.

[1541] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[1542] Terminal: Display the question to the user.

[1543] User: "A week ago."

[1544] Terminal: Sends the answer to the server.

[1545] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist" along with information about nearby dermatologists, and sends this to the device.

[1546] On the device: Display diagnostic results and recommended actions to the user.

[1547] In this way, the AI ​​diagnostic system of the present invention allows users to easily receive health consultations and receive prompt and accurate diagnoses and recommended actions. It is a system that promotes early detection of diseases and preventive medicine, and supports users' health management.

[1548] The processing flow will be explained below.

[1549] Step 1:

[1550] User: Installs the application, enters personal information (name, age, gender, medical history, etc.) to create an account, and sets authentication information for future logins.

[1551] Step 2:

[1552] Device: Sends personal information and authentication information to a server and requests the creation of a user account.

[1553] Step 3:

[1554] Server: Stores the received personal information in a database, generates an authentication token, and sends it to the device. This allows the user to log in using the authentication token from the next time onwards.

[1555] Step 4:

[1556] User: Launches the application and enters their health consultation via text, voice, or image. Once completed, they press the send button.

[1557] Step 5:

[1558] Terminal: Receives health information entered by the user in real time and sends it to the server as text data, audio data, or image data.

[1559] Step 6:

[1560] Server: The received health information is input into the AI ​​diagnostic module, which performs text, voice, or image analysis. Based on the analysis results, an initial diagnosis is made and possible illnesses and symptoms are identified.

[1561] Step 7:

[1562] Server: Based on the results of the initial diagnosis, generate additional questions to gather more detailed information, such as "How long has your cough been bad?"

[1563] Step 8:

[1564] Server: Sends the generated follow-up question to the terminal.

[1565] Step 9:

[1566] Terminal: Displays the question received from the server to the user.

[1567] Step 10:

[1568] User: Answer the questions displayed, enter text such as "It's been a week since" and submit.

[1569] Step 11:

[1570] Terminal: Sends the user's additional answers to the server.

[1571] Step 12:

[1572] Server: The additional answers received are input into the AI ​​diagnostic module again, where they are analyzed and a final diagnostic result is generated.

[1573] Step 13:

[1574] Server: Generates and sends the final diagnosis and recommended measures, such as "You may have a cold. Rest and drink plenty of fluids." to the device.

[1575] Step 14:

[1576] On the device: The diagnostic results and recommended actions from the server are displayed to the user.

[1577] Step 15:

[1578] Server: Based on the diagnosis results, generate an action plan to support the user's actions, including, for example, information on the nearest pharmacy, a link to make an appointment with a medical institution, or information on how to arrange a taxi.

[1579] Step 16:

[1580] Device: Notifies the user of the action plan from the server, making it easier for the user to act according to the plan.

[1581] Step 17:

[1582] User: Review the action plan displayed on the device and take specific action as needed using the links and information provided, for example, making a doctor's appointment or ordering a taxi.

[1583] This processing step allows users to easily consult about their health, and the AI ​​system quickly and accurately provides a diagnosis and recommended actions, enabling early diagnosis and preventive medicine.

[1584] Example 1

[1585] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1586] In modern society, it is difficult to obtain prompt and appropriate advice for daily health management or minor health consultations. Receiving expert advice at the appropriate time is especially important for early detection and prevention of illness. However, the time, effort, and cost involved in visiting a medical institution often make people hesitant to do so. Therefore, there is a need for a system that is easy for users to use and can provide accurate diagnoses and effective action plans.

[1587] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1588] In this invention, the server includes: a means for a user to input health-related inquiries via text, voice, or image; a means for a terminal to send the user-input information to the server; a means for the server to analyze the received information using a generative AI model diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and send them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis; a means for providing recommended behaviors and action plans based on the diagnosis results; and a means for linking with external services to make necessary reservations and arrangements. This allows users to easily receive health consultations via devices such as smartphones and tablets, and receive prompt and appropriate diagnoses and recommended actions. Linking with external services also makes it easy to make additional reservations and arrangements, facilitating access to actual medical institutions.

[1589] A "user" refers to a person who provides health consultation and uses the system via a device such as a smartphone or tablet.

[1590] "Health consultation" refers to providing information in text, audio, or image format about physical or mental abnormalities or concerns that a user is experiencing.

[1591] "Text" refers to information expressed by characters entered by a user.

[1592] "Voice" refers to information expressed by audible input from a user.

[1593] "Images" refers to photographs and image files taken by users using a camera or other device.

[1594] "Device" refers to electronic devices that users can use, such as smartphones, tablets, and personal computers.

[1595] "Server" refers to the central processing unit that manages the entire system and receives and analyzes information from users.

[1596] A "generative AI model diagnostic module" refers to a software module that uses machine learning technology to analyze data from users and diagnose their health condition based on the results.

[1597] "Initial diagnosis" refers to a provisional diagnosis based on initial input from the user.

[1598] "Additional questions" refer to additional information that needs to be asked of the user in order to make a more accurate diagnosis based on the results of the initial diagnosis.

[1599] "Final Diagnosis" refers to the final medical diagnosis based on all input information from the User.

[1600] "Recommended actions and action plans" refer to specific measures and next steps suggested to users based on the diagnostic results.

[1601] "External services" refers to external resources such as medical institutions, reservation systems, and e-commerce sites that are used by systems to work together and increase user convenience.

[1602] "Reservations and arrangements" refers to using external services to make reservations at medical institutions and arrange for necessary items.

[1603] This invention relates to a health checkup system using AI, in which users input their health-related inquiries using text, voice, or images, which are analyzed by a server, and diagnostic results and recommended actions are provided. This enables early detection and prevention of diseases, and supports the user's health management.

[1604] System Overview

[1605] 1. User Interface

[1606] User: Accesses the application using a device such as a smartphone or tablet. Users can input their health-related inquiries using text, voice, or images.

[1607] 2. Data Transmission

[1608] Device: The device sends the health information (text data, voice data, image data) entered by the user to the server in real time. The device also sends the user's identification information and timestamp along with the entered information.

[1609] 3. AI-based data analysis

[1610] Server: Inputs the received data into the AI ​​diagnostic module and begins analysis. The AI ​​diagnostic module analyzes the data using machine learning models such as natural language processing models (e.g., BERT), speech analysis models (e.g., WaveNet), and image analysis models (e.g., Inception V3) to perform an initial diagnosis.

[1611] 4. Generate and submit follow-up questions

[1612] Server: Based on the results of the initial diagnosis, generate additional questions as needed. For example, automatically generate a question such as "How long have you had a cough?" and send this question to the device.

[1613] 5. Providing diagnostic results

[1614] Server: After receiving the user's additional answers, it makes a final diagnosis. For example, it generates a diagnosis result and recommended actions such as "You may have a cold. Rest and drink plenty of fluids." and sends them to the device.

[1615] 6. Implementing the action plan

[1616] Server: Provides recommended actions and action plans based on the diagnosis results. These action plans include linking with related external services (e.g., medical institution reservation systems, e-commerce sites). The server automatically makes reservations and arrangements as necessary.

[1617] Specific examples

[1618] 1. If the user complains of a cough

[1619] User: Opens the app, types in the text "My cough has been getting worse lately," and submits.

[1620] Terminal: Sends input information to the server.

[1621] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start?" and sends them to the device.

[1622] Terminal: Display the question to the user.

[1623] User: "A week ago."

[1624] Terminal: Sends the answer to the server.

[1625] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids." along with recommended actions, which are sent to the device.

[1626] On the device: Display diagnostic results and recommended actions to the user.

[1627] 2. If the user complains of skin abnormalities

[1628] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[1629] Device: Sends photos and text information to the server.

[1630] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[1631] Terminal: Display the question to the user.

[1632] User: "A week ago."

[1633] Terminal: Sends the answer to the server.

[1634] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist," along with information about nearby dermatologists, and sends this to the device.

[1635] On the device: Display diagnostic results and recommended actions to the user.

[1636] Examples of prompt statements

[1637] Prompt 1: "I've had a bad cough lately. Could it be a cold?"

[1638] Prompt 2: "I'm worried about these red spots. Could I be allergic to something?"

[1639] In this way, the AI ​​diagnostic system of the present invention allows users to easily receive health consultations and receive prompt and accurate diagnoses and recommendations for action, promoting early detection of disease and preventive medicine, and supporting users' health management.

[1640] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1641] Step 1:

[1642] Accessing the User Interface

[1643] User: Launches the application on a device such as a smartphone or tablet. The user taps a button such as "Start health consultation" to proceed to the information input screen.

[1644] Input: Tap to launch the app.

[1645] Output: A health consultation input screen will be displayed.

[1646] Step 2:

[1647] Entering information

[1648] User: Enter a health question. For example, type "My cough has been getting worse recently" into the text box and press the send button. Or, tap the voice input button and say "My cough has been getting worse recently" to have the system recognize the voice.

[1649] Input: Health consultation information in text, audio, or image format.

[1650] Output: After pressing the send button, a message will be displayed indicating that the sending is complete. The input data will be prepared.

[1651] Step 3:

[1652] Data transmission

[1653] Terminal: The health information entered by the user is sent to the server. The data sent includes the input information, user identification information, and a timestamp.

[1654] Input: Text, voice, or image data entered by the user.

[1655] Data processing and calculation: For text data, data encoding and formatting. For audio data, audio file generation and compression. For image data, image file compression and format conversion.

[1656] Output: Data packet sent to the server, notification message of successful delivery.

[1657] Step 4:

[1658] Data analysis

[1659] Server: The received data is input into the generation AI model diagnostic module and analysis begins. A natural language processing model (e.g., BERT) analyzes the text and extracts keywords related to symptoms. For audio data, a voice analysis model (e.g., WaveNet) converts it into text and performs the same analysis. For image data, an image analysis model (e.g., Inception V3) analyzes the characteristics of abnormal areas.

[1660] Input: Text, audio, and image data sent by the user.

[1661] Data processing and computation: text analysis using natural language processing, speech recognition and text conversion, and image analysis to identify abnormalities.

[1662] Output: Results of the initial diagnosis and information about any symptoms detected.

[1663] Step 5:

[1664] Generate and submit follow-up questions

[1665] Server: Based on the results of the initial diagnosis, generate additional questions as needed. For example, generate a question such as "How long have you had a cough?" and send it to the device.

[1666] Input: Initial diagnostic results.

[1667] Data processing and calculation: Based on the results of the initial diagnosis, an automatic question generation algorithm is applied to generate follow-up questions.

[1668] Output: Text data of follow-up questions.

[1669] Step 6:

[1670] View additional questions and submit answers

[1671] Terminal: Show the follow-up questions to the user. The user answers the follow-up questions and presses the submit button again to send the answers to the server.

[1672] Input: Text data of the follow-up question.

[1673] Output: The user's answer text, and an indication that the answer was submitted.

[1674] Step 7:

[1675] Providing a final diagnosis and recommended actions

[1676] Server: Analyzes the user's additional answers and generates a final diagnosis, such as "You may have a cold. Rest and drink plenty of fluids.", along with recommended actions, and sends them to the device.

[1677] Input: Additional user response data.

[1678] Data processing and calculations: Analyzing additional response data, applying diagnostic models, and generating diagnostic results and recommended actions.

[1679] Output: Text data of the final diagnosis result and recommended actions.

[1680] Step 8:

[1681] Implementing the action plan

[1682] On the device: The diagnostic results and recommended actions are displayed to the user, along with supplemental information, such as nearby medical facilities and, if necessary, a link to make an appointment.

[1683] Server: Works with external services (such as medical institution reservation systems and e-commerce sites) to automatically make reservations and arrangements as needed.

[1684] Input: Final diagnostic results and recommended actions.

[1685] Data processing and calculation: Automatic processing of reservations and arrangements through integration with external service APIs.

[1686] Output: Notification to the user and confirmation of any related bookings or arrangements.

[1687] (Application example 1)

[1688] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1689] Conventional health checkup systems make it difficult for users to quickly check their health status and receive appropriate diagnostic results and recommended actions. In addition, there are problems with the lack of systems that can be easily used on a daily basis and the lack of integrated AI analysis functions, which makes it difficult for users to effectively manage their health.

[1690] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1691] In this invention, the server includes a means for a user to input health-related inquiries by text, voice, or image, a means for a terminal to transmit the user-input information to the server, and a means for the server to analyze the received information using an AI diagnostic module and perform an initial diagnosis. This enables users to check their health condition on a daily basis using their smart devices and receive prompt and appropriate diagnostic results and recommended actions.

[1692] A "user interface" is the means by which a user interacts with a system.

[1693] "Smart devices" refer to portable electronic devices such as smartphones and tablets.

[1694] The "AI diagnostic module" is a component that uses artificial intelligence technology to analyze a user's health information and make a diagnosis.

[1695] "Text data" refers to information entered by a user using characters.

[1696] "Voice data" means information that has been digitally recorded from a user's voice and converted into an analyzable format.

[1697] "Image data" refers to visual information such as photographs and illustrations taken by users.

[1698] "Analysis" is the process of processing input data and converting it into meaningful information.

[1699] An "initial diagnosis" is the first health assessment performed by the AI ​​diagnostic module.

[1700] "Additional questions" are questions generated to obtain more detailed information based on the initial diagnosis results.

[1701] The "final diagnosis" is the final assessment of health status derived based on the results of all data analysis.

[1702] "Recommended actions" are specific actions or measures suggested based on the diagnostic results.

[1703] A "prompt" is a textual instruction or question presented to the user to obtain additional information.

[1704] "External services" are services that exist outside the system and are linked to make reservations and arrangements.

[1705] This invention relates to an AI health checkup system that runs on smart devices. This system provides an application that allows users to easily check their daily health status, and presents AI-based analysis and diagnosis results, as well as specific recommended actions.

[1706] System Configuration

[1707] 1. User Interface

[1708] Users access the application using smart devices such as smartphones and tablets, where they can input their health concerns via text, voice, or images.

[1709] 2. Data Transmission

[1710] The device sends the health information entered by the user to a cloud server in real time. This information is sent to the server as text data, audio data, and image data.

[1711] 3. AI-based data analysis

[1712] The server inputs the received data into the AI ​​diagnostic module, which performs natural language processing (NLP), voice analysis, and image analysis to make an initial diagnosis. The hardware used is a high-performance cloud computing instance. The software used is nltk, sklearn, speech_recognition, keras, and tensorflow.

[1713] 4. Generate and submit follow-up questions

[1714] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[1715] 5. Providing diagnostic results

[1716] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results and recommended actions (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[1717] Specific examples

[1718] 1. If the user complains of a cough

[1719] User: Opens the app, texts "My cough has been getting worse lately," and sends it.

[1720] Terminal: Sends input information to the server.

[1721] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start?" and sends them to the device.

[1722] Terminal: Display the question to the user.

[1723] User: "A week ago."

[1724] Terminal: Sends the answer to the server.

[1725] Server: Analyzes the answer and generates a diagnosis such as "You may have a cold. Rest and drink plenty of fluids" along with recommended actions, which are then sent to the device.

[1726] On the device: Display diagnostic results and recommended actions to the user.

[1727] 2. If the user complains of skin abnormalities

[1728] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots."

[1729] Device: Sends photos and text information to the server.

[1730] Server: The AI ​​diagnostic module analyzes the image and generates an additional question, such as "How long have you had this symptom?", and sends it to the device.

[1731] Terminal: Display the question to the user.

[1732] User: "A week ago."

[1733] Terminal: Sends the answer to the server.

[1734] Server: Based on the answers, the server generates a diagnosis such as "You may have an allergy. Please visit a nearby dermatologist" along with information about nearby dermatologists, and sends this to the device.

[1735] On the device: Display diagnostic results and recommended actions to the user.

[1736] An example of a prompt sentence in this example is "Please diagnose the following symptoms: 'My cough has been getting worse recently.'" Through these specific examples, the AI ​​health diagnosis system of the present invention becomes a system that enables users to conduct daily health consultations and receive quick and accurate diagnoses and recommendations for action.

[1737] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1738] Step 1:

[1739] The user inputs their health-related question via text, voice, or image. The user opens the application on their smart device and inputs their health-related question via text, voice recording using a microphone, or taking an image with a camera and inputting it into the application. This generates user-acquired data.

[1740] Step 2:

[1741] The device sends user input information to the server. The device then sends the acquired text data, voice data, or image data to the cloud server in real time. The server then receives the user input data.

[1742] Step 3:

[1743] The server analyzes the information it receives using an AI diagnostic module and performs an initial diagnosis. Based on the received data, the server performs an initial diagnosis using natural language processing (nltk, sklearn) for text data, speech analysis (speech_recognition) for audio data, and image analysis (keras and tensorflow) for image data. The server then outputs the initial diagnosis results based on the generated diagnostic results.

[1744] Step 4:

[1745] The server generates follow-up questions and sends them to the user via the device. Based on the initial diagnosis results, the server generates follow-up questions to obtain more detailed information. The generative AI model is used to create a prompt sentence and send it to the device, which then displays the follow-up questions to the user.

[1746] Step 5:

[1747] Receive the user's additional answers. The user answers the additional questions displayed on the terminal and enters the answer data into the terminal. The terminal again sends this data to the server, which then receives the additional answer data.

[1748] Step 6:

[1749] The server provides a final diagnosis result based on the additional answers. The server re-inputs the additional answer data into the AI ​​diagnosis module for final analysis. Along with the final diagnosis result, it generates recommended actions for the user and sends them to the device. The device then displays the final diagnosis result to the user.

[1750] Step 7:

[1751] Based on the diagnosis results, the server connects with external services to make the necessary reservations and arrangements. Based on the final diagnosis results, the server generates a detailed action plan to support the user's actions, and connects with external services (such as medical institutions or taxi services) to automatically make the necessary reservations and arrangements. This information is sent to the terminal and notified to the user.

[1752] This series of processing steps allows users to receive a quick and accurate diagnosis and appropriate recommended actions.

[1753] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1754] This invention is an AI-based health checkup system in which users input their health-related inquiries via text, voice, or images, and the system provides diagnostic results and recommended actions after analysis by the server. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more precise diagnoses and responses can be achieved. This system enables early detection and prevention of disease, efficiently supporting the user's health management.

[1755] System Overview

[1756] 1. User Interface

[1757] Users access the application using a device (e.g., a smartphone or tablet) and input their health-related concerns using text, voice, or images. The input is analyzed through an emotion engine, which also determines the user's emotional state.

[1758] 2. Data Transmission

[1759] The device transmits the health information entered by the user and the emotional data analyzed by the emotion engine to the server in real time in the form of text data, voice data, image data, and emotional data.

[1760] 3. AI-based data analysis

[1761] The server inputs the received health information and emotional data into the AI ​​diagnostic module, which performs natural language processing, voice analysis, and image analysis to make an initial diagnosis. It also takes into account the emotional data and reflects it in its diagnosis and recommended actions.

[1762] 4. Generate and submit follow-up questions

[1763] If necessary, the server generates additional questions for the user, which allows for more detailed information and a more precise diagnosis, and these questions are sent to the user via the device.

[1764] 5. Providing diagnostic results

[1765] The server receives the user's additional responses and makes a final diagnosis. It generates the diagnosis results along with recommended measures and actions (e.g., treatment or referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis results and recommended actions through the device.

[1766] 6. Implementing the action plan

[1767] Based on the diagnosis results, the server provides an action plan to support the user's actions. This includes linking with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as necessary. This information is notified to the user via their device, allowing them to take actual action based on the information provided.

[1768] Specific examples

[1769] 1. If the user complains of stress and coughing symptoms

[1770] User: Opens the app, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[1771] Device: Sends input information and emotional data (how stressed the user is) to the server.

[1772] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[1773] Terminal: Display the question to the user.

[1774] User: "A week ago. Work pressure."

[1775] Terminal: Sends the answer to the server.

[1776] Server: Analyzes the responses and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest, drink plenty of fluids, and try relaxation techniques," and sends these to the device.

[1777] On the device: Display diagnostic results and recommended actions to the user.

[1778] 2. When users complain of skin abnormalities and concerns

[1779] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots and they're making me anxious."

[1780] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[1781] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[1782] Terminal: Display the question to the user.

[1783] User: "A week ago. I've been feeling unwell lately and I'm worried."

[1784] Terminal: Sends the answer to the server.

[1785] Server: Based on the answers and emotion data, the server generates a diagnosis and recommended actions, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends these to the device.

[1786] On the device: Display diagnostic results and recommended actions to the user.

[1787] In this way, by analyzing and responding to the user's emotional state, the system can provide more personalized and appropriate diagnoses and recommended actions, enabling users to take prompt and accurate measures to address health issues and improving the quality of their overall health management.

[1788] The processing flow will be explained below.

[1789] Step 1:

[1790] User: Installs the application, enters personal information (name, age, gender, medical history, etc.) to create an account, and sets authentication information for future logins.

[1791] Step 2:

[1792] Device: Sends personal information and authentication information to a server and requests the creation of a user account.

[1793] Step 3:

[1794] Server: Stores the received personal information in a database, generates an authentication token, and sends it to the device. The user can then use this authentication token to log in from the next time onwards.

[1795] Step 4:

[1796] User: Launches the application and enters their health consultation via text, voice, or image. Once completed, they press the send button.

[1797] Step 5:

[1798] Terminal: Receives health information entered by the user in real time and sends it to the server. The information sent can be text data, voice data, or image data. It also uses an emotion engine to analyze emotional data from the user's input and simultaneously sends it to the server.

[1799] Step 6:

[1800] Server: The received health information and emotional data is input into the AI ​​diagnostic module, which performs text, voice, or image analysis. Based on the analysis results, an initial diagnosis is made, identifying possible illnesses and symptoms. Emotional data is also taken into account to generate an appropriate diagnosis and recommended actions.

[1801] Step 7:

[1802] Server: Based on the initial diagnosis and emotion data, it generates additional questions to gather more detailed information. For example, it generates questions such as, "How long has your cough been bad?" It also generates emotion-based questions such as, "Have you been feeling stressed lately?" depending on the emotional state.

[1803] Step 8:

[1804] Server: Sends the generated follow-up question to the terminal.

[1805] Step 9:

[1806] Terminal: Displays the question received from the server to the user.

[1807] Step 10:

[1808] User: Answer the questions displayed, enter text such as "It's been a week since" and submit.

[1809] Step 11:

[1810] Terminal: Sends the user's additional answers to the server.

[1811] Step 12:

[1812] Server: The received additional answers are input again into the AI ​​diagnostic module, analyzed, and a final diagnostic result is generated. The emotional data is also evaluated again and applied to the diagnostic result.

[1813] Step 13:

[1814] Server: Generates the final diagnosis and recommended actions, and sends them to the device, including, for example, "You may have a cold. Rest, drink plenty of fluids, and adopt relaxation techniques."

[1815] Step 14:

[1816] On the device: The diagnostic results and recommended actions from the server are displayed to the user.

[1817] Step 15:

[1818] Server: Based on the diagnostic results, a detailed action plan is generated to support the user's actions. This includes connecting with external e-commerce sites, help apps, taxi services, etc., and automatically making reservations or arrangements as needed.

[1819] Step 16:

[1820] Device: Notifies the user of the action plan from the server, making it easier for the user to act according to the plan.

[1821] Step 17:

[1822] User: Review the action plan displayed on the device and take specific action as needed using the links and information provided, for example, making a doctor's appointment or ordering a taxi.

[1823] This processing step allows users to easily consult about their health, and the AI ​​system and emotion engine provide quick and accurate diagnoses and recommended actions, enabling early diagnosis and preventive medicine.

[1824] Example 2

[1825] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1826] Conventional health checkup systems only provided general diagnostic results based on the health information entered by the user, making it difficult to perform a detailed diagnosis that took into account the emotional state of each individual user. Furthermore, due to a lack of means to properly analyze and respond to emotional data such as stress and anxiety felt by users, more personalized measures were rarely provided. As a result, early disease detection and prevention, as well as the quality of health management, were not sufficiently improved, making it difficult to efficiently support users' health.

[1827] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to input health-related inquiries using text, voice, or images; a means for a terminal to transmit the user-input information and emotional data to the server; a means for the server to analyze the health information and emotional data received by the server using an AI diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and emotional data and providing a final diagnosis result; a means for providing recommended behaviors and an action plan based on the diagnosis result; and a means for coordinating with external services to make necessary reservations and arrangements. This enables a personalized and precise diagnosis that takes the user's emotional state into consideration, thereby providing more appropriate and prompt measures.

[1828] A "user" is an individual who uses the health checkup system and inputs health-related inquiries.

[1829] A "terminal" is an information and communication device that allows users to access the health checkup system and input and display consultation details and emotional data.

[1830] A "server" is a computing device that receives, analyzes, and diagnoses information sent by users.

[1831] The "AI diagnostic module" is an artificial intelligence program installed on the server that analyzes received health information and emotional data and performs an initial diagnosis.

[1832] An "emotion engine" is software that analyzes the emotional state from user input and generates that data.

[1833] "Health Information" means information in text, audio, or image format that a user inputs into the health examination system.

[1834] "Emotion data" is data that indicates the user's emotional state analyzed through the emotion engine.

[1835] An "initial diagnosis" is the first diagnosis that the AI ​​diagnostic module performs based on the health information and emotional data it receives.

[1836] "Additional questions" are questions that the server generates based on the results of the initial diagnosis, to obtain more detailed information from the user.

[1837] The "final diagnosis result" is a definitive diagnosis made by the server based on the additional answers and emotional data from the user.

[1838] "Recommended actions" are specific measures or suggestions for action for the user that are generated by the server based on the final diagnostic results.

[1839] An "action plan" is a specific plan of action that the user should take in accordance with the recommended action.

[1840] "External services" are external businesses or systems that work in conjunction with the health checkup system to make necessary reservations and arrangements.

[1841] This invention relates to a health checkup system using AI. Users input their health-related inquiries via text, voice, or images, and the system provides diagnostic results and recommended actions after analysis by a server. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more precise diagnoses and responses can be achieved. This system enables early detection and prevention of disease, efficiently supporting users' health management.

[1842] First, a user accesses the application using a device such as a smartphone or tablet and inputs their health-related inquiry using text, voice, or images. The input information is passed to an emotion engine by the device, which analyzes the user's emotional state. This emotion engine is software that extracts emotions from the input data (text, voice, images) and generates emotion data.

[1843] The device transmits the health information and emotional data entered by the user to the server in real time. This data is passed to the server in the form of text data, voice data, image data, and emotional data.

[1844] The server inputs the received health information and emotional data into the AI ​​diagnostic module. This AI diagnostic module is an artificial intelligence program that performs natural language processing (NLP), voice analysis, and image analysis, and has the function of conducting an initial diagnosis. Specifically, it uses NLP technology to analyze text data. It also applies each analysis technology to voice data and image data to perform an initial diagnosis of the user's health condition.

[1845] The server generates additional questions based on the initial diagnosis results and sends them to the user via the device, allowing for more detailed information to be obtained and a more precise diagnosis to be performed.The user answers the additional questions via the device, and the answers are sent back to the server.

[1846] The server analyzes the user's additional responses and emotional data to provide a final diagnosis. Along with the diagnosis, it generates recommended measures and actions (e.g., treatment, referral to a nearby medical institution) and sends them to the device. The user can view the diagnosis and recommended actions through the device.

[1847] Furthermore, the server provides an action plan based on the diagnosis results. This plan is designed to support the user's actions and automatically makes necessary reservations and arrangements by connecting with external services (e.g., e-commerce sites, help apps, taxi services). This information is then sent to the user via their device, allowing them to take action accordingly.

[1848] Specific examples

[1849] 1. If the user complains of stress and coughing symptoms

[1850] User: Opens the app, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[1851] Device: Sends input information and emotional data (how stressed the user is) to the server.

[1852] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[1853] Terminal: Display the question to the user.

[1854] User: "A week ago. Work pressure."

[1855] Terminal: Sends the answer to the server.

[1856] Server: Analyzes the responses and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest, drink plenty of fluids, and try relaxation techniques," and sends these to the device.

[1857] On the device: Display diagnostic results and recommended actions to the user.

[1858] 2. When users complain of skin abnormalities and concerns

[1859] User: Takes a photo of a skin abnormality and types, "I'm concerned about these red spots and they're making me anxious."

[1860] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[1861] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[1862] Terminal: Display the question to the user.

[1863] User: "A week ago. I've been feeling unwell lately and I'm worried."

[1864] Terminal: Sends the answer to the server.

[1865] Server: Based on the answers and emotion data, the server generates a diagnosis and recommended actions, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends these to the device.

[1866] On the device: Display diagnostic results and recommended actions to the user.

[1867] In this way, by analyzing and responding to a user's emotional state, it is possible to provide more personalized and relevant diagnoses and recommended actions, allowing users to take quicker and more accurate measures to address health issues and improve the quality of their overall health management.

[1868] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1869] Step 1:

[1870] The user accesses the application using a device such as a smartphone or tablet. The user inputs a health-related question in the form of text, voice, or image. For example, the user might input "My cough has gotten worse recently, and I'm feeling stressed." This input serves as the initial data for generating the question content and emotion data.

[1871] Step 2:

[1872] The device passes the health information entered by the user to the emotion engine. The emotion engine analyzes the user's emotional state from the input text, voice, or image data. The analysis generates emotion data, such as "the user is feeling stressed." This emotion data and health information become input data for subsequent data processing.

[1873] Step 3:

[1874] The device sends the emotional data and health information generated by the emotion engine to the server. The data sent includes text data, voice data, image data, and emotional data. The server receives this data and analyzes it in the next step.

[1875] Step 4:

[1876] The server inputs the received health information and emotional data into the AI ​​diagnostic module. This AI diagnostic module performs natural language processing (NLP), voice analysis, and image analysis to make an initial diagnosis. For example, it uses NLP technology to analyze text data and generate follow-up questions such as, "When did your coughing symptoms start?" Here, the initial diagnosis result and any necessary follow-up questions are generated.

[1877] Step 5:

[1878] The server sends the generated follow-up questions to the terminal. The terminal displays these follow-up questions to the user in real time. Based on the user's input, the terminal obtains more detailed information and prepares for a more precise diagnosis. For example, the user might be asked, "When did your coughing symptoms begin?"

[1879] Step 6:

[1880] The user answers additional questions through the device. For example, they might say, "It's been a week. It's because of work pressure." The user's answers are sent to the server through the device. These answers become data for further detailed diagnosis.

[1881] Step 7:

[1882] The server then inputs the user's additional responses and emotional data back into the AI ​​diagnostic module to make a final diagnosis. The AI ​​diagnostic module then analyzes the new data it has received and generates a final diagnosis and recommended actions. For example, it generates a diagnosis and recommended actions such as, "You may have a cold. Please rest, drink plenty of fluids, and adopt relaxation techniques."

[1883] Step 8:

[1884] The server sends the final diagnosis results and recommended actions to the device, which then displays them to the user. The user can check the diagnosis results and take necessary measures.

[1885] Step 9:

[1886] The server provides an action plan based on the diagnosis results. The device then connects with external e-commerce sites, help apps, taxi services, etc., and automatically makes the necessary reservations and arrangements. This information is then sent to the user, who can then take action based on the information provided.

[1887] Through the above steps, users can receive detailed health checkups and appropriate measures in real time, thereby improving the quality of their overall health management.

[1888] (Application example 2)

[1889] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1890] In modern society, there are a wide variety of health-related issues, and users need support to respond quickly and appropriately. However, there is a lack of diagnostic systems that take into account the user's emotional state, and systems that efficiently support health consultations in physical stores. This limits the means by which users can access appropriate medical institutions and products. Therefore, there is a need for a system that can respond to health consultations in physical stores while taking into account the user's health state and emotions.

[1891] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input health-related inquiries using text, voice, or images; a means for a terminal to transmit the user-input information and emotional data to the server; a means for the server to analyze the received information and emotional data using an AI diagnostic module and perform an initial diagnosis; a means for the server to generate additional questions and transmit them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis result; a means for providing recommended behaviors and an action plan based on the diagnosis result; a means for coordinating with external services to make necessary reservations and arrangements; and a means having a function for supporting the user's behavior in the store. This enables the provision of a quick and appropriate diagnosis and recommended actions that take the user's emotional state into consideration, thereby efficiently supporting health consultations in physical stores.

[1892] "User" means a person who inputs health-related questions or symptoms via text, voice, or image.

[1893] A "terminal" is a device that transmits information entered by a user to a server.

[1894] "Server" refers to a computer system that analyzes received information and emotion data and performs diagnosis.

[1895] An "AI diagnostic module" is an artificial intelligence program that analyzes received data and generates an initial diagnosis or follow-up questions.

[1896] "Emotional data" refers to information obtained by analyzing a user's emotional state.

[1897] "Additional questions" are questions generated based on the results of the initial diagnosis to obtain more detailed information.

[1898] "Diagnosis Result" refers to the final health assessment and recommended actions generated by the AI ​​diagnostic module.

[1899] "Recommended actions" are actions or measures suggested to users based on the diagnostic results.

[1900] An "action plan" is a specific plan provided to support the user's actions based on the diagnostic results.

[1901] "External services" refers to medical institutions, reservation services, product sales services, etc. that are linked based on the diagnosis results.

[1902] "Functions to support user behavior within the store" refers to functions that enable users to efficiently receive health consultations and purchase recommended products within the store.

[1903] MODE FOR CARRYING OUT THE INVENTION

[1904] The present invention is a system that provides prompt and appropriate responses when users seek health consultations at physical stores by combining AI diagnosis and emotion analysis. Detailed embodiments of this system are described below.

[1905] System Configuration

[1906] The system includes a means for users to input health-related inquiries via text, voice, or image; a means for the terminal to send the user's input information to a server; a means for the server to analyze the received information and emotional data; a means for the server to generate a diagnosis result and additional questions and send them to the user via the terminal; a means for receiving the user's additional answers and providing a final diagnosis result; a means for providing recommended behaviors and action plans based on the diagnosis result; a means for connecting with external services to make necessary reservations and arrangements; and a function for supporting the user's activities within the store.

[1907] What the program does

[1908] The program of this system performs the following processing.

[1909] 1. Data capture:

[1910] The device captures health-related information (text, voice, and images) entered by the user. The hardware used can be a smartphone, smart glasses, or a head-mounted display. Voice analysis software is used for voice recognition, and OpenCV is used for image recognition.

[1911] 2. Emotion analysis:

[1912] Extracts emotion data from user input data using a pre-trained EmotionRecognizer model for emotion analysis.

[1913] 3. Data transmission:

[1914] The device transmits the captured data and emotion data to a server, which allows for real-time data analysis.

[1915] 4. AI diagnosis:

[1916] The server inputs the received data into the AI ​​diagnostic module (HealthDiagnosisAI) to perform an initial diagnosis, generates diagnostic results and follow-up questions, and sends them to the user.

[1917] 5. Additional Questions and Answers:

[1918] The server receives the user's answers and analyzes them again, generating additional questions as needed to refine the diagnosis.

[1919] 6. Final diagnosis findings and recommended actions:

[1920] The server generates a final diagnosis and provides the user with a recommended course of action based on the diagnosis, which may include purchasing a product or referring the patient to a specialist.

[1921] 7. Integration with external services:

[1922] Based on the diagnosis results, the server will connect with external medical institutions and services and automatically make the necessary reservations and arrangements.

[1923] 8. In-store support:

[1924] The terminal guides and supports users so that they can efficiently receive health consultations and purchase recommended products in the store.

[1925] Specific examples

[1926] Example 1: User complains of stress and coughing

[1927] User: Opens the app on their smartphone, types in the text "My cough has been getting worse lately. I'm also feeling stressed," and sends it.

[1928] Terminal: Sends input information and emotional data (how stressed the user is) to the server.

[1929] Server: Inputs the received data into the AI ​​diagnostic module and performs an initial diagnosis. For example, it generates additional questions such as "When did the coughing symptoms start? What is the cause of stress?" and sends them to the device.

[1930] Terminal: Display the question to the user.

[1931] User: "A week ago. Work pressure."

[1932] Terminal: Sends the answer to the server.

[1933] Server: Analyzes the answers and emotional data, generates a diagnosis and recommended actions such as "You may have a cold. Rest and drink plenty of fluids. Also, try relaxation techniques," and sends these to the device.

[1934] Device: Display diagnostic results and recommended actions to the user.

[1935] Example 2: When a user complains of skin abnormalities and concerns

[1936] User: Takes a photo of a skin abnormality and types, "These red spots bother me and make me anxious."

[1937] Device: Sends photos, text information, and emotional data (user's anxiety) to the server.

[1938] Server: The AI ​​diagnostic module analyzes the image and generates additional questions such as, "How long have you had this symptom? What specifically is causing your anxiety?" and sends them to the device.

[1939] Terminal: Display the question to the user.

[1940] User: "It's been a week. I've been feeling unwell lately and I'm worried."

[1941] Terminal: Sends the answer to the server.

[1942] Server: Based on the response and emotional data, the server generates a diagnosis and recommended action, such as "You may have an allergy. Visit a nearby dermatologist. Also, consider counseling for stress management," and sends this to the device.

[1943] Device: Display diagnostic results and recommended actions to the user.

[1944] Prompt Sentence Examples

[1945] "I've been having a bad cough lately. What product would you recommend?"

[1946] "Please give me some advice about this skin condition."

[1947] In this way, the system of the present invention provides prompt and appropriate support for health consultations in physical stores while taking into account the user's emotional state.

[1948] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1949] Step 1:

[1950] The user uses a device (smartphone, smart glasses, head-mounted display, etc.) to input health questions and symptoms. The input format is text, voice, or image, and this data is captured on the device. The device then stores this as initial data.

[1951] Input: Text, voice, and image data related to the user's health.

[1952] Output: Initial data file on the terminal

[1953] Step 2:

[1954] The initial data captured by the device is collected and emotion analysis is performed. The EmotionRecognizer model is used to determine the user's emotional state (e.g., stress, anxiety, etc.) from the text and voice data. The analysis results are stored as emotion data.

[1955] Input: Initial data file

[1956] Output: Emotion data

[1957] Step 3:

[1958] The device transmits the health information and emotion data entered by the user to the server in real time, and the data format includes text data, voice data, image data, and emotion data.

[1959] Input: Health information, emotional data

[1960] Output: Data sent to the server

[1961] Step 4:

[1962] The server inputs the received data into the AI ​​diagnostic module (HealthDiagnosisAI) for an initial diagnosis. The AI ​​diagnostic module analyzes health information through natural language processing, voice analysis, and image analysis to generate an initial diagnosis.

[1963] Input: Health information, emotional data

[1964] Output: Initial diagnosis result

[1965] Step 5:

[1966] The server generates additional questions based on the initial diagnosis results. The additional questions are intended to obtain additional data necessary to improve the accuracy of the diagnosis. The generated questions are then sent to the device.

[1967] Input: Initial diagnosis result

[1968] Output: Additional questions

[1969] Step 6:

[1970] The device displays the received follow-up questions to the user and collects follow-up answers from the user. The user inputs the answers through the device, and the answers are again saved on the device.

[1971] Input: Additional Question

[1972] Output: User's additional answers

[1973] Step 7:

[1974] The device sends the additional answers it collects to the server, which then analyzes the data again using the AI ​​diagnostic module, resulting in a more accurate final diagnosis.

[1975] Input: Additional Answer

[1976] Output: Final diagnosis result

[1977] Step 8:

[1978] The server generates a recommended course of action or action plan based on the final diagnosis results. The diagnosis results may include, for example, referrals to nearby medical institutions or a list of recommended products. The generated action plan is then sent to the device.

[1979] Input: Final diagnosis result

[1980] Output: Recommended actions, action plan

[1981] Step 9:

[1982] The device will display the recommended actions and action plans received to the user, providing the user with information to take specific actions.

[1983] Input: Recommended Action, Action Plan

[1984] Output: What is displayed to the user

[1985] Step 10:

[1986] Based on the diagnosis results, the server connects with external services and automatically makes necessary reservations and arrangements, such as making appointments at medical institutions or ordering products. This information is then sent to the user via their device.

[1987] Input: Diagnostic results, recommended actions

[1988] Output: Linkage with external services, notification of reservation / arrangement completion

[1989] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1990] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1991] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1992] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1993] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1994] These emo...

Claims

1. A means for users to input their health concerns via text, voice, or image; A means for the terminal to transmit user input information to a server; A means for analyzing the information received by the server using an AI diagnostic module and performing an initial diagnosis; means for the server to generate and transmit a follow-up question to the user via the terminal; A means for receiving additional user responses and providing a final diagnosis; A means of providing recommended actions and action plans based on the diagnostic findings; A means of connecting with external services to make necessary reservations and arrangements, A system including:

2. The system of claim 1 , wherein the terminal displays diagnostic results and recommended actions from the server to the user.

3. The system of claim 1 , wherein the server generates a detailed action plan to support the user's actions based on the diagnostic results.

Images