system

Abstract

Provide a system. [Solution] means for receiving medical interview information input by a user; A means for analyzing the received medical interview information; A means for providing medical record information generated based on the analysis results to a medical institution; A means for receiving reservation information and making an appointment with a medical institution; means for notifying a user of medical examination result information received from a medical institution; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional medical systems, patients have to fill out their medical history information, which takes time, and then medical institutions have to manually enter the information, which wastes even more time. In addition, insufficient recording or errors in medical history information can affect the accuracy of diagnoses. This reduces the efficiency of medical consultations and causes inconvenience to both patients and doctors. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides a system that receives and analyzes medical interview information entered by a user and provides medical records generated based on the results to a medical institution. Specifically, the system includes a means for receiving medical interview information from a user, analyzing the information using a machine learning model, and generating medical records including a diagnosis prediction. The system also includes a means for receiving appointment information, making an appointment with a medical institution, and notifying the user of the examination result information received from the medical institution, thereby shortening the consultation time and improving the accuracy of the diagnosis.

[0006] A "user" is an individual or their agent who uses the system to input medical interview information and schedule an appointment.

[0007] "Medical interview information" is detailed information about a user's own health condition and symptoms that the user enters into the system.

[0008] The "means for receiving" is a function or process for loading the medical interview information and reservation information provided by the user into the system.

[0009] The "means for analyzing" is a function or process for processing the received medical interview information and predicting possible diagnoses or diseases.

[0010] A "machine learning model" is an algorithm that learns patterns and regularities from large amounts of data and makes predictions and classifications for new data.

[0011] "Medical record information" refers to medical record data used by medical institutions, including analyzed interview information and predicted diagnosis results.

[0012] "Means for providing" refers to the function or process for transmitting or making accessible the generated medical record information to a medical institution.

[0013] "Reservation information" is detailed information about the date and time of the consultation that the user desires through the system and the medical institution.

[0014] The "means for making an appointment" is a function or process for receiving appointment information, coordinating with a medical institution to set up a desired appointment date and time, and finalizing the appointment date and time.

[0015] "Examination result information" is data related to the results of examinations and tests performed on the user by the medical institution, prescription details, and the like.

[0016] The "notification means" is a function or process for notifying the user of the medical examination result information received from the medical institution. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system that uses a medical app to collect medical interview information from users, analyzes it to generate medical record information, and then connects with medical institutions. The program processing of this system is explained in natural language below. Specific examples are also provided for further explanation.

[0039] User Registration and Login

[0040] When a user uses the app for the first time, they tap the "New Registration" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the entered information to the server, which saves it in a database. Next, the user enters authentication information on the "Login" screen and taps the login button, which the device sends to the server. The server performs authentication and returns an access token to the device, completing the login.

[0041] Fill out the medical questionnaire form

[0042] When the user taps the "Start medical interview" button, the device sends a medical interview start request to the server. The server generates the first question and returns it to the device, which then displays it to the user. After that, the user answers each question, and the device sends them to the server. The server analyzes the answers, determines the next question, and returns it to the device. This process is repeated until all questions have been asked.

[0043] Analysis of interview results and reflection in medical records

[0044] Once all medical interviews are completed, the server analyzes the user's interview data using a machine learning model and makes a diagnosis prediction. The server then creates a request to send the medical record information, including the generated diagnosis prediction, to the target medical institution. The medical institution's system receives this medical record information and prepares it for use in consultations.

[0045] Appointments and doctor coordination

[0046] When a user makes an appointment for a desired consultation date within the app, the device sends the appointment information to the server. The server then connects with the medical institution to confirm whether the appointment is possible and returns the result to the device. If the appointment is approved, the device displays a success message to the user. The medical institution prepares to efficiently conduct the consultation based on the medical interview data received via the server. After the consultation, the medical institution sends the examination results and prescription information to the server, which then notifies the user. The user can then check the examination results through the app.

[0047] Specific examples

[0048] As an example, consider the case where a user complaining of migraines uses this system. The user enters "I often get headaches" in the medical interview information and provides detailed information about the frequency and other symptoms. The server receives this information, and the machine learning model predicts that the diagnosis is likely "migraine." The server generates medical record information including this diagnosis prediction and sends it to the medical institution. By receiving this information in advance, the medical institution can quickly begin consultation when the user visits the hospital without having to reconfirm the medical interview details. After the consultation, the medical institution sends the examination results and prescription details to the server, allowing the user to check them through the app.

[0049] This will improve the efficiency of medical interviews and the accuracy of diagnoses, making the system beneficial for both users and medical institutions.

[0050] The processing flow will be explained below.

[0051] User Registration and Login

[0052] Step 1:

[0053] User: The user launches the app and taps the "Sign Up" button.

[0054] Step 2:

[0055] Device: The device will display the new registration screen.

[0056] Step 3:

[0057] User: The user enters their name, email address, and password, then taps the submit button.

[0058] Step 4:

[0059] Device: The device creates a request to send the input information to the server.

[0060] Step 5:

[0061] Server: The server receives the request and performs validation of the input information.

[0062] Step 6:

[0063] Server: If the server is happy with the information, it stores it in the database.

[0064] Step 7:

[0065] Server: The server returns a registration success response to the device.

[0066] Step 8:

[0067] Terminal: The terminal displays a successful registration message to the user.

[0068] Step 9:

[0069] User: The user enters their credentials on the "Login" screen and taps the login button.

[0070] Step 10:

[0071] Device: The device sends the entered authentication information to the server.

[0072] Step 11:

[0073] Server: The server verifies the authentication information, generates an access token, and returns a response to the device.

[0074] Step 12:

[0075] Device: The device stores the access token and uses it in subsequent requests.

[0076] Fill out the medical questionnaire form

[0077] Step 1:

[0078] User: The user taps the "Start Interview" button.

[0079] Step 2:

[0080] Terminal: The terminal sends a request to start an interview to the server.

[0081] Step 3:

[0082] Server: The server generates the initial medical interview questions and returns them to the terminal.

[0083] Step 4:

[0084] Terminal: The terminal displays the initial question to the user.

[0085] Step 5:

[0086] User: The user answers the question and taps the Next button.

[0087] Step 6:

[0088] Device: The device sends the answer to the server.

[0089] Step 7:

[0090] Server: The server stores the received answer in a database and generates the next question.

[0091] Step 8:

[0092] Server: The server sends the following question to the device:

[0093] Step 9:

[0094] Terminal: The terminal displays the next question to the user. This process is repeated until all questions have been answered.

[0095] Analysis of interview results and reflection in medical records

[0096] Step 1:

[0097] Server: After all questionnaire responses are completed, the server inputs the questionnaire data into the machine learning model and begins analysis.

[0098] Step 2:

[0099] Server: The server obtains the analysis results and generates medical record information including diagnostic predictions.

[0100] Step 3:

[0101] Server: The server creates a request to send the medical record information to the medical institution.

[0102] Step 4:

[0103] Server: The server sends the medical record information to the target medical institution.

[0104] Step 5:

[0105] Medical institutions: Medical institutions receive the medical record information and use it for medical examinations.

[0106] Appointments and doctor coordination

[0107] Step 1:

[0108] User: The user opens the "Appointment" screen within the app and selects the desired appointment date and time.

[0109] Step 2:

[0110] Terminal: The terminal makes a request to send the reservation information to the server.

[0111] Step 3:

[0112] Server: The server receives the reservation request and checks whether the reservation is available by connecting with the medical institution's system.

[0113] Step 4:

[0114] Server: The server returns the reservation approval result to the terminal.

[0115] Step 5:

[0116] Terminal: The terminal displays a confirmation message to the user.

[0117] Step 6:

[0118] Medical institution: The medical institution receives the reservation information and prepares for the examination based on the user's medical interview data.

[0119] Step 7:

[0120] Medical institution: After the examination, the medical institution sends the examination result information to the server.

[0121] Step 8:

[0122] Server: The server receives the medical result information and stores it in association with the user's account.

[0123] Step 9:

[0124] Device: The device will notify the user of the medical examination results and allow them to view them within the app.

[0125] The above is a detailed description of the operations performed in each processing step. In this way, the user can efficiently carry out the process from interviewing the patient to examining the patient and confirming the examination results.

[0126] Example 1

[0127] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0128] Modern medical care involves many processes, such as collecting and analyzing medical history information, scheduling appointments, and notifying patients of diagnosis results. Each process requires time and effort, creating a challenge. Furthermore, doctors' time spent comprehending detailed medical history information during consultations reduces the efficiency of the entire consultation. Conventional systems have struggled to manage these processes in an integrated manner and process them efficiently and accurately.

[0129] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0130] In this invention, the server includes a means for receiving medical interview information input by a user, a means for analyzing the received medical interview information, a means for providing electronic medical record information generated based on the analysis results to a medical institution, a means for receiving appointment information and making an appointment with the medical institution, and a means for notifying the user of the examination result information received from the medical institution. This enables efficient collection and analysis of medical interview information, prompt notification of diagnosis results, and efficient examinations through cooperation with medical institutions.

[0131] "User" refers to an individual or corporation that uses the system.

[0132] "Medical interview information" refers to information about health conditions and symptoms provided by the user.

[0133] "Electronic medical record information" refers to health records generated from medical interview information and analysis results.

[0134] "Medical institution" refers to an organization such as a hospital or clinic that provides medical care or treatment.

[0135] "Appointment for medical examination" refers to the act of reserving a desired date and time for medical examination at a medical institution by a user.

[0136] "Examination result information" refers to information regarding diagnoses and prescriptions provided by medical institutions after examinations.

[0137] A "server" is a computer device that is the core of a system and that processes and stores data and communicates with other devices.

[0138] A "machine learning model" refers to an algorithm that learns from large amounts of data and makes predictions and classifications.

[0139] "Notification" refers to a means by which the system notifies the user of information.

[0140] "Receive" refers to the act of obtaining data from another device or system.

[0141] "Analysis" refers to the act of examining information in detail based on received data and understanding its meaning and structure.

[0142] The present invention is a system that uses a medical application to collect medical interview information from users, analyzes the information, generates electronic medical records, and collaborates with medical institutions. This system consists of a server, terminals, and users.

[0143] Program processing explanation

[0144] User Registration and Login

[0145] When a user launches the app and taps the "New Registration" button, the device displays the new registration screen. When the user enters their name, email address, and password and taps the "Submit" button, the device sends the input information to the server. The server stores the received information in a database.

[0146] When a user enters an email address and password on the "Login" screen and taps the login button, the device sends the entered information to the server. The server performs authentication and returns an access token to the device, completing the login. The technology used is a smartphone or tablet as the device, and a cloud service (e.g., Amazon Web Services, Microsoft® Azure®) as the server.

[0147] Fill out the medical questionnaire form

[0148] When the user taps the "Start medical interview" button, the device sends a request to the server to start the medical interview. The server generates the first questions and returns them to the device. The device displays the questions, and the user answers each one. When the device sends the answers to the server, the server analyzes the answers and determines the next question. This process is repeated until all questions have been asked.

[0149] The technologies used include stream processing using Apache (registered trademark) Kafka for real-time analysis on the server side, and natural language processing (NLP) libraries implemented in Python (e.g., SpaCy, NLTK).

[0150] Analysis of interview results and reflection in medical records

[0151] Once all interviews are completed, the device sends an end request to the server. The server inputs the interview data into a machine learning model for analysis. Specific technologies used include Scikit-Learn and TENSORFLOW (registered trademark). The server generates a diagnostic prediction and formats it as electronic medical record information. The server then creates and sends a request to send this electronic medical record information to the medical institution.

[0152] The medical institution's system receives electronic medical record information and prepares it so that it can be used for consultations. The technology used utilizes the standard FHIR (Fast Healthcare Interoperability Resources).

[0153] Appointments and doctor coordination

[0154] When a user makes a reservation for a desired consultation date within the app, the device sends the reservation information to the server. The server then connects with the medical institution to confirm whether the reservation is possible. If the reservation is approved, the device displays a success message to the user.

[0155] On the day of the consultation, the medical institution will prepare for the consultation efficiently based on the medical interview data received through the server. After the consultation, the medical institution will send the consultation results and prescription information to the server, which will notify the user. The user can then check the consultation results through the application.

[0156] As a specific example of how it works, for example, a user who complains of migraines uses the system and enters "I often get headaches" into the medical interview information, and provides detailed information about the frequency and other symptoms. Based on this information, the server uses a machine learning model to predict the possibility of a "migraine," generates electronic medical record information, and sends it to the medical institution. By receiving this information in advance, the medical institution can prepare for a prompt examination.

[0157] Prompt Sentence Examples

[0158] Here are some example prompts to input to a generative AI model:

[0159] "What API endpoints do you use to process user registration?"

[0160] "Please explain how to analyze medical interview data using a machine learning model."

[0161] "Please tell me about the process for sending medical record information according to the FHIR standard."

[0162] In this way, this system uses technically linked components to improve the efficiency of medical interviews and the accuracy of diagnoses.

[0163] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0164] System program processing steps

[0165] User Registration and Login

[0166] Step 1:

[0167] The user launches the app and taps the "New Registration" button. The device displays the new registration screen. The user enters their name, email address, and password, and taps the send button. The input is their name, email address, and password, and this information is sent from the device to the server as output.

[0168] Step 2:

[0169] The server stores the received information in a database. In this storage process, an INSERT query is executed using a database management system (e.g., MySQL (registered trademark)) to record the user information. The input is the user information received from the terminal, and the output is the result stored in the database.

[0170] Step 3:

[0171] The user enters their email address and password on the "Login" screen and taps the login button. The device sends the authentication information to the server. The input is the email address and password, and the output is this information sent from the device to the server.

[0172] Step 4:

[0173] The server references the database and checks the authentication information. If authentication is successful, it generates a JWT (JSON Web Token) and returns it to the terminal as an access token. If authentication fails, it generates an error message and returns it to the terminal. The input is authentication information, and the output is an access token or an error message.

[0174] Fill out the medical questionnaire form

[0175] Step 5:

[0176] The user taps the "Start medical interview" button. The terminal sends a medical interview start request to the server. The input is the medical interview start request, and the output is that this request is sent from the terminal to the server.

[0177] Step 6:

[0178] The server generates the initial question and returns it to the terminal. At this time, the server executes an algorithm that randomly selects a question from a list of questions prepared in advance. The input is the request to start the medical interview, and the output is the initial question.

[0179] Step 7:

[0180] The terminal displays a question, the user answers the question, the input is the initial question, and the output is the user's answer.

[0181] Step 8:

[0182] The device sends the user's answer to the server. The server analyzes the received answer using an NLP algorithm (e.g., SpaCy, NLTK). Based on the analysis results, the next question is dynamically generated. The input is the user's answer, and the output is the next question.

[0183] Step 9:

[0184] The server generates the next question and sends it back to the terminal, which displays it to the user. This process is repeated until all questions have been asked. The input is the next question, and the output is that question will be displayed on the terminal.

[0185] Analysis of interview results and reflection in medical records

[0186] Step 10:

[0187] When all the inquiries are completed, the terminal sends an end request to the server. The input is the inquiry end request, and the output is that the request is sent from the terminal to the server.

[0188] Step 11:

[0189] The server inputs the received medical interview data into a machine learning model (e.g., Scikit-Learn, TensorFlow) for analysis. The machine learning model generates a diagnostic prediction. The input is the medical interview data, and the output is a diagnostic prediction.

[0190] Step 12:

[0191] The server formalizes the generated diagnosis prediction as electronic medical record information. Then, a request including this electronic medical record information is sent to the medical institution. The input is the diagnosis prediction, and the output is the electronic medical record information.

[0192] Step 13:

[0193] The medical institution's system receives the electronic medical record information and prepares it so that it can be used for consultations. The input is the electronic medical record information, and the output is the completion of consultation preparation.

[0194] Appointments and doctor coordination

[0195] Step 14:

[0196] The user makes a reservation for the desired consultation date within the app, and the device sends the reservation information to the server. The input is the reservation information, and the output is that information is sent from the device to the server.

[0197] Step 15:

[0198] The server receives the reservation information and sends a request to the medical institution to confirm whether the reservation is available. The input is the reservation information, and the output is confirmation of whether the reservation is available.

[0199] Step 16:

[0200] The server receives a response from the medical institution, and if the reservation is approved, it returns a success message to the terminal. The input is the response from the medical institution, and the output is the success message.

[0201] Step 17:

[0202] On the day of the consultation, the server retransmits the received medical interview data to the medical institution, allowing the medical institution to conduct the consultation efficiently. The input is the medical interview data and reservation information, and the output is the preparation for the consultation at the medical institution.

[0203] Step 18:

[0204] After the consultation, the medical institution sends the consultation results and prescription information to the server, and the server notifies the user of the information. The input is the consultation results and prescription information, and the output is a notification to the user. This notification allows the user to check the consultation results and prescription information through the app.

[0205] (Application example 1)

[0206] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0207] In today's medical environment, the process from medical interview to examination is complicated, resulting in time-consuming and laborious tasks for both users and medical facilities. Furthermore, there are limited ways for users to conduct medical interviews and receive diagnostic predictions without visiting a medical institution, making it difficult to provide prompt and efficient medical services. Furthermore, methods for conducting medical interviews and diagnoses in virtual spaces are not widespread, preventing an improved user experience and hindering the improvement of the quality of medical services.

[0208] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0209] In this invention, the server includes means for receiving medical interview information input by a user, means for analyzing the received medical interview information and making a diagnosis prediction, means for providing medical record information generated based on the analysis results to a medical facility, means for receiving consultation appointment information and making a consultation appointment with the medical facility, means for notifying the user of the examination result information received from the medical facility, and means for the user to input medical interview information in a virtual store and collaborate with a virtual doctor. This enables the user to efficiently conduct a medical interview in a virtual space, quickly receive a diagnosis prediction, and, if necessary, make a consultation appointment.

[0210] "User" refers to an individual who uses medical services.

[0211] "Medical interview information" refers to information regarding symptoms and health conditions entered by the user.

[0212] "Means for receiving" refers to the function by which the server receives medical interview information and reservation information from the user.

[0213] "Means of analysis" refers to the function of using machine learning models and algorithms to make diagnostic predictions based on the received medical interview information.

[0214] "Diagnosis prediction" refers to the results of predicting the user's symptoms and condition based on medical interview information.

[0215] "Medical record information" refers to digital information that summarizes the user's health status and diagnostic predictions.

[0216] "Means for providing" refers to the function for transmitting the generated medical record information to a medical facility.

[0217] "Reservation information" refers to information regarding the date and time of the consultation desired by the user.

[0218] "Means for making appointments" refers to the function of making appointments with medical facilities based on the user's desired date and time.

[0219] "Means for notifying" refers to the function for transmitting medical examination result information received from medical facilities to users.

[0220] A "virtual store" refers to a virtual space accessible via the Internet where users can receive medical services.

[0221] "Virtual doctor" refers to a digital doctor who conducts medical interviews with users within a virtual store.

[0222] A "machine learning model" refers to an artificial intelligence (AI) algorithm used for data analysis and prediction.

[0223] An "interactive experience" refers to an experience that is provided through two-way interaction with the user.

[0224] This invention provides a system that enables users to efficiently conduct medical interviews in a virtual store, quickly receive a diagnosis, and, if necessary, schedule an appointment. The system has a series of functions that receive and analyze the medical interview information entered by the user, perform a diagnosis prediction, and provide medical record information to medical facilities.

[0225] The system consists of an application installed on the user's smartphone and a backend service running on a server. When the user enters medical history information through the application, the information is sent to the server. The server analyzes the received information and makes a diagnosis prediction using a machine learning model. The specific software used for this analysis is the latest machine learning libraries such as TensorFlow and PyTorch.

[0226] The server then generates a medical record based on the diagnosis prediction and provides it to the medical facility. This medical record details the user's symptoms and diagnosis prediction. When the user enters the desired consultation date and time, the reservation information is also sent to the server, and the appointment is completed through collaboration with the medical facility.

[0227] After the user receives a medical examination, the medical facility sends the examination result information to the server, which then notifies the user, who can then check the information through the application.

[0228] As a concrete example, the following scenario shows a user named Tom using the app to complete a headache interview and ultimately receive a diagnosis of migraine.

[0229] User: Tom, please enter your name and email.

[0230] System: Name: Tom

[0231] System: Email: tom@example.com

[0232] System: Please answer the following questions about your symptoms.

[0233] System: Question: How often do you experience headaches?

[0234] User: Almost daily.

[0235] System: Question: How severe are your headaches on a scale of 1 to 10?

[0236] User: 7

[0237] System: Analyzing your answers...

[0238] System: Based on your answers, there is a high likelihood of migraine. Would you like to book an appointment with a doctor?

[0239] User: Yes.

[0240] System: Booking an appointment on November 1, 2023...

[0241] System: Appointment confirmed. You will receive further instructions via email.

[0242] In this way, users can use a system that integrates medical interviews, diagnosis predictions, and appointment bookings in a virtual space, allowing them to receive efficient and prompt medical services.

[0243] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0244] Step 1:

[0245] A user launches the app and registers or logs in.

[0246] Specific operation: The device displays a new registration screen or login screen, and the user enters their name, email address, and password. The input information is sent to the server, which stores it in a database or authenticates it, and if authentication is successful, returns an access token to the device.

[0247] Input: User's name, email address, and password

[0248] Output: Access token

[0249] Step 2:

[0250] The user taps the "Start medical interview" button.

[0251] Specific operation: The terminal sends a request to start an interview to the server, and the server generates the first question and sends it to the terminal. The terminal displays the question to the user.

[0252] Input: Access token, medical interview start request

[0253] Output: First question

[0254] Step 3:

[0255] The user answers each question.

[0256] Specific operation: The user enters an answer, and the device sends the answer to the server. The server analyzes the answer, generates the next question, and returns it to the device. This process is repeated until all questions have been answered.

[0257] Input: User's answer

[0258] Output: Next question

[0259] Step 4:

[0260] The interview information is analyzed to generate a diagnostic prediction.

[0261] Specific operation: The server receives the answers to all questions, analyzes the user's medical interview data using a machine learning model, makes a diagnosis prediction, and generates the results as medical record information.

[0262] Input: All response data

[0263] Output: Diagnosis prediction, medical record information

[0264] Step 5:

[0265] Provide medical record information to medical facilities.

[0266] Specific operation: The server creates a request to send the medical record information including the generated diagnosis prediction to the medical facility, and sends it to the medical facility. The medical facility receives the medical record information and prepares it for use in consultations.

[0267] Input: Medical record information

[0268] Output: Medical record information sent

[0269] Step 6:

[0270] A user makes an appointment.

[0271] Specific operation: The user inputs the desired appointment date and time, and the terminal sends the reservation information to the server. The server then connects with the medical facility to check whether the appointment is available. The result is returned to the terminal, which then displays a reservation confirmation message to the user.

[0272] Input: Consultation appointment information

[0273] Output: Reservation confirmation message

[0274] Step 7:

[0275] Notify the user of the medical examination result information.

[0276] Specific operation: After the examination, the medical facility sends the examination result information to the server, which then notifies the user's device, where the user can check the examination result through the app.

[0277] Input: Examination result information

[0278] Output: Examination result notification

[0279] Through these steps, users can conduct a virtual medical interview, receive a diagnosis, and schedule an appointment if necessary.

[0280] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0281] The present invention is a system that uses a medical app to collect medical interview information and emotional data from users, analyzes this data to generate medical record information, and then collaborates with medical institutions. The program processing of this system is explained in natural language below. Specific examples are also provided for further explanation.

[0282] User Registration and Login

[0283] When a user uses the app for the first time, they tap the "New Registration" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the entered information to the server, which saves it in a database. Next, the user enters authentication information on the "Login" screen and taps the login button, which the device sends to the server. The server performs authentication and returns an access token to the device, completing the login.

[0284] Filling out a medical questionnaire and collecting emotional data

[0285] When the user taps the "Start medical interview" button, the device sends a request to start the medical interview to the server. The server generates the first question and returns it to the device, which then displays it to the user. The user then answers each question, which the device then sends to the server. The server analyzes the answers, determines the next question, and returns it to the device. During this process, the device also activates an emotion engine that analyzes the user's voice and facial expressions, collecting the user's emotional data.

[0286] Analysis of interview results and emotional data and reflection in medical records

[0287] After all interviews and emotional data have been collected, the server inputs this data into a machine learning model and begins analysis. The server obtains the analysis results and generates medical record information including a diagnosis prediction. The generated medical record information also includes information about the user's emotional state. The server creates a request to send the medical record information to the target medical institution and sends it to the target medical institution. The medical institution's system receives this medical record information and it is ready to be used for consultations.

[0288] Appointments and doctor coordination

[0289] When a user makes an appointment for a desired consultation date within the app, the device sends the appointment information to the server. The server then connects with the medical institution to confirm whether the appointment is possible and returns the result to the device. If the appointment is approved, the device displays a success message to the user. The medical institution prepares for an efficient consultation based on the medical interview data and emotion data received via the server. After the consultation, the medical institution sends the examination results and prescription information to the server, which then notifies the user. The user can then check the examination results through the app.

[0290] Specific examples

[0291] As an example, consider a case where a user complains of a migraine while feeling emotionally stressed. The user enters "I often get headaches" in the medical interview information and provides detailed responses about the frequency and other symptoms. The device uses an emotion engine to analyze the user's emotional state from their voice and facial expressions, detecting, for example, that the user is feeling stressed. The server receives the medical interview information and emotion data, and a machine learning model predicts that there is a high correlation between the possibility of "migraine" and "stress." The server generates a medical record including this diagnosis prediction and emotional state and sends it to the medical institution. By receiving this information in advance, the medical institution can quickly begin a consultation that takes into account not only the medical interview content but also the user's emotional state when they visit. After the consultation, the medical institution sends the examination results and prescription details to the server, allowing the user to check them through the app.

[0292] This will improve the efficiency of medical interviews and the accuracy of diagnoses, making the system beneficial for both users and medical institutions. Utilizing emotion data will enable deeper understanding and appropriate responses, improving the quality of patient care.

[0293] The processing flow will be explained below.

[0294] User Registration and Login

[0295] Step 1:

[0296] User: The user launches the app and taps the "Sign Up" button.

[0297] Step 2:

[0298] Device: The device will display the new registration screen.

[0299] Step 3:

[0300] User: The user enters their name, email address, and password, then taps the submit button.

[0301] Step 4:

[0302] Device: The device creates a request to send the input information to the server.

[0303] Step 5:

[0304] Server: The server receives the request and performs validation of the input information.

[0305] Step 6:

[0306] Server: If the server is happy with the information, it stores it in the database.

[0307] Step 7:

[0308] Server: The server returns a registration success response to the device.

[0309] Step 8:

[0310] Terminal: The terminal displays a successful registration message to the user.

[0311] Step 9:

[0312] User: The user enters their credentials on the "Login" screen and taps the login button.

[0313] Step 10:

[0314] Device: The device sends the entered authentication information to the server.

[0315] Step 11:

[0316] Server: The server verifies the authentication information, generates an access token, and returns a response to the device.

[0317] Step 12:

[0318] Device: The device stores the access token and uses it in subsequent requests.

[0319] Filling out a medical questionnaire and collecting emotional data

[0320] Step 1:

[0321] User: The user taps the "Start Interview" button.

[0322] Step 2:

[0323] Terminal: The terminal sends a request to start an interview to the server.

[0324] Step 3:

[0325] Server: The server generates the initial medical interview questions and returns them to the terminal.

[0326] Step 4:

[0327] Terminal: The terminal displays the initial question to the user.

[0328] Step 5:

[0329] User: The user answers the question and taps the Next button.

[0330] Step 6:

[0331] Device: The device makes a request that sends the answer to the server.

[0332] Step 7:

[0333] Server: The server stores the received answer in a database and generates the next question.

[0334] Step 8:

[0335] Device: At the same time, the device activates an emotion engine that analyzes the user's voice and facial expressions.

[0336] Step 9:

[0337] Emotion engine: The emotion engine collects the user's emotional data and returns the analysis results to the device.

[0338] Step 10:

[0339] Server: The server sends the following question to the device:

[0340] Step 11:

[0341] Terminal: The terminal displays the next question to the user. This process is repeated until all questions have been answered.

[0342] Analysis of interview results and emotional data and reflection in medical records

[0343] Step 1:

[0344] Server: Once all the questionnaire responses and emotional data have been collected, the server inputs the data into the machine learning model and begins analysis.

[0345] Step 2:

[0346] Server: The server obtains the analysis results and generates medical record information including diagnostic predictions.

[0347] Step 3:

[0348] Server: The generated medical record information also includes information about the user's emotional state.

[0349] Step 4:

[0350] Server: The server creates a request to send the medical record information to the medical institution.

[0351] Step 5:

[0352] Server: The server sends the medical record information to the target medical institution.

[0353] Step 6:

[0354] Medical institutions: Medical institutions receive the medical record information and use it for medical examinations.

[0355] Appointments and doctor coordination

[0356] Step 1:

[0357] User: The user opens the "Appointment" screen within the app and selects the desired appointment date and time.

[0358] Step 2:

[0359] Terminal: The terminal makes a request to send the reservation information to the server.

[0360] Step 3:

[0361] Server: The server receives the reservation request and checks whether the reservation is available by connecting with the medical institution's system.

[0362] Step 4:

[0363] Server: The server returns the reservation approval result to the terminal.

[0364] Step 5:

[0365] Terminal: The terminal displays a confirmation message to the user.

[0366] Step 6:

[0367] Medical institution: The medical institution receives the appointment information and prepares for the examination based on the user's medical interview data and emotion data.

[0368] Step 7:

[0369] Medical institution: After the examination, the medical institution sends the examination result information to the server.

[0370] Step 8:

[0371] Server: The server receives the medical result information and stores it in association with the user's account.

[0372] Step 9:

[0373] Device: The device will notify the user of the medical examination results and allow them to view them within the app.

[0374] The above are the specific operations in each processing step. In this way, the user can efficiently carry out the process from interviewing the patient to examining the patient, collecting emotional data, and checking the examination results.

[0375] Example 2

[0376] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0377] Conventional medical systems provide diagnostic predictions based solely on the user's medical interview information, and are unable to consider the user's emotional state. This makes it difficult to improve diagnostic accuracy and to properly reflect the impact of the user's emotions on the diagnosis. It is also difficult to provide a rapid examination and efficient appointment system that takes the user's emotional state into account.

[0378] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0379] In this invention, the server includes means for receiving medical interview information input by a user, means for analyzing the received medical interview information, means for providing medical institutions with medical record information generated based on the analysis results, means for receiving appointment information and making an appointment with the medical institution, means for notifying the user of medical examination result information received from the medical institution, means for collecting and analyzing emotional data of the user, and means for generating medical record information by integrating the medical interview information and emotional data. This enables highly accurate diagnosis prediction that integrates the user's medical interview information and emotional state, and realizes efficient and prompt examinations and appointments.

[0380] "User" refers to an individual or patient who uses the healthcare system.

[0381] A "terminal" is a hardware device used by a user, and refers to an electronic device such as a smartphone or tablet.

[0382] "Server" refers to a computer system that receives, processes, and stores data sent from users and terminals.

[0383] "Medical interview information" refers to data regarding health conditions and symptoms entered by the user.

[0384] "Emotional data" refers to data relating to the psychological state of a user that is analyzed from the user's voice, facial expressions, etc.

[0385] "Analysis" refers to the process of analyzing information and deriving certain results or predictions based on received data.

[0386] A "machine learning model" refers to an algorithm that learns patterns from large amounts of data and uses that learning to make predictions and judgments about new data.

[0387] "Medical record information" refers to information related to a medical diagnosis generated based on medical interview information and emotion data.

[0388] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0389] "Appointment for medical examination" refers to the act of a user reserving a date and time for a medical examination at a medical institution.

[0390] "Notification" refers to the act of sending information from a server to a user.

[0391] MODE FOR CARRYING OUT THE INVENTION

[0392] This invention relates to a system that collects and analyzes user interview information and emotional data to collaborate with medical institutions. The program processing of this system is described in detail below. The hardware and software used include smartphones and tablets as terminals, servers, databases, and machine learning models.

[0393] User Registration and Login

[0394] When a user uses the app for the first time, they tap the "Sign Up" button and the device displays the new registration screen. When the user enters their name, email address, and password and taps the "Submit" button, the device sends the entered data to the server. The server receives this data and stores it in a database (for example, MySQL). Next, when the user enters authentication information on the "Login" screen and taps the Login button, the device sends that information to the server. The server compares the authentication information with the database, and if it is correct, it generates an access token and returns it to the device, allowing the user to log in successfully.

[0395] Filling out a medical questionnaire and collecting emotional data

[0396] When the user taps the "Start Questionnaire" button, the device sends a request to start the questionnaire to the server. The server generates the first questions using Python (registered trademark)-based logic and returns them to the device. The device displays the questions to the user, and the user answers each question. The device sends the answers to the server, which analyzes the answers, generates the next question, and returns it to the device. During this process, the device launches an emotion analysis engine such as EmotionEngine or Affectiva to analyze the user's voice and facial expressions to collect emotional data.

[0397] Analysis of interview results and emotional data and reflection in medical records

[0398] Once all medical interview and emotion data collection is complete, the server receives this data. The server analyzes the data using machine learning models such as TensorFlow and PyTorch to obtain a diagnostic prediction. The server generates medical record information based on the diagnostic prediction. The generated medical record information includes the user's medical interview information and emotion data. The server provides the medical record information to medical institutions using the FHIR protocol, and the medical institutions prepare it so that it can be used for consultations.

[0399] Appointments and doctor coordination

[0400] The user selects the desired appointment date within the app and enters the appointment information. When the user taps the "Confirm appointment" button, the device sends that information to the server. The server communicates this information to the medical institution and checks whether the appointment is possible. The confirmation result is returned to the device, and if the appointment is approved, a success message is displayed to the user. The medical institution prepares for the appointment based on the interview data and emotion data received through the server. After the examination, the medical institution sends the examination results and prescription information to the server, which notifies the user. The user can check the examination results and prescription information through the app.

[0401] Specific examples

[0402] As an example, consider a case where a user complains of a migraine while feeling emotionally stressed. The user enters "I often get headaches" in the medical interview information and provides detailed information about the frequency and other symptoms. The device uses Emotion Engine to analyze the emotional state from the user's voice and facial expression and detects that the user is feeling stressed. The server receives the medical interview information and emotional data and uses a machine learning model to predict that there is a high correlation between "migraine" and "stress." The server generates medical record information based on this diagnosis prediction and emotional state and sends it to the medical institution. The medical institution receives this information in advance and makes preparations to respond quickly and appropriately when the user visits. After the examination, the medical institution sends the examination results and prescription information to the server, which the user can check through the app.

[0403] Examples of prompt statements

[0404] "The user complains of a headache, and you want to analyze their emotional state. Based on the interview information and emotional data, make a diagnosis prediction and generate a medical record."

[0405] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0406] User Registration and Login

[0407] Step 1:

[0408] When a user uses the app for the first time, they tap the "Sign Up" button. The inputs are the user's name, email address, and password. The output is a successful registration message.

[0409] Step 2:

[0410] The device displays the new registration screen, and the user enters their name, email address, and password and taps the "Submit" button. The input is the user's registration information. The output is the data to be sent to the server.

[0411] Step 3:

[0412] The terminal sends the entered data to the server. The input includes the user's registration information. The output is a message indicating successful data transmission to the server.

[0413] Step 4:

[0414] The server receives the data and stores it in a database. The input is the user's registration information. The output is the registered user information stored in the database.

[0415] Step 5:

[0416] The server returns a message to the terminal indicating that new registration has been completed. As input, there is information indicating successful registration. As output, a notification message is generated for the terminal.

[0417] Step 6:

[0418] The user taps the "Login" button. The input is the user's authentication information (email address, password). The output is a login request message.

[0419] Step 7:

[0420] The device displays the authentication information input screen, the user enters the information, and taps the "Login" button. The input is the user's authentication information. The output is authentication data to be sent to the server.

[0421] Step 8:

[0422] The server authenticates the received authentication information by checking it against a database. The input is the user's authentication information. The output is a success / failure message.

[0423] Step 9:

[0424] If authentication is successful, the server generates an access token and returns it to the terminal. The input is the user's successful authentication information. The output is the generated access token.

[0425] Step 10:

[0426] The user successfully logs in. The input is an access token. The output is a login successful message.

[0427] Filling out a medical questionnaire and collecting emotional data

[0428] Step 1:

[0429] The user taps the "Start medical interview" button. As input, there is a request to start medical interview. As output, a medical interview start request message is generated.

[0430] Step 2:

[0431] The terminal sends a consultation start request to the server. The input is the consultation start request data. The output is a message indicating that the transmission to the server was successful.

[0432] Step 3:

[0433] The server generates the initial question and returns it to the terminal. The input is a request to start the medical interview. The output is the medical interview question data.

[0434] Step 4:

[0435] The terminal displays the question to the user. The input is the question data. The output is a question screen that is displayed to the user.

[0436] Step 5:

[0437] The user answers the questions. The input is the user's answers to the medical interview questions. The output is the answer data.

[0438] Step 6:

[0439] The terminal sends the user's answer to the server. The input is the user's answer data. The output is a successful transmission message to the server.

[0440] Step 7:

[0441] The server analyzes the received answer and generates the next question. The input is the user's answer data. The output is the next question data.

[0442] Step 8:

[0443] The server returns the next question to the terminal, which then displays it to the user. The input is the next question data. The output is a question screen that is displayed to the user.

[0444] Step 9:

[0445] The device launches an emotion analysis engine such as EmotionEngine or Affectiva, and analyzes the user's voice and facial expressions to collect emotional data. The emotion analysis engine takes input, and emotional data is generated as output.

[0446] Step 10:

[0447] Send emotion data to the server. The emotion data is the input. The output is a message indicating that the transmission to the server was successful.

[0448] Analysis of interview results and emotional data and reflection in medical records

[0449] Step 1:

[0450] After all the questionnaire and emotion data collection is completed, the server receives these data. The inputs are the user's questionnaire data and emotion data. The output is stored in the server.

[0451] Step 2:

[0452] The server inputs the received data into a machine learning model such as TensorFlow or PyTorch and analyzes the data. The inputs include medical interview data and emotion data. The output is a diagnostic prediction.

[0453] Step 3:

[0454] Medical record information is generated based on the diagnostic prediction. The input is diagnostic prediction data. The output is medical record information.

[0455] Step 4:

[0456] The generated medical record information is provided to the medical institution using the FHIR protocol. The input is the medical record information. The output is a message indicating successful transmission to the medical institution.

[0457] Appointments and doctor coordination

[0458] Step 1:

[0459] The user selects the desired appointment date within the app and enters the reservation information. The input includes the desired appointment date and reservation information. The output is the reservation information.

[0460] Step 2:

[0461] When the user taps the "Confirm Reservation" button, the device sends the information to the server. The input is the reservation information. The output is a message indicating that the transmission to the server was successful.

[0462] Step 3:

[0463] The server sends the reservation information to the medical institution and checks whether the reservation is possible. The input is the reservation information. The output is reservation confirmation information.

[0464] Step 4:

[0465] The server returns the confirmation result to the terminal. The input is the reservation confirmation information. The output is a transmission success message to the terminal.

[0466] Step 5:

[0467] If the reservation is approved, the terminal displays a success message to the user. The input is the reservation approval information. The output is a success message.

[0468] Step 6:

[0469] The medical institution prepares for the examination based on the medical interview data and emotion data received through the server. The inputs are the medical interview data and emotion data. The output is the examination preparation.

[0470] Step 7:

[0471] After the consultation, the medical institution sends the consultation results and prescription information to the server. The inputs are the consultation results and prescription information. The output is a message indicating that the transmission to the server was successful.

[0472] Step 8:

[0473] The server notifies the user. The inputs are the medical examination results and prescription information. The output is a notification message for the user.

[0474] Step 9:

[0475] The user checks the medical examination results and prescription information through the app. The input is a notification message. The output is the medical examination results and prescription information.

[0476] (Application example 2)

[0477] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0478] Current medical systems and security service systems do not fully utilize user interview information and emotional data, making it difficult to provide appropriate diagnoses and propose security measures. Furthermore, few systems take into consideration the user's stress or anxiety, which can lead to a decline in user satisfaction. Furthermore, collaboration between medical institutions and security experts is not smooth, making it difficult to provide efficient responses.

[0479] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving medical interview information and emotion data entered by the user; means for analyzing the received medical interview information and emotion data; means for providing medical record information generated based on the analysis results to the medical institution; means for receiving appointment information and making an appointment with the medical institution; means for notifying the user of the examination result information received from the medical institution; means for inputting a security diagnosis form and collecting emotion data; means for analyzing the collected data using a machine learning model and generating a security diagnosis report; means for proposing security measures to the user based on the analysis results; and means for realizing collaboration with security experts. This enables efficient diagnosis and provision of security measures that take the user's emotional state into consideration.

[0480] "Medical interview information" refers to data about symptoms and health conditions that are entered by the user themselves.

[0481] "Emotion data" refers to information about the user's emotional state that is analyzed from the user's tone of voice, facial expression, and so on.

[0482] "Analysis" refers to making diagnostic predictions and risk assessments based on collected interview information and emotional data.

[0483] "Medical record information" refers to data generated based on analysis results and provided to medical institutions, including diagnostic predictions and emotional states.

[0484] A "medical institution" is a facility that provides medical examinations and treatment, such as a hospital or clinic.

[0485] "Reservation information" refers to data on the consultation date and time desired by the user.

[0486] "Consultation appointment" refers to the procedure for reserving a date and time for consultation with a medical institution.

[0487] "Examination result information" refers to information generated by a medical institution and including examination results, prescription details, and the like.

[0488] The "security assessment form" is a list of questions to assess the user's security status.

[0489] A "security diagnostic report" is a document that evaluates a user's security status based on collected data and suggests risk levels and countermeasures.

[0490] A "security expert" is someone with specialized knowledge in the field of security.

[0491] A "machine learning model" is an algorithm that learns patterns and features based on data and makes predictions and diagnoses.

[0492] The system of this invention is a mechanism for collecting medical interview information and emotion data entered by the user, analyzing this information, generating medical record information and security diagnostic reports, and realizing collaboration with medical institutions and security experts. Specific program processing and the hardware and software used to realize this are described below.

[0493] User Registration and Login

[0494] When a user uses the system for the first time, they tap the "New Registration" button on their smartphone, tablet, or other device, enter their registration information, and send it to the server. This information is stored in a database such as Amazon RDS. When the user uses the system again, they enter their authentication information on the login screen and the server authenticates them before they can access the system.

[0495] Filling out a medical questionnaire and collecting emotional data

[0496] When a user taps the "Start Questionnaire" button while using the system, the server receives the request to start the questionnaire, generates the first question, and returns it to the device. The user answers the question, and the answer data is sent back to the server. During this process, the device uses libraries such as OpenCV and Dlib to analyze the user's voice and facial expressions and collect emotional data. This data is used for subsequent analysis.

[0497] Analysis of interview results and emotional data and reflection in medical records

[0498] Once all interviews and emotional data collection is complete, the server inputs this data into a machine learning model (e.g., scikit-learn, TensorFlow, PyTorch) for analysis. The generated medical record information also includes information about the user's emotional state. The server then sends this medical record information to the medical institution and prepares it for use in the consultation.

[0499] Appointments and doctor coordination

[0500] When a user makes a reservation for a desired consultation date, the information is sent to the medical institution via the server, and the reservation is confirmed. If the reservation is approved, a success message is displayed on the terminal. The medical institution uses this information to efficiently prepare for the consultation.

[0501] Fill out a security assessment form and collect emotional data

[0502] When a user taps the "Start Security Assessment" button using the system, the server receives the assessment start request, generates an initial set of questions, and returns them to the device. The user answers the questions, and the answer data is sent back to the server. During this process, the device uses libraries such as OpenCV and Dlib to analyze the user's voice and facial expressions and collect emotional data. This data is used for subsequent analysis.

[0503] Analysis of security diagnostic results and sentiment data and report generation

[0504] Once all interviews and emotional data collection is complete, the server inputs this data into a machine learning model (e.g., scikit-learn, TensorFlow, PyTorch) for analysis. The generated security assessment report includes information about the user's emotional state. The server provides this report to the user and allows them to connect with security experts if necessary.

[0505] Examples of specific examples and prompts

[0506] For example, if a user answers the questionnaire information by saying, "I change my password every six months. I don't use two-factor authentication," and the emotional data indicates that they are in a state of moderate stress, the system will assess them as "medium risk" and suggest the following countermeasures: specifically, "enable two-factor authentication," "use a password manager," and "regular data backup."

[0507] Prompt for the generative AI model:

[0508] Please conduct a security assessment based on the following user data and provide the risk level and specific measures.

[0509] User Data:

[0510] Answer: "I change my password every six months" and "I don't use two-factor authentication"

[0511] Emotional data: Slightly stressed

[0512] This program and mechanism makes it possible to provide efficient security measures that take into account the user's emotional state.

[0513] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0514] Step 1:

[0515] When a user uses the app for the first time, they tap the "Sign Up" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the input information to the server, which stores it in a database. This creates a user account for authentication.

[0516] Step 2:

[0517] The user enters authentication information (email address, password) on the login screen and taps the login button. The device sends the authentication information to the server. The server compares it with the information in the database, and if authentication is successful, generates an access token and returns it to the device. The device saves this access token and uses it in subsequent communications.

[0518] Step 3:

[0519] When the user taps the "Start medical interview" button, the device sends a medical interview start request to the server. The server generates a medical interview form and returns the initial questions to the device. The device displays the questions to the user and provides an interface for inputting answers.

[0520] Step 4:

[0521] The user enters their answers into the questionnaire form and taps the send button. The device then sends the answer data to the server. At that time, the device uses a camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional data. The analysis results are also sent to the server.

[0522] Step 5:

[0523] The server analyzes the medical interview information and emotional data it receives. Specifically, it uses machine learning models (such as scikit-learn, TensorFlow, and PyTorch) to predict diagnoses and generate analysis results, including the user's emotional state. It then generates medical record information based on the analysis results and prepares the data for transmission to medical institutions.

[0524] Step 6:

[0525] The server sends the generated medical record information to the designated medical institution, which receives the information and prepares for diagnosis and treatment.

[0526] Step 7:

[0527] When a user makes a reservation for a desired consultation date within the app, the device sends the reservation information to the server. The server then contacts the medical institution to confirm whether the reservation can be made and returns the result to the device. If the reservation is approved, the device displays a success message to the user.

[0528] Step 8:

[0529] After the consultation, the medical institution sends the consultation results and prescription information to the server, which then notifies the user, who can then check the consultation results through the app.

[0530] Step 9:

[0531] When the user taps the "Start Security Assessment" button, the device sends a request to the server to start the assessment. The server generates an assessment form and returns the initial questions to the device. The device then displays the questions to the user and provides an interface for entering answers.

[0532] Step 10:

[0533] The user enters their answers into the security diagnostic form and taps the submit button. The device then sends the answer data to the server. At that time, the device uses the camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional data. The analysis results are also sent to the server.

[0534] Step 11:

[0535] The server analyzes the security diagnostic information and emotion data received, performs security diagnostics using machine learning models and evaluates the risk level, generates a security diagnostic report based on the analysis results, and prepares the data to provide to the user.

[0536] Step 12:

[0537] Provides the user with the server-generated security assessment report and suggests countermeasures. If necessary, creates a request to connect the user with a security expert and submits it with the appropriate information.

[0538] The above processing steps make it possible to provide efficient diagnostic and security measures that take into account the emotional state of the user.

[0539] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0540] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0541] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0542] [Second embodiment]

[0543] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0544] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0545] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0546] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0547] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0548] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0549] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0550] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0551] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0552] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0553] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0554] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0555] The present invention is a system that uses a medical app to collect medical interview information from users, analyzes it to generate medical record information, and then connects with medical institutions. The program processing of this system is explained in natural language below. Specific examples are also provided for further explanation.

[0556] User Registration and Login

[0557] When a user uses the app for the first time, they tap the "New Registration" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the entered information to the server, which saves it in a database. Next, the user enters authentication information on the "Login" screen and taps the login button, which the device sends to the server. The server performs authentication and returns an access token to the device, completing the login.

[0558] Fill out the medical questionnaire form

[0559] When the user taps the "Start medical interview" button, the device sends a medical interview start request to the server. The server generates the first question and returns it to the device, which then displays it to the user. After that, the user answers each question, and the device sends them to the server. The server analyzes the answers, determines the next question, and returns it to the device. This process is repeated until all questions have been asked.

[0560] Analysis of interview results and reflection in medical records

[0561] Once all medical interviews are completed, the server analyzes the user's interview data using a machine learning model and makes a diagnosis prediction. The server then creates a request to send the medical record information, including the generated diagnosis prediction, to the target medical institution. The medical institution's system receives this medical record information and prepares it for use in consultations.

[0562] Appointments and doctor coordination

[0563] When a user makes an appointment for a desired consultation date within the app, the device sends the appointment information to the server. The server then connects with the medical institution to confirm whether the appointment is possible and returns the result to the device. If the appointment is approved, the device displays a success message to the user. The medical institution prepares to efficiently conduct the consultation based on the medical interview data received via the server. After the consultation, the medical institution sends the examination results and prescription information to the server, which then notifies the user. The user can then check the examination results through the app.

[0564] Specific examples

[0565] As an example, consider the case where a user complaining of migraines uses this system. The user enters "I often get headaches" in the medical interview information and provides detailed information about the frequency and other symptoms. The server receives this information, and the machine learning model predicts that the diagnosis is likely "migraine." The server generates medical record information including this diagnosis prediction and sends it to the medical institution. By receiving this information in advance, the medical institution can quickly begin consultation when the user visits the hospital without having to reconfirm the medical interview details. After the consultation, the medical institution sends the examination results and prescription details to the server, allowing the user to check them through the app.

[0566] This will improve the efficiency of medical interviews and the accuracy of diagnoses, making the system beneficial for both users and medical institutions.

[0567] The processing flow will be explained below.

[0568] User Registration and Login

[0569] Step 1:

[0570] User: The user launches the app and taps the "Sign Up" button.

[0571] Step 2:

[0572] Device: The device will display the new registration screen.

[0573] Step 3:

[0574] User: The user enters their name, email address, and password, then taps the submit button.

[0575] Step 4:

[0576] Device: The device creates a request to send the input information to the server.

[0577] Step 5:

[0578] Server: The server receives the request and performs validation of the input information.

[0579] Step 6:

[0580] Server: If the server is happy with the information, it stores it in the database.

[0581] Step 7:

[0582] Server: The server returns a registration success response to the device.

[0583] Step 8:

[0584] Terminal: The terminal displays a successful registration message to the user.

[0585] Step 9:

[0586] User: The user enters their credentials on the "Login" screen and taps the login button.

[0587] Step 10:

[0588] Device: The device sends the entered authentication information to the server.

[0589] Step 11:

[0590] Server: The server verifies the authentication information, generates an access token, and returns a response to the device.

[0591] Step 12:

[0592] Device: The device stores the access token and uses it in subsequent requests.

[0593] Fill out the medical questionnaire form

[0594] Step 1:

[0595] User: The user taps the "Start Interview" button.

[0596] Step 2:

[0597] Terminal: The terminal sends a request to start an interview to the server.

[0598] Step 3:

[0599] Server: The server generates the initial medical interview questions and returns them to the terminal.

[0600] Step 4:

[0601] Terminal: The terminal displays the initial question to the user.

[0602] Step 5:

[0603] User: The user answers the question and taps the Next button.

[0604] Step 6:

[0605] Device: The device sends the answer to the server.

[0606] Step 7:

[0607] Server: The server stores the received answer in a database and generates the next question.

[0608] Step 8:

[0609] Server: The server sends the following question to the device:

[0610] Step 9:

[0611] Terminal: The terminal displays the next question to the user. This process is repeated until all questions have been answered.

[0612] Analysis of interview results and reflection in medical records

[0613] Step 1:

[0614] Server: After all questionnaire responses are completed, the server inputs the questionnaire data into the machine learning model and begins analysis.

[0615] Step 2:

[0616] Server: The server obtains the analysis results and generates medical record information including diagnostic predictions.

[0617] Step 3:

[0618] Server: The server creates a request to send the medical record information to the medical institution.

[0619] Step 4:

[0620] Server: The server sends the medical record information to the target medical institution.

[0621] Step 5:

[0622] Medical institutions: Medical institutions receive the medical record information and use it for medical examinations.

[0623] Appointments and doctor coordination

[0624] Step 1:

[0625] User: The user opens the "Appointment" screen within the app and selects the desired appointment date and time.

[0626] Step 2:

[0627] Terminal: The terminal makes a request to send the reservation information to the server.

[0628] Step 3:

[0629] Server: The server receives the reservation request and checks whether the reservation is available by connecting with the medical institution's system.

[0630] Step 4:

[0631] Server: The server returns the reservation approval result to the terminal.

[0632] Step 5:

[0633] Terminal: The terminal displays a confirmation message to the user.

[0634] Step 6:

[0635] Medical institution: The medical institution receives the reservation information and prepares for the examination based on the user's medical interview data.

[0636] Step 7:

[0637] Medical institution: After the examination, the medical institution sends the examination result information to the server.

[0638] Step 8:

[0639] Server: The server receives the medical result information and stores it in association with the user's account.

[0640] Step 9:

[0641] Device: The device will notify the user of the medical examination results and allow them to view them within the app.

[0642] The above is a detailed description of the operations performed in each processing step. In this way, the user can efficiently carry out the process from interviewing the patient to examining the patient and confirming the examination results.

[0643] Example 1

[0644] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0645] Modern medical care involves many processes, such as collecting and analyzing medical history information, scheduling appointments, and notifying patients of diagnosis results. Each process requires time and effort, creating a challenge. Furthermore, doctors' time spent comprehending detailed medical history information during consultations reduces the efficiency of the entire consultation. Conventional systems have struggled to manage these processes in an integrated manner and process them efficiently and accurately.

[0646] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0647] In this invention, the server includes a means for receiving medical interview information input by a user, a means for analyzing the received medical interview information, a means for providing electronic medical record information generated based on the analysis results to a medical institution, a means for receiving appointment information and making an appointment with the medical institution, and a means for notifying the user of the examination result information received from the medical institution. This enables efficient collection and analysis of medical interview information, prompt notification of diagnosis results, and efficient examinations through cooperation with medical institutions.

[0648] "User" refers to an individual or corporation that uses the system.

[0649] "Medical interview information" refers to information about health conditions and symptoms provided by the user.

[0650] "Electronic medical record information" refers to health records generated from medical interview information and analysis results.

[0651] "Medical institution" refers to an organization such as a hospital or clinic that provides medical care or treatment.

[0652] "Appointment for medical examination" refers to the act of reserving a desired date and time for medical examination at a medical institution by a user.

[0653] "Examination result information" refers to information regarding diagnoses and prescriptions provided by medical institutions after examinations.

[0654] A "server" is a computer device that is the core of a system and that processes and stores data and communicates with other devices.

[0655] A "machine learning model" refers to an algorithm that learns from large amounts of data and makes predictions and classifications.

[0656] "Notification" refers to a means by which the system notifies the user of information.

[0657] "Receive" refers to the act of obtaining data from another device or system.

[0658] "Analysis" refers to the act of examining information in detail based on received data and understanding its meaning and structure.

[0659] The present invention is a system that uses a medical application to collect medical interview information from users, analyzes the information, generates electronic medical records, and collaborates with medical institutions. This system consists of a server, terminals, and users.

[0660] Program processing explanation

[0661] User Registration and Login

[0662] When a user launches the app and taps the "New Registration" button, the device displays the new registration screen. When the user enters their name, email address, and password and taps the "Submit" button, the device sends the input information to the server. The server stores the received information in a database.

[0663] When a user enters their email address and password on the "Login" screen and taps the login button, the device sends the entered information to the server. The server performs authentication and returns an access token to the device, completing the login. The technology used is a smartphone or tablet as the device, and a cloud service (e.g., Amazon Web Services, Microsoft Azure) as the server.

[0664] Fill out the medical questionnaire form

[0665] When the user taps the "Start medical interview" button, the device sends a request to the server to start the medical interview. The server generates the first questions and returns them to the device. The device displays the questions, and the user answers each one. When the device sends the answers to the server, the server analyzes the answers and determines the next question. This process is repeated until all questions have been asked.

[0666] The technologies used include stream processing using Apache Kafka for real-time analysis on the server side, and natural language processing (NLP) libraries implemented in Python (e.g., SpaCy, NLTK).

[0667] Analysis of interview results and reflection in medical records

[0668] Once all interviews are completed, the device sends an end request to the server. The server inputs the interview data into a machine learning model for analysis. Specific technologies used include Scikit-Learn and TensorFlow. The server generates a diagnostic prediction and formats it as electronic medical record information. The server then creates and sends a request to send this electronic medical record information to the medical institution.

[0669] The medical institution's system receives electronic medical record information and prepares it so that it can be used for consultations. The technology used utilizes the standard FHIR (Fast Healthcare Interoperability Resources).

[0670] Appointments and doctor coordination

[0671] When a user makes a reservation for a desired consultation date within the app, the device sends the reservation information to the server. The server then connects with the medical institution to confirm whether the reservation is possible. If the reservation is approved, the device displays a success message to the user.

[0672] On the day of the consultation, the medical institution will prepare for the consultation efficiently based on the medical interview data received through the server. After the consultation, the medical institution will send the consultation results and prescription information to the server, which will notify the user. The user can then check the consultation results through the application.

[0673] As a specific example of how it works, for example, a user who complains of migraines uses the system and enters "I often get headaches" into the medical interview information, and provides detailed information about the frequency and other symptoms. Based on this information, the server uses a machine learning model to predict the possibility of a "migraine," generates electronic medical record information, and sends it to the medical institution. By receiving this information in advance, the medical institution can prepare for a prompt examination.

[0674] Prompt Sentence Examples

[0675] Here are some example prompts to input to a generative AI model:

[0676] "What API endpoints do you use to process user registration?"

[0677] "Please explain how to analyze medical interview data using a machine learning model."

[0678] "Please tell me about the process for sending medical record information according to the FHIR standard."

[0679] In this way, this system uses technically linked components to improve the efficiency of medical interviews and the accuracy of diagnoses.

[0680] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0681] System program processing steps

[0682] User Registration and Login

[0683] Step 1:

[0684] The user launches the app and taps the "New Registration" button. The device displays the new registration screen. The user enters their name, email address, and password, and taps the send button. The input is their name, email address, and password, and this information is sent from the device to the server as output.

[0685] Step 2:

[0686] The server stores the received information in a database. In this storage process, an INSERT query is executed using a database management system (e.g., MySQL) to record the user information. The input is the user information received from the terminal, and the output is the result stored in the database.

[0687] Step 3:

[0688] The user enters their email address and password on the "Login" screen and taps the login button. The device sends the authentication information to the server. The input is the email address and password, and the output is this information sent from the device to the server.

[0689] Step 4:

[0690] The server references the database and checks the authentication information. If authentication is successful, it generates a JWT (JSON Web Token) and returns it to the terminal as an access token. If authentication fails, it generates an error message and returns it to the terminal. The input is authentication information, and the output is an access token or an error message.

[0691] Fill out the medical questionnaire form

[0692] Step 5:

[0693] The user taps the "Start medical interview" button. The terminal sends a medical interview start request to the server. The input is the medical interview start request, and the output is that this request is sent from the terminal to the server.

[0694] Step 6:

[0695] The server generates the initial question and returns it to the terminal. At this time, the server executes an algorithm that randomly selects a question from a list of questions prepared in advance. The input is the request to start the medical interview, and the output is the initial question.

[0696] Step 7:

[0697] The terminal displays a question, the user answers the question, the input is the initial question, and the output is the user's answer.

[0698] Step 8:

[0699] The device sends the user's answer to the server. The server analyzes the received answer using an NLP algorithm (e.g., SpaCy, NLTK). Based on the analysis results, the next question is dynamically generated. The input is the user's answer, and the output is the next question.

[0700] Step 9:

[0701] The server generates the next question and sends it back to the terminal, which displays it to the user. This process is repeated until all questions have been asked. The input is the next question, and the output is that question will be displayed on the terminal.

[0702] Analysis of interview results and reflection in medical records

[0703] Step 10:

[0704] When all the inquiries are completed, the terminal sends an end request to the server. The input is the inquiry end request, and the output is that the request is sent from the terminal to the server.

[0705] Step 11:

[0706] The server inputs the received medical interview data into a machine learning model (e.g., Scikit-Learn, TensorFlow) for analysis. The machine learning model generates a diagnostic prediction. The input is the medical interview data, and the output is a diagnostic prediction.

[0707] Step 12:

[0708] The server formalizes the generated diagnosis prediction as electronic medical record information. Then, a request including this electronic medical record information is sent to the medical institution. The input is the diagnosis prediction, and the output is the electronic medical record information.

[0709] Step 13:

[0710] The medical institution's system receives the electronic medical record information and prepares it so that it can be used for consultations. The input is the electronic medical record information, and the output is the completion of consultation preparation.

[0711] Appointments and doctor coordination

[0712] Step 14:

[0713] The user makes a reservation for the desired consultation date within the app, and the device sends the reservation information to the server. The input is the reservation information, and the output is that information is sent from the device to the server.

[0714] Step 15:

[0715] The server receives the reservation information and sends a request to the medical institution to confirm whether the reservation is available. The input is the reservation information, and the output is confirmation of whether the reservation is available.

[0716] Step 16:

[0717] The server receives a response from the medical institution, and if the reservation is approved, it returns a success message to the terminal. The input is the response from the medical institution, and the output is the success message.

[0718] Step 17:

[0719] On the day of the consultation, the server retransmits the received medical interview data to the medical institution, allowing the medical institution to conduct the consultation efficiently. The input is the medical interview data and reservation information, and the output is the preparation for the consultation at the medical institution.

[0720] Step 18:

[0721] After the consultation, the medical institution sends the consultation results and prescription information to the server, and the server notifies the user of the information. The input is the consultation results and prescription information, and the output is a notification to the user. This notification allows the user to check the consultation results and prescription information through the app.

[0722] (Application example 1)

[0723] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0724] In today's medical environment, the process from medical interview to examination is complicated, resulting in time-consuming and laborious tasks for both users and medical facilities. Furthermore, there are limited ways for users to conduct medical interviews and receive diagnostic predictions without visiting a medical institution, making it difficult to provide prompt and efficient medical services. Furthermore, methods for conducting medical interviews and diagnoses in virtual spaces are not widespread, preventing an improved user experience and hindering the improvement of the quality of medical services.

[0725] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0726] In this invention, the server includes means for receiving medical interview information input by a user, means for analyzing the received medical interview information and making a diagnosis prediction, means for providing medical record information generated based on the analysis results to a medical facility, means for receiving consultation appointment information and making a consultation appointment with the medical facility, means for notifying the user of the examination result information received from the medical facility, and means for the user to input medical interview information in a virtual store and collaborate with a virtual doctor. This enables the user to efficiently conduct a medical interview in a virtual space, quickly receive a diagnosis prediction, and, if necessary, make a consultation appointment.

[0727] "User" refers to an individual who uses medical services.

[0728] "Medical interview information" refers to information regarding symptoms and health conditions entered by the user.

[0729] "Means for receiving" refers to the function by which the server receives medical interview information and reservation information from the user.

[0730] "Means of analysis" refers to the function of using machine learning models and algorithms to make diagnostic predictions based on the received medical interview information.

[0731] "Diagnosis prediction" refers to the results of predicting the user's symptoms and condition based on medical interview information.

[0732] "Medical record information" refers to digital information that summarizes the user's health status and diagnostic predictions.

[0733] "Means for providing" refers to the function for transmitting the generated medical record information to a medical facility.

[0734] "Reservation information" refers to information regarding the date and time of the consultation desired by the user.

[0735] "Means for making appointments" refers to the function of making appointments with medical facilities based on the user's desired date and time.

[0736] "Means for notifying" refers to the function for transmitting medical examination result information received from medical facilities to users.

[0737] A "virtual store" refers to a virtual space accessible via the Internet where users can receive medical services.

[0738] "Virtual doctor" refers to a digital doctor who conducts medical interviews with users within a virtual store.

[0739] A "machine learning model" refers to an artificial intelligence (AI) algorithm used for data analysis and prediction.

[0740] An "interactive experience" refers to an experience that is provided through two-way interaction with the user.

[0741] This invention provides a system that enables users to efficiently conduct medical interviews in a virtual store, quickly receive a diagnosis, and, if necessary, schedule an appointment. The system has a series of functions that receive and analyze the medical interview information entered by the user, perform a diagnosis prediction, and provide medical record information to medical facilities.

[0742] The system consists of an application installed on the user's smartphone and a backend service running on a server. When the user enters medical history information through the application, the information is sent to the server. The server analyzes the received information and makes a diagnosis prediction using a machine learning model. The specific software used for this analysis is the latest machine learning libraries such as TensorFlow and PyTorch.

[0743] The server then generates a medical record based on the diagnosis prediction and provides it to the medical facility. This medical record details the user's symptoms and diagnosis prediction. When the user enters the desired consultation date and time, the reservation information is also sent to the server, and the appointment is completed through collaboration with the medical facility.

[0744] After the user receives a medical examination, the medical facility sends the examination result information to the server, which then notifies the user, who can then check the information through the application.

[0745] As a concrete example, the following scenario shows a user named Tom using the app to complete a headache interview and ultimately receive a diagnosis of migraine.

[0746] User: Tom, please enter your name and email.

[0747] System: Name: Tom

[0748] System: Email: tom@example.com

[0749] System: Please answer the following questions about your symptoms.

[0750] System: Question: How often do you experience headaches?

[0751] User: Almost daily.

[0752] System: Question: How severe are your headaches on a scale of 1 to 10?

[0753] User: 7

[0754] System: Analyzing your answers...

[0755] System: Based on your answers, there is a high likelihood of migraine. Would you like to book an appointment with a doctor?

[0756] User: Yes.

[0757] System: Booking an appointment on November 1, 2023...

[0758] System: Appointment confirmed. You will receive further instructions via email.

[0759] In this way, users can use a system that integrates medical interviews, diagnosis predictions, and appointment bookings in a virtual space, allowing them to receive efficient and prompt medical services.

[0760] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0761] Step 1:

[0762] A user launches the app and registers or logs in.

[0763] Specific operation: The device displays a new registration screen or login screen, and the user enters their name, email address, and password. The input information is sent to the server, which stores it in a database or authenticates it, and if authentication is successful, returns an access token to the device.

[0764] Input: User's name, email address, and password

[0765] Output: Access token

[0766] Step 2:

[0767] The user taps the "Start medical interview" button.

[0768] Specific operation: The terminal sends a request to start an interview to the server, and the server generates the first question and sends it to the terminal. The terminal displays the question to the user.

[0769] Input: Access token, medical interview start request

[0770] Output: First question

[0771] Step 3:

[0772] The user answers each question.

[0773] Specific operation: The user enters an answer, and the device sends the answer to the server. The server analyzes the answer, generates the next question, and returns it to the device. This process is repeated until all questions have been answered.

[0774] Input: User's answer

[0775] Output: Next question

[0776] Step 4:

[0777] The interview information is analyzed to generate a diagnostic prediction.

[0778] Specific operation: The server receives the answers to all questions, analyzes the user's medical interview data using a machine learning model, makes a diagnosis prediction, and generates the results as medical record information.

[0779] Input: All response data

[0780] Output: Diagnosis prediction, medical record information

[0781] Step 5:

[0782] Provide medical record information to medical facilities.

[0783] Specific operation: The server creates a request to send the medical record information including the generated diagnosis prediction to the medical facility, and sends it to the medical facility. The medical facility receives the medical record information and prepares it for use in consultations.

[0784] Input: Medical record information

[0785] Output: Medical record information sent

[0786] Step 6:

[0787] A user makes an appointment.

[0788] Specific operation: The user inputs the desired appointment date and time, and the terminal sends the reservation information to the server. The server then connects with the medical facility to check whether the appointment is available. The result is returned to the terminal, which then displays a reservation confirmation message to the user.

[0789] Input: Consultation appointment information

[0790] Output: Reservation confirmation message

[0791] Step 7:

[0792] Notify the user of the medical examination result information.

[0793] Specific operation: After the examination, the medical facility sends the examination result information to the server, which then notifies the user's device, where the user can check the examination result through the app.

[0794] Input: Examination result information

[0795] Output: Examination result notification

[0796] Through these steps, users can conduct a virtual medical interview, receive a diagnosis, and schedule an appointment if necessary.

[0797] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0798] The present invention is a system that uses a medical app to collect medical interview information and emotional data from users, analyzes this data to generate medical record information, and then collaborates with medical institutions. The program processing of this system is explained in natural language below. Specific examples are also provided for further explanation.

[0799] User Registration and Login

[0800] When a user uses the app for the first time, they tap the "New Registration" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the entered information to the server, which saves it in a database. Next, the user enters authentication information on the "Login" screen and taps the login button, which the device sends to the server. The server performs authentication and returns an access token to the device, completing the login.

[0801] Filling out a medical questionnaire and collecting emotional data

[0802] When the user taps the "Start medical interview" button, the device sends a request to start the medical interview to the server. The server generates the first question and returns it to the device, which then displays it to the user. The user then answers each question, which the device then sends to the server. The server analyzes the answers, determines the next question, and returns it to the device. During this process, the device also activates an emotion engine that analyzes the user's voice and facial expressions, collecting the user's emotional data.

[0803] Analysis of interview results and emotional data and reflection in medical records

[0804] After all interviews and emotional data have been collected, the server inputs this data into a machine learning model and begins analysis. The server obtains the analysis results and generates medical record information including a diagnosis prediction. The generated medical record information also includes information about the user's emotional state. The server creates a request to send the medical record information to the target medical institution and sends it to the target medical institution. The medical institution's system receives this medical record information and it is ready to be used for consultations.

[0805] Appointments and doctor coordination

[0806] When a user makes an appointment for a desired consultation date within the app, the device sends the appointment information to the server. The server then connects with the medical institution to confirm whether the appointment is possible and returns the result to the device. If the appointment is approved, the device displays a success message to the user. The medical institution prepares for an efficient consultation based on the medical interview data and emotion data received via the server. After the consultation, the medical institution sends the examination results and prescription information to the server, which then notifies the user. The user can then check the examination results through the app.

[0807] Specific examples

[0808] As an example, consider a case where a user complains of a migraine while feeling emotionally stressed. The user enters "I often get headaches" in the medical interview information and provides detailed responses about the frequency and other symptoms. The device uses an emotion engine to analyze the user's emotional state from their voice and facial expressions, detecting, for example, that the user is feeling stressed. The server receives the medical interview information and emotion data, and a machine learning model predicts that there is a high correlation between the possibility of "migraine" and "stress." The server generates a medical record including this diagnosis prediction and emotional state and sends it to the medical institution. By receiving this information in advance, the medical institution can quickly begin a consultation that takes into account not only the medical interview content but also the user's emotional state when they visit. After the consultation, the medical institution sends the examination results and prescription details to the server, allowing the user to check them through the app.

[0809] This will improve the efficiency of medical interviews and the accuracy of diagnoses, making the system beneficial for both users and medical institutions. Utilizing emotion data will enable deeper understanding and appropriate responses, improving the quality of patient care.

[0810] The processing flow will be explained below.

[0811] User Registration and Login

[0812] Step 1:

[0813] User: The user launches the app and taps the "Sign Up" button.

[0814] Step 2:

[0815] Device: The device will display the new registration screen.

[0816] Step 3:

[0817] User: The user enters their name, email address, and password, then taps the submit button.

[0818] Step 4:

[0819] Device: The device creates a request to send the input information to the server.

[0820] Step 5:

[0821] Server: The server receives the request and performs validation of the input information.

[0822] Step 6:

[0823] Server: If the server is happy with the information, it stores it in the database.

[0824] Step 7:

[0825] Server: The server returns a registration success response to the device.

[0826] Step 8:

[0827] Terminal: The terminal displays a successful registration message to the user.

[0828] Step 9:

[0829] User: The user enters their credentials on the "Login" screen and taps the login button.

[0830] Step 10:

[0831] Device: The device sends the entered authentication information to the server.

[0832] Step 11:

[0833] Server: The server verifies the authentication information, generates an access token, and returns a response to the device.

[0834] Step 12:

[0835] Device: The device stores the access token and uses it in subsequent requests.

[0836] Filling out a medical questionnaire and collecting emotional data

[0837] Step 1:

[0838] User: The user taps the "Start Interview" button.

[0839] Step 2:

[0840] Terminal: The terminal sends a request to start an interview to the server.

[0841] Step 3:

[0842] Server: The server generates the initial medical interview questions and returns them to the terminal.

[0843] Step 4:

[0844] Terminal: The terminal displays the initial question to the user.

[0845] Step 5:

[0846] User: The user answers the question and taps the Next button.

[0847] Step 6:

[0848] Device: The device makes a request that sends the answer to the server.

[0849] Step 7:

[0850] Server: The server stores the received answer in a database and generates the next question.

[0851] Step 8:

[0852] Device: At the same time, the device activates an emotion engine that analyzes the user's voice and facial expressions.

[0853] Step 9:

[0854] Emotion engine: The emotion engine collects the user's emotional data and returns the analysis results to the device.

[0855] Step 10:

[0856] Server: The server sends the following question to the device:

[0857] Step 11:

[0858] Terminal: The terminal displays the next question to the user. This process is repeated until all questions have been answered.

[0859] Analysis of interview results and emotional data and reflection in medical records

[0860] Step 1:

[0861] Server: Once all the questionnaire responses and emotional data have been collected, the server inputs the data into the machine learning model and begins analysis.

[0862] Step 2:

[0863] Server: The server obtains the analysis results and generates medical record information including diagnostic predictions.

[0864] Step 3:

[0865] Server: The generated medical record information also includes information about the user's emotional state.

[0866] Step 4:

[0867] Server: The server creates a request to send the medical record information to the medical institution.

[0868] Step 5:

[0869] Server: The server sends the medical record information to the target medical institution.

[0870] Step 6:

[0871] Medical institutions: Medical institutions receive the medical record information and use it for medical examinations.

[0872] Appointments and doctor coordination

[0873] Step 1:

[0874] User: The user opens the "Appointment" screen within the app and selects the desired appointment date and time.

[0875] Step 2:

[0876] Terminal: The terminal makes a request to send the reservation information to the server.

[0877] Step 3:

[0878] Server: The server receives the reservation request and checks whether the reservation is available by connecting with the medical institution's system.

[0879] Step 4:

[0880] Server: The server returns the reservation approval result to the terminal.

[0881] Step 5:

[0882] Terminal: The terminal displays a confirmation message to the user.

[0883] Step 6:

[0884] Medical institution: The medical institution receives the appointment information and prepares for the examination based on the user's medical interview data and emotion data.

[0885] Step 7:

[0886] Medical institution: After the examination, the medical institution sends the examination result information to the server.

[0887] Step 8:

[0888] Server: The server receives the medical result information and stores it in association with the user's account.

[0889] Step 9:

[0890] Device: The device will notify the user of the medical examination results and allow them to view them within the app.

[0891] The above are the specific operations in each processing step. In this way, the user can efficiently carry out the process from interviewing the patient to examining the patient, collecting emotional data, and checking the examination results.

[0892] Example 2

[0893] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0894] Conventional medical systems provide diagnostic predictions based solely on the user's medical interview information, and are unable to consider the user's emotional state. This makes it difficult to improve diagnostic accuracy and to properly reflect the impact of the user's emotions on the diagnosis. It is also difficult to provide a rapid examination and efficient appointment system that takes the user's emotional state into account.

[0895] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0896] In this invention, the server includes means for receiving medical interview information input by a user, means for analyzing the received medical interview information, means for providing medical institutions with medical record information generated based on the analysis results, means for receiving appointment information and making an appointment with the medical institution, means for notifying the user of medical examination result information received from the medical institution, means for collecting and analyzing emotional data of the user, and means for generating medical record information by integrating the medical interview information and emotional data. This enables highly accurate diagnosis prediction that integrates the user's medical interview information and emotional state, and realizes efficient and prompt examinations and appointments.

[0897] "User" refers to an individual or patient who uses the healthcare system.

[0898] A "terminal" is a hardware device used by a user, and refers to an electronic device such as a smartphone or tablet.

[0899] "Server" refers to a computer system that receives, processes, and stores data sent from users and terminals.

[0900] "Medical interview information" refers to data regarding health conditions and symptoms entered by the user.

[0901] "Emotional data" refers to data relating to the psychological state of a user that is analyzed from the user's voice, facial expressions, etc.

[0902] "Analysis" refers to the process of analyzing information and deriving certain results or predictions based on received data.

[0903] A "machine learning model" refers to an algorithm that learns patterns from large amounts of data and uses that learning to make predictions and judgments about new data.

[0904] "Medical record information" refers to information related to a medical diagnosis generated based on medical interview information and emotion data.

[0905] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0906] "Appointment for medical examination" refers to the act of a user reserving a date and time for a medical examination at a medical institution.

[0907] "Notification" refers to the act of sending information from a server to a user.

[0908] MODE FOR CARRYING OUT THE INVENTION

[0909] This invention relates to a system that collects and analyzes user interview information and emotional data to collaborate with medical institutions. The program processing of this system is described in detail below. The hardware and software used include smartphones and tablets as terminals, servers, databases, and machine learning models.

[0910] User Registration and Login

[0911] When a user uses the app for the first time, they tap the "Sign Up" button and the device displays the new registration screen. When the user enters their name, email address, and password and taps the "Submit" button, the device sends the entered data to the server. The server receives this data and stores it in a database (for example, MySQL). Next, when the user enters authentication information on the "Login" screen and taps the Login button, the device sends that information to the server. The server compares the authentication information with the database, and if it is correct, it generates an access token and returns it to the device, allowing the user to log in successfully.

[0912] Filling out a medical questionnaire and collecting emotional data

[0913] When the user taps the "Start Questionnaire" button, the device sends a request to start the questionnaire to the server. The server generates the first questions using Python-based logic and returns them to the device. The device displays the questions to the user, who answers each question. The device then sends the answers to the server, which analyzes the answers, generates the next question, and returns it to the device. During this process, the device launches an emotion analysis engine such as EmotionEngine or Affectiva to analyze the user's voice and facial expressions to collect emotional data.

[0914] Analysis of interview results and emotional data and reflection in medical records

[0915] Once all medical interview and emotion data collection is complete, the server receives this data. The server analyzes the data using machine learning models such as TensorFlow and PyTorch to obtain a diagnostic prediction. The server generates medical record information based on the diagnostic prediction. The generated medical record information includes the user's medical interview information and emotion data. The server provides the medical record information to medical institutions using the FHIR protocol, and the medical institutions prepare it so that it can be used for consultations.

[0916] Appointments and doctor coordination

[0917] The user selects the desired appointment date within the app and enters the appointment information. When the user taps the "Confirm appointment" button, the device sends that information to the server. The server communicates this information to the medical institution and checks whether the appointment is possible. The confirmation result is returned to the device, and if the appointment is approved, a success message is displayed to the user. The medical institution prepares for the appointment based on the interview data and emotion data received through the server. After the examination, the medical institution sends the examination results and prescription information to the server, which notifies the user. The user can check the examination results and prescription information through the app.

[0918] Specific examples

[0919] As an example, consider a case where a user complains of a migraine while feeling emotionally stressed. The user enters "I often get headaches" in the medical interview information and provides detailed information about the frequency and other symptoms. The device uses Emotion Engine to analyze the emotional state from the user's voice and facial expression and detects that the user is feeling stressed. The server receives the medical interview information and emotional data and uses a machine learning model to predict that there is a high correlation between "migraine" and "stress." The server generates medical record information based on this diagnosis prediction and emotional state and sends it to the medical institution. The medical institution receives this information in advance and makes preparations to respond quickly and appropriately when the user visits. After the examination, the medical institution sends the examination results and prescription information to the server, which the user can check through the app.

[0920] Examples of prompt statements

[0921] "The user complains of a headache, and you want to analyze their emotional state. Based on the interview information and emotional data, make a diagnosis prediction and generate a medical record."

[0922] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0923] User Registration and Login

[0924] Step 1:

[0925] When a user uses the app for the first time, they tap the "Sign Up" button. The inputs are the user's name, email address, and password. The output is a successful registration message.

[0926] Step 2:

[0927] The device displays the new registration screen, and the user enters their name, email address, and password and taps the "Submit" button. The input is the user's registration information. The output is the data to be sent to the server.

[0928] Step 3:

[0929] The terminal sends the entered data to the server. The input includes the user's registration information. The output is a message indicating successful data transmission to the server.

[0930] Step 4:

[0931] The server receives the data and stores it in a database. The input is the user's registration information. The output is the registered user information stored in the database.

[0932] Step 5:

[0933] The server returns a message to the terminal indicating that new registration has been completed. As input, there is information indicating successful registration. As output, a notification message is generated for the terminal.

[0934] Step 6:

[0935] The user taps the "Login" button. The input is the user's authentication information (email address, password). The output is a login request message.

[0936] Step 7:

[0937] The device displays the authentication information input screen, the user enters the information, and taps the "Login" button. The input is the user's authentication information. The output is authentication data to be sent to the server.

[0938] Step 8:

[0939] The server authenticates the received authentication information by checking it against a database. The input is the user's authentication information. The output is a success / failure message.

[0940] Step 9:

[0941] If authentication is successful, the server generates an access token and returns it to the terminal. The input is the user's successful authentication information. The output is the generated access token.

[0942] Step 10:

[0943] The user successfully logs in. The input is an access token. The output is a login successful message.

[0944] Filling out a medical questionnaire and collecting emotional data

[0945] Step 1:

[0946] The user taps the "Start medical interview" button. As input, there is a request to start medical interview. As output, a medical interview start request message is generated.

[0947] Step 2:

[0948] The terminal sends a consultation start request to the server. The input is the consultation start request data. The output is a message indicating that the transmission to the server was successful.

[0949] Step 3:

[0950] The server generates the initial question and returns it to the terminal. The input is a request to start the medical interview. The output is the medical interview question data.

[0951] Step 4:

[0952] The terminal displays the question to the user. The input is the question data. The output is a question screen that is displayed to the user.

[0953] Step 5:

[0954] The user answers the questions. The input is the user's answers to the medical interview questions. The output is the answer data.

[0955] Step 6:

[0956] The terminal sends the user's answer to the server. The input is the user's answer data. The output is a successful transmission message to the server.

[0957] Step 7:

[0958] The server analyzes the received answer and generates the next question. The input is the user's answer data. The output is the next question data.

[0959] Step 8:

[0960] The server returns the next question to the terminal, which then displays it to the user. The input is the next question data. The output is a question screen that is displayed to the user.

[0961] Step 9:

[0962] The device launches an emotion analysis engine such as EmotionEngine or Affectiva, and analyzes the user's voice and facial expressions to collect emotional data. The emotion analysis engine takes input, and emotional data is generated as output.

[0963] Step 10:

[0964] Send emotion data to the server. The emotion data is the input. The output is a message indicating that the transmission to the server was successful.

[0965] Analysis of interview results and emotional data and reflection in medical records

[0966] Step 1:

[0967] After all the questionnaire and emotion data collection is completed, the server receives these data. The inputs are the user's questionnaire data and emotion data. The output is stored in the server.

[0968] Step 2:

[0969] The server inputs the received data into a machine learning model such as TensorFlow or PyTorch and analyzes the data. The inputs include medical interview data and emotion data. The output is a diagnostic prediction.

[0970] Step 3:

[0971] Medical record information is generated based on the diagnostic prediction. The input is diagnostic prediction data. The output is medical record information.

[0972] Step 4:

[0973] The generated medical record information is provided to the medical institution using the FHIR protocol. The input is the medical record information. The output is a message indicating successful transmission to the medical institution.

[0974] Appointments and doctor coordination

[0975] Step 1:

[0976] The user selects the desired appointment date within the app and enters the reservation information. The input includes the desired appointment date and reservation information. The output is the reservation information.

[0977] Step 2:

[0978] When the user taps the "Confirm Reservation" button, the device sends the information to the server. The input is the reservation information. The output is a message indicating that the transmission to the server was successful.

[0979] Step 3:

[0980] The server sends the reservation information to the medical institution and checks whether the reservation is possible. The input is the reservation information. The output is reservation confirmation information.

[0981] Step 4:

[0982] The server returns the confirmation result to the terminal. The input is the reservation confirmation information. The output is a transmission success message to the terminal.

[0983] Step 5:

[0984] If the reservation is approved, the terminal displays a success message to the user. The input is the reservation approval information. The output is a success message.

[0985] Step 6:

[0986] The medical institution prepares for the examination based on the medical interview data and emotion data received through the server. The inputs are the medical interview data and emotion data. The output is the examination preparation.

[0987] Step 7:

[0988] After the consultation, the medical institution sends the consultation results and prescription information to the server. The inputs are the consultation results and prescription information. The output is a message indicating that the transmission to the server was successful.

[0989] Step 8:

[0990] The server notifies the user. The inputs are the medical examination results and prescription information. The output is a notification message for the user.

[0991] Step 9:

[0992] The user checks the medical examination results and prescription information through the app. The input is a notification message. The output is the medical examination results and prescription information.

[0993] (Application example 2)

[0994] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0995] Current medical systems and security service systems do not fully utilize user interview information and emotional data, making it difficult to provide appropriate diagnoses and propose security measures. Furthermore, few systems take into consideration the user's stress or anxiety, which can lead to a decline in user satisfaction. Furthermore, collaboration between medical institutions and security experts is not smooth, making it difficult to provide efficient responses.

[0996] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving medical interview information and emotion data entered by the user; means for analyzing the received medical interview information and emotion data; means for providing medical record information generated based on the analysis results to the medical institution; means for receiving appointment information and making an appointment with the medical institution; means for notifying the user of the examination result information received from the medical institution; means for inputting a security diagnosis form and collecting emotion data; means for analyzing the collected data using a machine learning model and generating a security diagnosis report; means for proposing security measures to the user based on the analysis results; and means for realizing collaboration with security experts. This enables efficient diagnosis and provision of security measures that take the user's emotional state into consideration.

[0997] "Medical interview information" refers to data about symptoms and health conditions that are entered by the user themselves.

[0998] "Emotion data" refers to information about the user's emotional state that is analyzed from the user's tone of voice, facial expression, and so on.

[0999] "Analysis" refers to making diagnostic predictions and risk assessments based on collected interview information and emotional data.

[1000] "Medical record information" refers to data generated based on analysis results and provided to medical institutions, including diagnostic predictions and emotional states.

[1001] A "medical institution" is a facility that provides medical examinations and treatment, such as a hospital or clinic.

[1002] "Reservation information" refers to data on the consultation date and time desired by the user.

[1003] "Consultation appointment" refers to the procedure for reserving a date and time for consultation with a medical institution.

[1004] "Examination result information" refers to information generated by a medical institution and including examination results, prescription details, and the like.

[1005] The "security assessment form" is a list of questions to assess the user's security status.

[1006] A "security diagnostic report" is a document that evaluates a user's security status based on collected data and suggests risk levels and countermeasures.

[1007] A "security expert" is someone with specialized knowledge in the field of security.

[1008] A "machine learning model" is an algorithm that learns patterns and features based on data and makes predictions and diagnoses.

[1009] The system of this invention is a mechanism for collecting medical interview information and emotion data entered by the user, analyzing this information, generating medical record information and security diagnostic reports, and realizing collaboration with medical institutions and security experts. Specific program processing and the hardware and software used to realize this are described below.

[1010] User Registration and Login

[1011] When a user uses the system for the first time, they tap the "New Registration" button on their smartphone, tablet, or other device, enter their registration information, and send it to the server. This information is stored in a database such as Amazon RDS. When the user uses the system again, they enter their authentication information on the login screen and the server authenticates them before they can access the system.

[1012] Filling out a medical questionnaire and collecting emotional data

[1013] When a user taps the "Start Questionnaire" button while using the system, the server receives the request to start the questionnaire, generates the first question, and returns it to the device. The user answers the question, and the answer data is sent back to the server. During this process, the device uses libraries such as OpenCV and Dlib to analyze the user's voice and facial expressions and collect emotional data. This data is used for subsequent analysis.

[1014] Analysis of interview results and emotional data and reflection in medical records

[1015] Once all interviews and emotional data collection is complete, the server inputs this data into a machine learning model (e.g., scikit-learn, TensorFlow, PyTorch) for analysis. The generated medical record information also includes information about the user's emotional state. The server then sends this medical record information to the medical institution and prepares it for use in the consultation.

[1016] Appointments and doctor coordination

[1017] When a user makes a reservation for a desired consultation date, the information is sent to the medical institution via the server, and the reservation is confirmed. If the reservation is approved, a success message is displayed on the terminal. The medical institution uses this information to efficiently prepare for the consultation.

[1018] Fill out a security assessment form and collect emotional data

[1019] When a user taps the "Start Security Assessment" button using the system, the server receives the assessment start request, generates an initial set of questions, and returns them to the device. The user answers the questions, and the answer data is sent back to the server. During this process, the device uses libraries such as OpenCV and Dlib to analyze the user's voice and facial expressions and collect emotional data. This data is used for subsequent analysis.

[1020] Analysis of security diagnostic results and sentiment data and report generation

[1021] Once all interviews and emotional data collection is complete, the server inputs this data into a machine learning model (e.g., scikit-learn, TensorFlow, PyTorch) for analysis. The generated security assessment report includes information about the user's emotional state. The server provides this report to the user and allows them to connect with security experts if necessary.

[1022] Examples of specific examples and prompts

[1023] For example, if a user answers the questionnaire information by saying, "I change my password every six months. I don't use two-factor authentication," and the emotional data indicates that they are in a state of moderate stress, the system will assess them as "medium risk" and suggest the following countermeasures: specifically, "enable two-factor authentication," "use a password manager," and "regular data backup."

[1024] Prompt for the generative AI model:

[1025] Please conduct a security assessment based on the following user data and provide the risk level and specific measures.

[1026] User Data:

[1027] Answer: "I change my password every six months" and "I don't use two-factor authentication"

[1028] Emotional data: Slightly stressed

[1029] This program and mechanism makes it possible to provide efficient security measures that take into account the user's emotional state.

[1030] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1031] Step 1:

[1032] When a user uses the app for the first time, they tap the "Sign Up" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the input information to the server, which stores it in a database. This creates a user account for authentication.

[1033] Step 2:

[1034] The user enters authentication information (email address, password) on the login screen and taps the login button. The device sends the authentication information to the server. The server compares it with the information in the database, and if authentication is successful, generates an access token and returns it to the device. The device saves this access token and uses it in subsequent communications.

[1035] Step 3:

[1036] When the user taps the "Start medical interview" button, the device sends a medical interview start request to the server. The server generates a medical interview form and returns the initial questions to the device. The device displays the questions to the user and provides an interface for inputting answers.

[1037] Step 4:

[1038] The user enters their answers into the questionnaire form and taps the send button. The device then sends the answer data to the server. At that time, the device uses a camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional data. The analysis results are also sent to the server.

[1039] Step 5:

[1040] The server analyzes the medical interview information and emotional data it receives. Specifically, it uses machine learning models (such as scikit-learn, TensorFlow, and PyTorch) to predict diagnoses and generate analysis results, including the user's emotional state. It then generates medical record information based on the analysis results and prepares the data for transmission to medical institutions.

[1041] Step 6:

[1042] The server sends the generated medical record information to the designated medical institution, which receives the information and prepares for diagnosis and treatment.

[1043] Step 7:

[1044] When a user makes a reservation for a desired consultation date within the app, the device sends the reservation information to the server. The server then contacts the medical institution to confirm whether the reservation can be made and returns the result to the device. If the reservation is approved, the device displays a success message to the user.

[1045] Step 8:

[1046] After the consultation, the medical institution sends the consultation results and prescription information to the server, which then notifies the user, who can then check the consultation results through the app.

[1047] Step 9:

[1048] When the user taps the "Start Security Assessment" button, the device sends a request to the server to start the assessment. The server generates an assessment form and returns the initial questions to the device. The device then displays the questions to the user and provides an interface for entering answers.

[1049] Step 10:

[1050] The user enters their answers into the security diagnostic form and taps the submit button. The device then sends the answer data to the server. At that time, the device uses the camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional data. The analysis results are also sent to the server.

[1051] Step 11:

[1052] The server analyzes the security diagnostic information and emotion data received, performs security diagnostics using machine learning models and evaluates the risk level, generates a security diagnostic report based on the analysis results, and prepares the data to provide to the user.

[1053] Step 12:

[1054] Provides the user with the server-generated security assessment report and suggests countermeasures. If necessary, creates a request to connect the user with a security expert and submits it with the appropriate information.

[1055] The above processing steps make it possible to provide efficient diagnostic and security measures that take into account the emotional state of the user.

[1056] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1057] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1058] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1059] [Third embodiment]

[1060] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1061] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1062] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1063] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1064] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1065] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1066] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1067] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1068] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1069] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1070] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1071] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1072] The present invention is a system that uses a medical app to collect medical interview information from users, analyzes it to generate medical record information, and then connects with medical institutions. The program processing of this system is explained in natural language below. Specific examples are also provided for further explanation.

[1073] User Registration and Login

[1074] When a user uses the app for the first time, they tap the "New Registration" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the entered information to the server, which saves it in a database. Next, the user enters authentication information on the "Login" screen and taps the login button, which the device sends to the server. The server performs authentication and returns an access token to the device, completing the login.

[1075] Fill out the medical questionnaire form

[1076] When the user taps the "Start medical interview" button, the device sends a medical interview start request to the server. The server generates the first question and returns it to the device, which then displays it to the user. After that, the user answers each question, and the device sends them to the server. The server analyzes the answers, determines the next question, and returns it to the device. This process is repeated until all questions have been asked.

[1077] Analysis of interview results and reflection in medical records

[1078] Once all medical interviews are completed, the server analyzes the user's interview data using a machine learning model and makes a diagnosis prediction. The server then creates a request to send the medical record information, including the generated diagnosis prediction, to the target medical institution. The medical institution's system receives this medical record information and prepares it for use in consultations.

[1079] Appointments and doctor coordination

[1080] When a user makes an appointment for a desired consultation date within the app, the device sends the appointment information to the server. The server then connects with the medical institution to confirm whether the appointment is possible and returns the result to the device. If the appointment is approved, the device displays a success message to the user. The medical institution prepares to efficiently conduct the consultation based on the medical interview data received via the server. After the consultation, the medical institution sends the examination results and prescription information to the server, which then notifies the user. The user can then check the examination results through the app.

[1081] Specific examples

[1082] As an example, consider the case where a user complaining of migraines uses this system. The user enters "I often get headaches" in the medical interview information and provides detailed information about the frequency and other symptoms. The server receives this information, and the machine learning model predicts that the diagnosis is likely "migraine." The server generates medical record information including this diagnosis prediction and sends it to the medical institution. By receiving this information in advance, the medical institution can quickly begin consultation when the user visits the hospital without having to reconfirm the medical interview details. After the consultation, the medical institution sends the examination results and prescription details to the server, allowing the user to check them through the app.

[1083] This will improve the efficiency of medical interviews and the accuracy of diagnoses, making the system beneficial for both users and medical institutions.

[1084] The processing flow will be explained below.

[1085] User Registration and Login

[1086] Step 1:

[1087] User: The user launches the app and taps the "Sign Up" button.

[1088] Step 2:

[1089] Device: The device will display the new registration screen.

[1090] Step 3:

[1091] User: The user enters their name, email address, and password, then taps the submit button.

[1092] Step 4:

[1093] Device: The device creates a request to send the input information to the server.

[1094] Step 5:

[1095] Server: The server receives the request and performs validation of the input information.

[1096] Step 6:

[1097] Server: If the server is happy with the information, it stores it in the database.

[1098] Step 7:

[1099] Server: The server returns a registration success response to the device.

[1100] Step 8:

[1101] Terminal: The terminal displays a successful registration message to the user.

[1102] Step 9:

[1103] User: The user enters their credentials on the "Login" screen and taps the login button.

[1104] Step 10:

[1105] Device: The device sends the entered authentication information to the server.

[1106] Step 11:

[1107] Server: The server verifies the authentication information, generates an access token, and returns a response to the device.

[1108] Step 12:

[1109] Device: The device stores the access token and uses it in subsequent requests.

[1110] Fill out the medical questionnaire form

[1111] Step 1:

[1112] User: The user taps the "Start Interview" button.

[1113] Step 2:

[1114] Terminal: The terminal sends a request to start an interview to the server.

[1115] Step 3:

[1116] Server: The server generates the initial medical interview questions and returns them to the terminal.

[1117] Step 4:

[1118] Terminal: The terminal displays the initial question to the user.

[1119] Step 5:

[1120] User: The user answers the question and taps the Next button.

[1121] Step 6:

[1122] Device: The device sends the answer to the server.

[1123] Step 7:

[1124] Server: The server stores the received answer in a database and generates the next question.

[1125] Step 8:

[1126] Server: The server sends the following question to the device:

[1127] Step 9:

[1128] Terminal: The terminal displays the next question to the user. This process is repeated until all questions have been answered.

[1129] Analysis of interview results and reflection in medical records

[1130] Step 1:

[1131] Server: After all questionnaire responses are completed, the server inputs the questionnaire data into the machine learning model and begins analysis.

[1132] Step 2:

[1133] Server: The server obtains the analysis results and generates medical record information including diagnostic predictions.

[1134] Step 3:

[1135] Server: The server creates a request to send the medical record information to the medical institution.

[1136] Step 4:

[1137] Server: The server sends the medical record information to the target medical institution.

[1138] Step 5:

[1139] Medical institutions: Medical institutions receive the medical record information and use it for medical examinations.

[1140] Appointments and doctor coordination

[1141] Step 1:

[1142] User: The user opens the "Appointment" screen within the app and selects the desired appointment date and time.

[1143] Step 2:

[1144] Terminal: The terminal makes a request to send the reservation information to the server.

[1145] Step 3:

[1146] Server: The server receives the reservation request and checks whether the reservation is available by connecting with the medical institution's system.

[1147] Step 4:

[1148] Server: The server returns the reservation approval result to the terminal.

[1149] Step 5:

[1150] Terminal: The terminal displays a confirmation message to the user.

[1151] Step 6:

[1152] Medical institution: The medical institution receives the reservation information and prepares for the examination based on the user's medical interview data.

[1153] Step 7:

[1154] Medical institution: After the examination, the medical institution sends the examination result information to the server.

[1155] Step 8:

[1156] Server: The server receives the medical result information and stores it in association with the user's account.

[1157] Step 9:

[1158] Device: The device will notify the user of the medical examination results and allow them to view them within the app.

[1159] The above is a detailed description of the operations performed in each processing step. In this way, the user can efficiently carry out the process from interviewing the patient to examining the patient and confirming the examination results.

[1160] Example 1

[1161] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1162] Modern medical care involves many processes, such as collecting and analyzing medical history information, scheduling appointments, and notifying patients of diagnosis results. Each process requires time and effort, creating a challenge. Furthermore, doctors' time spent comprehending detailed medical history information during consultations reduces the efficiency of the entire consultation. Conventional systems have struggled to manage these processes in an integrated manner and process them efficiently and accurately.

[1163] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1164] In this invention, the server includes a means for receiving medical interview information input by a user, a means for analyzing the received medical interview information, a means for providing electronic medical record information generated based on the analysis results to a medical institution, a means for receiving appointment information and making an appointment with the medical institution, and a means for notifying the user of the examination result information received from the medical institution. This enables efficient collection and analysis of medical interview information, prompt notification of diagnosis results, and efficient examinations through cooperation with medical institutions.

[1165] "User" refers to an individual or corporation that uses the system.

[1166] "Medical interview information" refers to information about health conditions and symptoms provided by the user.

[1167] "Electronic medical record information" refers to health records generated from medical interview information and analysis results.

[1168] "Medical institution" refers to an organization such as a hospital or clinic that provides medical care or treatment.

[1169] "Appointment for medical examination" refers to the act of reserving a desired date and time for medical examination at a medical institution by a user.

[1170] "Examination result information" refers to information regarding diagnoses and prescriptions provided by medical institutions after examinations.

[1171] A "server" is a computer device that is the core of a system and that processes and stores data and communicates with other devices.

[1172] A "machine learning model" refers to an algorithm that learns from large amounts of data and makes predictions and classifications.

[1173] "Notification" refers to a means by which the system notifies the user of information.

[1174] "Receive" refers to the act of obtaining data from another device or system.

[1175] "Analysis" refers to the act of examining information in detail based on received data and understanding its meaning and structure.

[1176] The present invention is a system that uses a medical application to collect medical interview information from users, analyzes the information, generates electronic medical records, and collaborates with medical institutions. This system consists of a server, terminals, and users.

[1177] Program processing explanation

[1178] User Registration and Login

[1179] When a user launches the app and taps the "New Registration" button, the device displays the new registration screen. When the user enters their name, email address, and password and taps the "Submit" button, the device sends the input information to the server. The server stores the received information in a database.

[1180] When a user enters their email address and password on the "Login" screen and taps the login button, the device sends the entered information to the server. The server performs authentication and returns an access token to the device, completing the login. The technology used is a smartphone or tablet as the device, and a cloud service (e.g., Amazon Web Services, Microsoft Azure) as the server.

[1181] Fill out the medical questionnaire form

[1182] When the user taps the "Start medical interview" button, the device sends a request to the server to start the medical interview. The server generates the first questions and returns them to the device. The device displays the questions, and the user answers each one. When the device sends the answers to the server, the server analyzes the answers and determines the next question. This process is repeated until all questions have been asked.

[1183] The technologies used include stream processing using Apache Kafka for real-time analysis on the server side, and natural language processing (NLP) libraries implemented in Python (e.g., SpaCy, NLTK).

[1184] Analysis of interview results and reflection in medical records

[1185] Once all interviews are completed, the device sends an end request to the server. The server inputs the interview data into a machine learning model for analysis. Specific technologies used include Scikit-Learn and TensorFlow. The server generates a diagnostic prediction and formats it as electronic medical record information. The server then creates and sends a request to send this electronic medical record information to the medical institution.

[1186] The medical institution's system receives electronic medical record information and prepares it so that it can be used for consultations. The technology used utilizes the standard FHIR (Fast Healthcare Interoperability Resources).

[1187] Appointments and doctor coordination

[1188] When a user makes a reservation for a desired consultation date within the app, the device sends the reservation information to the server. The server then connects with the medical institution to confirm whether the reservation is possible. If the reservation is approved, the device displays a success message to the user.

[1189] On the day of the consultation, the medical institution will prepare for the consultation efficiently based on the medical interview data received through the server. After the consultation, the medical institution will send the consultation results and prescription information to the server, which will notify the user. The user can then check the consultation results through the application.

[1190] As a specific example of how it works, for example, a user who complains of migraines uses the system and enters "I often get headaches" into the medical interview information, and provides detailed information about the frequency and other symptoms. Based on this information, the server uses a machine learning model to predict the possibility of a "migraine," generates electronic medical record information, and sends it to the medical institution. By receiving this information in advance, the medical institution can prepare for a prompt examination.

[1191] Prompt Sentence Examples

[1192] Here are some example prompts to input to a generative AI model:

[1193] "What API endpoints do you use to process user registration?"

[1194] "Please explain how to analyze medical interview data using a machine learning model."

[1195] "Please tell me about the process for sending medical record information according to the FHIR standard."

[1196] In this way, this system uses technically linked components to improve the efficiency of medical interviews and the accuracy of diagnoses.

[1197] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1198] System program processing steps

[1199] User Registration and Login

[1200] Step 1:

[1201] The user launches the app and taps the "New Registration" button. The device displays the new registration screen. The user enters their name, email address, and password, and taps the send button. The input is their name, email address, and password, and this information is sent from the device to the server as output.

[1202] Step 2:

[1203] The server stores the received information in a database. In this storage process, an INSERT query is executed using a database management system (e.g., MySQL) to record the user information. The input is the user information received from the terminal, and the output is the result stored in the database.

[1204] Step 3:

[1205] The user enters their email address and password on the "Login" screen and taps the login button. The device sends the authentication information to the server. The input is the email address and password, and the output is this information sent from the device to the server.

[1206] Step 4:

[1207] The server references the database and checks the authentication information. If authentication is successful, it generates a JWT (JSON Web Token) and returns it to the terminal as an access token. If authentication fails, it generates an error message and returns it to the terminal. The input is authentication information, and the output is an access token or an error message.

[1208] Fill out the medical questionnaire form

[1209] Step 5:

[1210] The user taps the "Start medical interview" button. The terminal sends a medical interview start request to the server. The input is the medical interview start request, and the output is that this request is sent from the terminal to the server.

[1211] Step 6:

[1212] The server generates the initial question and returns it to the terminal. At this time, the server executes an algorithm that randomly selects a question from a list of questions prepared in advance. The input is the request to start the medical interview, and the output is the initial question.

[1213] Step 7:

[1214] The terminal displays a question, the user answers the question, the input is the initial question, and the output is the user's answer.

[1215] Step 8:

[1216] The device sends the user's answer to the server. The server analyzes the received answer using an NLP algorithm (e.g., SpaCy, NLTK). Based on the analysis results, the next question is dynamically generated. The input is the user's answer, and the output is the next question.

[1217] Step 9:

[1218] The server generates the next question and sends it back to the terminal, which displays it to the user. This process is repeated until all questions have been asked. The input is the next question, and the output is that question will be displayed on the terminal.

[1219] Analysis of interview results and reflection in medical records

[1220] Step 10:

[1221] When all the inquiries are completed, the terminal sends an end request to the server. The input is the inquiry end request, and the output is that the request is sent from the terminal to the server.

[1222] Step 11:

[1223] The server inputs the received medical interview data into a machine learning model (e.g., Scikit-Learn, TensorFlow) for analysis. The machine learning model generates a diagnostic prediction. The input is the medical interview data, and the output is a diagnostic prediction.

[1224] Step 12:

[1225] The server formalizes the generated diagnosis prediction as electronic medical record information. Then, a request including this electronic medical record information is sent to the medical institution. The input is the diagnosis prediction, and the output is the electronic medical record information.

[1226] Step 13:

[1227] The medical institution's system receives the electronic medical record information and prepares it so that it can be used for consultations. The input is the electronic medical record information, and the output is the completion of consultation preparation.

[1228] Appointments and doctor coordination

[1229] Step 14:

[1230] The user makes a reservation for the desired consultation date within the app, and the device sends the reservation information to the server. The input is the reservation information, and the output is that information is sent from the device to the server.

[1231] Step 15:

[1232] The server receives the reservation information and sends a request to the medical institution to confirm whether the reservation is available. The input is the reservation information, and the output is confirmation of whether the reservation is available.

[1233] Step 16:

[1234] The server receives a response from the medical institution, and if the reservation is approved, it returns a success message to the terminal. The input is the response from the medical institution, and the output is the success message.

[1235] Step 17:

[1236] On the day of the consultation, the server retransmits the received medical interview data to the medical institution, allowing the medical institution to conduct the consultation efficiently. The input is the medical interview data and reservation information, and the output is the preparation for the consultation at the medical institution.

[1237] Step 18:

[1238] After the consultation, the medical institution sends the consultation results and prescription information to the server, and the server notifies the user of the information. The input is the consultation results and prescription information, and the output is a notification to the user. This notification allows the user to check the consultation results and prescription information through the app.

[1239] (Application example 1)

[1240] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1241] In today's medical environment, the process from medical interview to examination is complicated, resulting in time-consuming and laborious tasks for both users and medical facilities. Furthermore, there are limited ways for users to conduct medical interviews and receive diagnostic predictions without visiting a medical institution, making it difficult to provide prompt and efficient medical services. Furthermore, methods for conducting medical interviews and diagnoses in virtual spaces are not widespread, preventing an improved user experience and hindering the improvement of the quality of medical services.

[1242] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1243] In this invention, the server includes means for receiving medical interview information input by a user, means for analyzing the received medical interview information and making a diagnosis prediction, means for providing medical record information generated based on the analysis results to a medical facility, means for receiving consultation appointment information and making a consultation appointment with the medical facility, means for notifying the user of the examination result information received from the medical facility, and means for the user to input medical interview information in a virtual store and collaborate with a virtual doctor. This enables the user to efficiently conduct a medical interview in a virtual space, quickly receive a diagnosis prediction, and, if necessary, make a consultation appointment.

[1244] "User" refers to an individual who uses medical services.

[1245] "Medical interview information" refers to information regarding symptoms and health conditions entered by the user.

[1246] "Means for receiving" refers to the function by which the server receives medical interview information and reservation information from the user.

[1247] "Means of analysis" refers to the function of using machine learning models and algorithms to make diagnostic predictions based on the received medical interview information.

[1248] "Diagnosis prediction" refers to the results of predicting the user's symptoms and condition based on medical interview information.

[1249] "Medical record information" refers to digital information that summarizes the user's health status and diagnostic predictions.

[1250] "Means for providing" refers to the function for transmitting the generated medical record information to a medical facility.

[1251] "Reservation information" refers to information regarding the date and time of the consultation desired by the user.

[1252] "Means for making appointments" refers to the function of making appointments with medical facilities based on the user's desired date and time.

[1253] "Means for notifying" refers to the function for transmitting medical examination result information received from medical facilities to users.

[1254] A "virtual store" refers to a virtual space accessible via the Internet where users can receive medical services.

[1255] "Virtual doctor" refers to a digital doctor who conducts medical interviews with users within a virtual store.

[1256] A "machine learning model" refers to an artificial intelligence (AI) algorithm used for data analysis and prediction.

[1257] An "interactive experience" refers to an experience that is provided through two-way interaction with the user.

[1258] This invention provides a system that enables users to efficiently conduct medical interviews in a virtual store, quickly receive a diagnosis, and, if necessary, schedule an appointment. The system has a series of functions that receive and analyze the medical interview information entered by the user, perform a diagnosis prediction, and provide medical record information to medical facilities.

[1259] The system consists of an application installed on the user's smartphone and a backend service running on a server. When the user enters medical history information through the application, the information is sent to the server. The server analyzes the received information and makes a diagnosis prediction using a machine learning model. The specific software used for this analysis is the latest machine learning libraries such as TensorFlow and PyTorch.

[1260] The server then generates a medical record based on the diagnosis prediction and provides it to the medical facility. This medical record details the user's symptoms and diagnosis prediction. When the user enters their desired consultation date and time, the reservation information is also sent to the server, and the appointment is completed through collaboration with the medical facility.

[1261] After the user receives a medical examination, the medical facility sends the examination result information to the server, which then notifies the user, who can then check the information through the application.

[1262] As a concrete example, the following scenario shows a user named Tom using the app to complete a headache interview and ultimately receive a diagnosis of migraine.

[1263] User: Tom, please enter your name and email.

[1264] System: Name: Tom

[1265] System: Email: tom@example.com

[1266] System: Please answer the following questions about your symptoms.

[1267] System: Question: How often do you experience headaches?

[1268] User: Almost daily.

[1269] System: Question: How severe are your headaches on a scale of 1 to 10?

[1270] User: 7

[1271] System: Analyzing your answers...

[1272] System: Based on your answers, there is a high likelihood of migraine. Would you like to book an appointment with a doctor?

[1273] User: Yes.

[1274] System: Booking an appointment on November 1, 2023...

[1275] System: Appointment confirmed. You will receive further instructions via email.

[1276] In this way, users can use a system that integrates medical interviews, diagnosis predictions, and appointment bookings in a virtual space, allowing them to receive efficient and prompt medical services.

[1277] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1278] Step 1:

[1279] A user launches the app and registers or logs in.

[1280] Specific operation: The device displays a new registration screen or login screen, and the user enters their name, email address, and password. The input information is sent to the server, which stores it in a database or authenticates it, and if authentication is successful, returns an access token to the device.

[1281] Input: User's name, email address, and password

[1282] Output: Access token

[1283] Step 2:

[1284] The user taps the "Start medical interview" button.

[1285] Specific operation: The terminal sends a request to start an interview to the server, and the server generates the first question and sends it to the terminal. The terminal displays the question to the user.

[1286] Input: Access token, medical interview start request

[1287] Output: First question

[1288] Step 3:

[1289] The user answers each question.

[1290] Specific operation: The user enters an answer, and the device sends the answer to the server. The server analyzes the answer, generates the next question, and returns it to the device. This process is repeated until all questions have been answered.

[1291] Input: User's answer

[1292] Output: Next question

[1293] Step 4:

[1294] The interview information is analyzed to generate a diagnostic prediction.

[1295] Specific operation: The server receives the answers to all questions, analyzes the user's medical interview data using a machine learning model, makes a diagnosis prediction, and generates the results as medical record information.

[1296] Input: All response data

[1297] Output: Diagnosis prediction, medical record information

[1298] Step 5:

[1299] Provide medical record information to medical facilities.

[1300] Specific operation: The server creates a request to send the medical record information including the generated diagnosis prediction to the medical facility, and sends it to the medical facility. The medical facility receives the medical record information and prepares it for use in consultations.

[1301] Input: Medical record information

[1302] Output: Medical record information sent

[1303] Step 6:

[1304] A user makes an appointment.

[1305] Specific operation: The user inputs the desired appointment date and time, and the terminal sends the reservation information to the server. The server then connects with the medical facility to check whether the appointment is available. The result is returned to the terminal, which then displays a reservation confirmation message to the user.

[1306] Input: Consultation appointment information

[1307] Output: Reservation confirmation message

[1308] Step 7:

[1309] Notify the user of the medical examination result information.

[1310] Specific operation: After the examination, the medical facility sends the examination result information to the server, which then notifies the user's device, where the user can check the examination result through the app.

[1311] Input: Examination result information

[1312] Output: Examination result notification

[1313] Through these steps, users can conduct a virtual medical interview, receive a diagnosis, and schedule an appointment if necessary.

[1314] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1315] The present invention is a system that uses a medical app to collect medical interview information and emotional data from users, analyzes this data to generate medical record information, and then collaborates with medical institutions. The program processing of this system is explained in natural language below. Specific examples are also provided for further explanation.

[1316] User Registration and Login

[1317] When a user uses the app for the first time, they tap the "New Registration" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the entered information to the server, which saves it in a database. Next, the user enters authentication information on the "Login" screen and taps the login button, which the device sends to the server. The server performs authentication and returns an access token to the device, completing the login.

[1318] Filling out a medical questionnaire and collecting emotional data

[1319] When the user taps the "Start medical interview" button, the device sends a request to start the medical interview to the server. The server generates the first question and returns it to the device, which then displays it to the user. The user then answers each question, which the device then sends to the server. The server analyzes the answers, determines the next question, and returns it to the device. During this process, the device also activates an emotion engine that analyzes the user's voice and facial expressions, collecting the user's emotional data.

[1320] Analysis of interview results and emotional data and reflection in medical records

[1321] After all interviews and emotional data have been collected, the server inputs this data into a machine learning model and begins analysis. The server obtains the analysis results and generates medical record information including a diagnosis prediction. The generated medical record information also includes information about the user's emotional state. The server creates a request to send the medical record information to the target medical institution and sends it to the target medical institution. The medical institution's system receives this medical record information and it is ready to be used for consultations.

[1322] Appointments and doctor coordination

[1323] When a user makes an appointment for a desired consultation date within the app, the device sends the appointment information to the server. The server then connects with the medical institution to confirm whether the appointment is possible and returns the result to the device. If the appointment is approved, the device displays a success message to the user. The medical institution prepares for an efficient consultation based on the medical interview data and emotion data received via the server. After the consultation, the medical institution sends the examination results and prescription information to the server, which then notifies the user. The user can then check the examination results through the app.

[1324] Specific examples

[1325] As an example, consider a case where a user complains of a migraine while feeling emotionally stressed. The user enters "I often get headaches" in the medical interview information and provides detailed responses about the frequency and other symptoms. The device uses an emotion engine to analyze the user's emotional state from their voice and facial expressions, detecting, for example, that the user is feeling stressed. The server receives the medical interview information and emotion data, and a machine learning model predicts that there is a high correlation between the possibility of "migraine" and "stress." The server generates a medical record including this diagnosis prediction and emotional state and sends it to the medical institution. By receiving this information in advance, the medical institution can quickly begin a consultation that takes into account not only the medical interview content but also the user's emotional state when they visit. After the consultation, the medical institution sends the examination results and prescription details to the server, allowing the user to check them through the app.

[1326] This will improve the efficiency of medical interviews and the accuracy of diagnoses, making the system beneficial for both users and medical institutions. Utilizing emotion data will enable deeper understanding and appropriate responses, improving the quality of patient care.

[1327] The processing flow will be explained below.

[1328] User Registration and Login

[1329] Step 1:

[1330] User: The user launches the app and taps the "Sign Up" button.

[1331] Step 2:

[1332] Device: The device will display the new registration screen.

[1333] Step 3:

[1334] User: The user enters their name, email address, and password, then taps the submit button.

[1335] Step 4:

[1336] Device: The device creates a request to send the input information to the server.

[1337] Step 5:

[1338] Server: The server receives the request and performs validation of the input information.

[1339] Step 6:

[1340] Server: If the server is happy with the information, it stores it in the database.

[1341] Step 7:

[1342] Server: The server returns a registration success response to the device.

[1343] Step 8:

[1344] Terminal: The terminal displays a successful registration message to the user.

[1345] Step 9:

[1346] User: The user enters their credentials on the "Login" screen and taps the login button.

[1347] Step 10:

[1348] Device: The device sends the entered authentication information to the server.

[1349] Step 11:

[1350] Server: The server verifies the authentication information, generates an access token, and returns a response to the device.

[1351] Step 12:

[1352] Device: The device stores the access token and uses it in subsequent requests.

[1353] Filling out a medical questionnaire and collecting emotional data

[1354] Step 1:

[1355] User: The user taps the "Start Interview" button.

[1356] Step 2:

[1357] Terminal: The terminal sends a request to start an interview to the server.

[1358] Step 3:

[1359] Server: The server generates the initial medical interview questions and returns them to the terminal.

[1360] Step 4:

[1361] Terminal: The terminal displays the initial question to the user.

[1362] Step 5:

[1363] User: The user answers the question and taps the Next button.

[1364] Step 6:

[1365] Device: The device makes a request that sends the answer to the server.

[1366] Step 7:

[1367] Server: The server stores the received answer in a database and generates the next question.

[1368] Step 8:

[1369] Device: At the same time, the device activates an emotion engine that analyzes the user's voice and facial expressions.

[1370] Step 9:

[1371] Emotion engine: The emotion engine collects the user's emotional data and returns the analysis results to the device.

[1372] Step 10:

[1373] Server: The server sends the following question to the device:

[1374] Step 11:

[1375] Terminal: The terminal displays the next question to the user. This process is repeated until all questions have been answered.

[1376] Analysis of interview results and emotional data and reflection in medical records

[1377] Step 1:

[1378] Server: Once all the questionnaire responses and emotional data have been collected, the server inputs the data into the machine learning model and begins analysis.

[1379] Step 2:

[1380] Server: The server obtains the analysis results and generates medical record information including diagnostic predictions.

[1381] Step 3:

[1382] Server: The generated medical record information also includes information about the user's emotional state.

[1383] Step 4:

[1384] Server: The server creates a request to send the medical record information to the medical institution.

[1385] Step 5:

[1386] Server: The server sends the medical record information to the target medical institution.

[1387] Step 6:

[1388] Medical institutions: Medical institutions receive the medical record information and use it for medical examinations.

[1389] Appointments and doctor coordination

[1390] Step 1:

[1391] User: The user opens the "Appointment" screen within the app and selects the desired appointment date and time.

[1392] Step 2:

[1393] Terminal: The terminal makes a request to send the reservation information to the server.

[1394] Step 3:

[1395] Server: The server receives the reservation request and checks whether the reservation is available by connecting with the medical institution's system.

[1396] Step 4:

[1397] Server: The server returns the reservation approval result to the terminal.

[1398] Step 5:

[1399] Terminal: The terminal displays a confirmation message to the user.

[1400] Step 6:

[1401] Medical institution: The medical institution receives the appointment information and prepares for the examination based on the user's medical interview data and emotion data.

[1402] Step 7:

[1403] Medical institution: After the examination, the medical institution sends the examination result information to the server.

[1404] Step 8:

[1405] Server: The server receives the medical result information and stores it in association with the user's account.

[1406] Step 9:

[1407] Device: The device will notify the user of the medical examination results and allow them to view them within the app.

[1408] The above are the specific operations in each processing step. In this way, the user can efficiently carry out the process from interviewing the patient to examining the patient, collecting emotional data, and checking the examination results.

[1409] Example 2

[1410] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1411] Conventional medical systems provide diagnostic predictions based solely on the user's medical interview information, and are unable to consider the user's emotional state. This makes it difficult to improve diagnostic accuracy and to properly reflect the impact of the user's emotions on the diagnosis. It is also difficult to provide a rapid examination and efficient appointment system that takes the user's emotional state into account.

[1412] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1413] In this invention, the server includes means for receiving medical interview information input by a user, means for analyzing the received medical interview information, means for providing medical institutions with medical record information generated based on the analysis results, means for receiving appointment information and making an appointment with the medical institution, means for notifying the user of medical examination result information received from the medical institution, means for collecting and analyzing emotional data of the user, and means for generating medical record information by integrating the medical interview information and the emotional data. This enables highly accurate diagnosis prediction that integrates the user's medical interview information and emotional state, and realizes efficient and prompt examinations and appointments.

[1414] "User" refers to an individual or patient who uses the healthcare system.

[1415] A "terminal" is a hardware device used by a user, and refers to an electronic device such as a smartphone or tablet.

[1416] "Server" refers to a computer system that receives, processes, and stores data sent from users and terminals.

[1417] "Medical interview information" refers to data regarding health conditions and symptoms entered by the user.

[1418] "Emotional data" refers to data relating to the psychological state of a user that is analyzed from the user's voice, facial expressions, etc.

[1419] "Analysis" refers to the process of analyzing information and deriving certain results or predictions based on received data.

[1420] A "machine learning model" refers to an algorithm that learns patterns from large amounts of data and uses that learning to make predictions and judgments about new data.

[1421] "Medical record information" refers to information related to a medical diagnosis generated based on medical interview information and emotion data.

[1422] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1423] "Appointment for medical examination" refers to the act of a user reserving a date and time for a medical examination at a medical institution.

[1424] "Notification" refers to the act of sending information from a server to a user.

[1425] MODE FOR CARRYING OUT THE INVENTION

[1426] This invention relates to a system that collects and analyzes user interview information and emotional data to collaborate with medical institutions. The program processing of this system is described in detail below. The hardware and software used include smartphones and tablets as terminals, servers, databases, and machine learning models.

[1427] User Registration and Login

[1428] When a user uses the app for the first time, they tap the "Sign Up" button and the device displays the new registration screen. When the user enters their name, email address, and password and taps the "Submit" button, the device sends the entered data to the server. The server receives this data and stores it in a database (for example, MySQL). Next, when the user enters authentication information on the "Login" screen and taps the Login button, the device sends that information to the server. The server compares the authentication information with the database, and if it is correct, it generates an access token and returns it to the device, allowing the user to log in successfully.

[1429] Filling out a medical questionnaire and collecting emotional data

[1430] When the user taps the "Start Questionnaire" button, the device sends a request to start the questionnaire to the server. The server generates the first questions using Python-based logic and returns them to the device. The device displays the questions to the user, who answers each question. The device then sends the answers to the server, which analyzes the answers, generates the next question, and returns it to the device. During this process, the device launches an emotion analysis engine such as EmotionEngine or Affectiva to analyze the user's voice and facial expressions to collect emotional data.

[1431] Analysis of interview results and emotional data and reflection in medical records

[1432] Once all medical interview and emotion data collection is complete, the server receives this data. The server analyzes the data using machine learning models such as TensorFlow and PyTorch to obtain a diagnostic prediction. The server generates medical record information based on the diagnostic prediction. The generated medical record information includes the user's medical interview information and emotion data. The server provides the medical record information to medical institutions using the FHIR protocol, and the medical institutions prepare it so that it can be used for consultations.

[1433] Appointments and doctor coordination

[1434] The user selects the desired appointment date within the app and enters the appointment information. When the user taps the "Confirm appointment" button, the device sends that information to the server. The server communicates this information to the medical institution and checks whether the appointment is possible. The confirmation result is returned to the device, and if the appointment is approved, a success message is displayed to the user. The medical institution prepares for the appointment based on the interview data and emotion data received through the server. After the examination, the medical institution sends the examination results and prescription information to the server, which notifies the user. The user can check the examination results and prescription information through the app.

[1435] Specific examples

[1436] As an example, consider a case where a user complains of a migraine while feeling emotionally stressed. The user enters "I often get headaches" in the medical interview information and provides detailed information about the frequency and other symptoms. The device uses Emotion Engine to analyze the emotional state from the user's voice and facial expression and detects that the user is feeling stressed. The server receives the medical interview information and emotional data and uses a machine learning model to predict that there is a high correlation between "migraine" and "stress." The server generates medical record information based on this diagnosis prediction and emotional state and sends it to the medical institution. The medical institution receives this information in advance and makes preparations to respond quickly and appropriately when the user visits. After the examination, the medical institution sends the examination results and prescription information to the server, which the user can check through the app.

[1437] Examples of prompt statements

[1438] "The user complains of a headache, and you want to analyze their emotional state. Based on the interview information and emotional data, make a diagnosis prediction and generate a medical record."

[1439] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1440] User Registration and Login

[1441] Step 1:

[1442] When a user uses the app for the first time, they tap the "Sign Up" button. The inputs are the user's name, email address, and password. The output is a successful registration message.

[1443] Step 2:

[1444] The device displays the new registration screen, and the user enters their name, email address, and password and taps the "Submit" button. The input is the user's registration information. The output is the data to be sent to the server.

[1445] Step 3:

[1446] The terminal sends the entered data to the server. The input includes the user's registration information. The output is a message indicating successful data transmission to the server.

[1447] Step 4:

[1448] The server receives the data and stores it in a database. The input is the user's registration information. The output is the registered user information stored in the database.

[1449] Step 5:

[1450] The server returns a message to the terminal indicating that new registration has been completed. As input, there is information indicating successful registration. As output, a notification message is generated for the terminal.

[1451] Step 6:

[1452] The user taps the "Login" button. The input is the user's authentication information (email address, password). The output is a login request message.

[1453] Step 7:

[1454] The device displays the authentication information input screen, the user enters the information, and taps the "Login" button. The input is the user's authentication information. The output is authentication data to be sent to the server.

[1455] Step 8:

[1456] The server authenticates the received authentication information by checking it against a database. The input is the user's authentication information. The output is a success / failure message.

[1457] Step 9:

[1458] If authentication is successful, the server generates an access token and returns it to the terminal. The input is the user's successful authentication information. The output is the generated access token.

[1459] Step 10:

[1460] The user successfully logs in. The input is an access token. The output is a login successful message.

[1461] Filling out a medical questionnaire and collecting emotional data

[1462] Step 1:

[1463] The user taps the "Start medical interview" button. As input, there is a request to start medical interview. As output, a medical interview start request message is generated.

[1464] Step 2:

[1465] The terminal sends a consultation start request to the server. The input is the consultation start request data. The output is a message indicating that the transmission to the server was successful.

[1466] Step 3:

[1467] The server generates the initial question and returns it to the terminal. The input is a request to start the medical interview. The output is the medical interview question data.

[1468] Step 4:

[1469] The terminal displays the question to the user. The input is the question data. The output is a question screen that is displayed to the user.

[1470] Step 5:

[1471] The user answers the questions. The input is the user's answers to the medical interview questions. The output is the answer data.

[1472] Step 6:

[1473] The terminal sends the user's answer to the server. The input is the user's answer data. The output is a successful transmission message to the server.

[1474] Step 7:

[1475] The server analyzes the received answer and generates the next question. The input is the user's answer data. The output is the next question data.

[1476] Step 8:

[1477] The server returns the next question to the terminal, which then displays it to the user. The input is the next question data. The output is a question screen that is displayed to the user.

[1478] Step 9:

[1479] The device launches an emotion analysis engine such as EmotionEngine or Affectiva, and analyzes the user's voice and facial expressions to collect emotional data. The emotion analysis engine takes input, and emotional data is generated as output.

[1480] Step 10:

[1481] Send emotion data to the server. The emotion data is the input. The output is a message indicating that the transmission to the server was successful.

[1482] Analysis of interview results and emotional data and reflection in medical records

[1483] Step 1:

[1484] After all the questionnaire and emotion data collection is completed, the server receives these data. The inputs are the user's questionnaire data and emotion data. The output is stored in the server.

[1485] Step 2:

[1486] The server inputs the received data into a machine learning model such as TensorFlow or PyTorch and analyzes the data. The inputs include medical interview data and emotion data. The output is a diagnostic prediction.

[1487] Step 3:

[1488] Medical record information is generated based on the diagnostic prediction. The input is diagnostic prediction data. The output is medical record information.

[1489] Step 4:

[1490] The generated medical record information is provided to the medical institution using the FHIR protocol. The input is the medical record information. The output is a message indicating successful transmission to the medical institution.

[1491] Appointments and doctor coordination

[1492] Step 1:

[1493] The user selects the desired appointment date within the app and enters the reservation information. The input includes the desired appointment date and reservation information. The output is the reservation information.

[1494] Step 2:

[1495] When the user taps the "Confirm Reservation" button, the device sends the information to the server. The input is the reservation information. The output is a message indicating that the transmission to the server was successful.

[1496] Step 3:

[1497] The server sends the reservation information to the medical institution and checks whether the reservation is possible. The input is the reservation information. The output is reservation confirmation information.

[1498] Step 4:

[1499] The server returns the confirmation result to the terminal. The input is the reservation confirmation information. The output is a transmission success message to the terminal.

[1500] Step 5:

[1501] If the reservation is approved, the terminal displays a success message to the user. The input is the reservation approval information. The output is a success message.

[1502] Step 6:

[1503] The medical institution prepares for the examination based on the medical interview data and emotion data received through the server. The inputs are the medical interview data and emotion data. The output is the examination preparation.

[1504] Step 7:

[1505] After the consultation, the medical institution sends the consultation results and prescription information to the server. The inputs are the consultation results and prescription information. The output is a message indicating that transmission to the server was successful.

[1506] Step 8:

[1507] The server notifies the user. The inputs are the medical examination results and prescription information. The output is a notification message for the user.

[1508] Step 9:

[1509] The user checks the medical examination results and prescription information through the app. The input is a notification message. The output is the medical examination results and prescription information.

[1510] (Application example 2)

[1511] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1512] Current medical systems and security service systems do not fully utilize user interview information and emotional data, making it difficult to provide appropriate diagnoses and propose security measures. Furthermore, few systems take into consideration the user's stress or anxiety, which can lead to a decline in user satisfaction. Furthermore, collaboration between medical institutions and security experts is not smooth, making it difficult to provide efficient responses.

[1513] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving medical interview information and emotion data entered by the user; means for analyzing the received medical interview information and emotion data; means for providing medical record information generated based on the analysis results to the medical institution; means for receiving appointment information and making an appointment with the medical institution; means for notifying the user of the examination result information received from the medical institution; means for inputting a security diagnosis form and collecting emotion data; means for analyzing the collected data using a machine learning model and generating a security diagnosis report; means for proposing security measures to the user based on the analysis results; and means for realizing collaboration with security experts. This enables efficient diagnosis and provision of security measures that take the user's emotional state into consideration.

[1514] "Medical interview information" refers to data about symptoms and health conditions that are entered by the user themselves.

[1515] "Emotion data" refers to information about the user's emotional state that is analyzed from the user's tone of voice, facial expression, and so on.

[1516] "Analysis" refers to making diagnostic predictions and risk assessments based on collected interview information and emotional data.

[1517] "Medical record information" refers to data generated based on analysis results and provided to medical institutions, including diagnostic predictions and emotional states.

[1518] A "medical institution" is a facility that provides medical examinations and treatment, such as a hospital or clinic.

[1519] "Reservation information" refers to data on the consultation date and time desired by the user.

[1520] "Consultation appointment" refers to the procedure for reserving a date and time for consultation with a medical institution.

[1521] "Examination result information" refers to information generated by a medical institution and including examination results, prescription details, and the like.

[1522] The "security assessment form" is a list of questions to assess the user's security status.

[1523] A "security diagnostic report" is a document that evaluates a user's security status based on collected data and suggests risk levels and countermeasures.

[1524] A "security expert" is someone with specialized knowledge in the field of security.

[1525] A "machine learning model" is an algorithm that learns patterns and features based on data and makes predictions and diagnoses.

[1526] The system of this invention is a mechanism for collecting medical interview information and emotion data entered by the user, analyzing this information, generating medical record information and security diagnostic reports, and realizing collaboration with medical institutions and security experts. Specific program processing and the hardware and software used to realize this are described below.

[1527] User Registration and Login

[1528] When a user uses the system for the first time, they tap the "New Registration" button on their smartphone, tablet, or other device, enter their registration information, and send it to the server. This information is stored in a database such as Amazon RDS. When the user uses the system again, they enter their authentication information on the login screen and the server authenticates them before they can access the system.

[1529] Filling out a medical questionnaire and collecting emotional data

[1530] When a user taps the "Start Questionnaire" button while using the system, the server receives the request to start the questionnaire, generates the first question, and returns it to the device. The user answers the question, and the answer data is sent back to the server. During this process, the device uses libraries such as OpenCV and Dlib to analyze the user's voice and facial expressions and collect emotional data. This data is used for subsequent analysis.

[1531] Analysis of interview results and emotional data and reflection in medical records

[1532] Once all interviews and emotional data collection is complete, the server inputs this data into a machine learning model (e.g., scikit-learn, TensorFlow, PyTorch) for analysis. The generated medical record information also includes information about the user's emotional state. The server then sends this medical record information to the medical institution and prepares it for use in the consultation.

[1533] Appointments and doctor coordination

[1534] When a user makes a reservation for a desired consultation date, the information is sent to the medical institution via the server, and the reservation is confirmed. If the reservation is approved, a success message is displayed on the terminal. The medical institution uses this information to efficiently prepare for the consultation.

[1535] Fill out a security assessment form and collect emotional data

[1536] When a user taps the "Start Security Assessment" button using the system, the server receives the assessment start request, generates an initial set of questions, and returns them to the device. The user answers the questions, and the answer data is sent back to the server. During this process, the device uses libraries such as OpenCV and Dlib to analyze the user's voice and facial expressions and collect emotional data. This data is used for subsequent analysis.

[1537] Analysis of security diagnostic results and sentiment data and report generation

[1538] Once all interviews and emotional data collection is complete, the server inputs this data into a machine learning model (e.g., scikit-learn, TensorFlow, PyTorch) for analysis. The generated security assessment report includes information about the user's emotional state. The server provides this report to the user and allows them to connect with security experts if necessary.

[1539] Examples of specific examples and prompts

[1540] For example, if a user answers the questionnaire information by saying, "I change my password every six months. I don't use two-factor authentication," and the emotional data indicates that they are in a state of moderate stress, the system will assess them as "medium risk" and suggest the following countermeasures: specifically, "enable two-factor authentication," "use a password manager," and "regular data backup."

[1541] Prompt for the generative AI model:

[1542] Please conduct a security assessment based on the following user data and provide the risk level and specific measures.

[1543] User Data:

[1544] Answer: "I change my password every six months" and "I don't use two-factor authentication"

[1545] Emotional data: Slightly stressed

[1546] This program and mechanism makes it possible to provide efficient security measures that take into account the user's emotional state.

[1547] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1548] Step 1:

[1549] When a user uses the app for the first time, they tap the "Sign Up" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the input information to the server, which stores it in a database. This creates a user account for authentication.

[1550] Step 2:

[1551] The user enters authentication information (email address, password) on the login screen and taps the login button. The device sends the authentication information to the server. The server compares it with the information in the database, and if authentication is successful, generates an access token and returns it to the device. The device saves this access token and uses it in subsequent communications.

[1552] Step 3:

[1553] When the user taps the "Start medical interview" button, the device sends a medical interview start request to the server. The server generates a medical interview form and returns the initial questions to the device. The device displays the questions to the user and provides an interface for inputting answers.

[1554] Step 4:

[1555] The user enters their answers into the questionnaire form and taps the send button. The device then sends the answer data to the server. At that time, the device uses a camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional data. The analysis results are also sent to the server.

[1556] Step 5:

[1557] The server analyzes the medical interview information and emotional data it receives. Specifically, it uses machine learning models (such as scikit-learn, TensorFlow, and PyTorch) to predict diagnoses and generate analysis results, including the user's emotional state. It then generates medical record information based on the analysis results and prepares the data for transmission to medical institutions.

[1558] Step 6:

[1559] The server sends the generated medical record information to the designated medical institution, which receives the information and prepares for diagnosis and treatment.

[1560] Step 7:

[1561] When a user makes a reservation for a desired consultation date within the app, the device sends the reservation information to the server. The server then contacts the medical institution to confirm whether the reservation can be made and returns the result to the device. If the reservation is approved, the device displays a success message to the user.

[1562] Step 8:

[1563] After the consultation, the medical institution sends the consultation results and prescription information to the server, which then notifies the user, who can then check the consultation results through the app.

[1564] Step 9:

[1565] When the user taps the "Start Security Assessment" button, the device sends a request to the server to start the assessment. The server generates an assessment form and returns the initial questions to the device. The device then displays the questions to the user and provides an interface for entering answers.

[1566] Step 10:

[1567] The user enters their answers into the security diagnostic form and taps the submit button. The device then sends the answer data to the server. At that time, the device uses the camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional data. The analysis results are also sent to the server.

[1568] Step 11:

[1569] The server analyzes the security diagnostic information and emotion data received, performs security diagnostics using machine learning models and evaluates the risk level, generates a security diagnostic report based on the analysis results, and prepares the data to provide to the user.

[1570] Step 12:

[1571] Provides the user with the server-generated security assessment report and suggests countermeasures. If necessary, creates a request to connect the user with a security expert and submits it with the appropriate information.

[1572] The above processing steps make it possible to provide efficient diagnostic and security measures that take into account the emotional state of the user.

[1573] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1574] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1575] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1576] [Fourth embodiment]

[1577] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1578] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1579] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1580] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1581] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1582] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1583] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1584] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1585] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1586] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1587] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1588] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1589] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1590] The present invention is a system that uses a medical app to collect medical interview information from users, analyzes it to generate medical record information, and then connects with medical institutions. The program processing of this system is explained in natural language below. Specific examples are also provided for further explanation.

[1591] User Registration and Login

[1592] When a user uses the app for the first time, they tap the "New Registration" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the entered information to the server, which saves it in a database. Next, the user enters authentication information on the "Login" screen and taps the login button, which the device sends to the server. The server performs authentication and returns an access token to the device, completing the login.

[1593] Fill out the medical questionnaire form

[1594] When the user taps the "Start medical interview" button, the device sends a medical interview start request to the server. The server generates the first question and returns it to the device, which then displays it to the user. After that, the user answers each question, and the device sends them to the server. The server analyzes the answers, determines the next question, and returns it to the device. This process is repeated until all questions have been asked.

[1595] Analysis of interview results and reflection in medical records

[1596] Once all medical interviews are completed, the server analyzes the user's interview data using a machine learning model and makes a diagnosis prediction. The server then creates a request to send the medical record information, including the generated diagnosis prediction, to the target medical institution. The medical institution's system receives this medical record information and prepares it for use in consultations.

[1597] Appointments and doctor coordination

[1598] When a user makes an appointment for a desired consultation date within the app, the device sends the appointment information to the server. The server then connects with the medical institution to confirm whether the appointment is possible and returns the result to the device. If the appointment is approved, the device displays a success message to the user. The medical institution prepares to efficiently conduct the consultation based on the medical interview data received via the server. After the consultation, the medical institution sends the examination results and prescription information to the server, which then notifies the user. The user can then check the examination results through the app.

[1599] Specific examples

[1600] As an example, consider the case where a user complaining of migraines uses this system. The user enters "I often get headaches" in the medical interview information and provides detailed information about the frequency and other symptoms. The server receives this information, and the machine learning model predicts that the diagnosis is likely "migraine." The server generates medical record information including this diagnosis prediction and sends it to the medical institution. By receiving this information in advance, the medical institution can quickly begin consultation when the user visits the hospital without having to reconfirm the medical interview details. After the consultation, the medical institution sends the examination results and prescription details to the server, allowing the user to check them through the app.

[1601] This will improve the efficiency of medical interviews and the accuracy of diagnoses, making the system beneficial for both users and medical institutions.

[1602] The processing flow will be explained below.

[1603] User Registration and Login

[1604] Step 1:

[1605] User: The user launches the app and taps the "Sign Up" button.

[1606] Step 2:

[1607] Device: The device will display the new registration screen.

[1608] Step 3:

[1609] User: The user enters their name, email address, and password, then taps the submit button.

[1610] Step 4:

[1611] Device: The device creates a request to send the input information to the server.

[1612] Step 5:

[1613] Server: The server receives the request and performs validation of the input information.

[1614] Step 6:

[1615] Server: If the server is happy with the information, it stores it in the database.

[1616] Step 7:

[1617] Server: The server returns a registration success response to the device.

[1618] Step 8:

[1619] Terminal: The terminal displays a successful registration message to the user.

[1620] Step 9:

[1621] User: The user enters their credentials on the "Login" screen and taps the login button.

[1622] Step 10:

[1623] Device: The device sends the entered authentication information to the server.

[1624] Step 11:

[1625] Server: The server verifies the authentication information, generates an access token, and returns a response to the device.

[1626] Step 12:

[1627] Device: The device stores the access token and uses it in subsequent requests.

[1628] Fill out the medical questionnaire form

[1629] Step 1:

[1630] User: The user taps the "Start Interview" button.

[1631] Step 2:

[1632] Terminal: The terminal sends a request to start an interview to the server.

[1633] Step 3:

[1634] Server: The server generates the initial medical interview questions and returns them to the terminal.

[1635] Step 4:

[1636] Terminal: The terminal displays the initial question to the user.

[1637] Step 5:

[1638] User: The user answers the question and taps the Next button.

[1639] Step 6:

[1640] Device: The device sends the answer to the server.

[1641] Step 7:

[1642] Server: The server stores the received answer in a database and generates the next question.

[1643] Step 8:

[1644] Server: The server sends the following question to the device:

[1645] Step 9:

[1646] Terminal: The terminal displays the next question to the user. This process is repeated until all questions have been answered.

[1647] Analysis of interview results and reflection in medical records

[1648] Step 1:

[1649] Server: After all questionnaire responses are completed, the server inputs the questionnaire data into the machine learning model and begins analysis.

[1650] Step 2:

[1651] Server: The server obtains the analysis results and generates medical record information including diagnostic predictions.

[1652] Step 3:

[1653] Server: The server creates a request to send the medical record information to the medical institution.

[1654] Step 4:

[1655] Server: The server sends the medical record information to the target medical institution.

[1656] Step 5:

[1657] Medical institutions: Medical institutions receive the medical record information and use it for medical examinations.

[1658] Appointments and doctor coordination

[1659] Step 1:

[1660] User: The user opens the "Appointment" screen within the app and selects the desired appointment date and time.

[1661] Step 2:

[1662] Terminal: The terminal makes a request to send the reservation information to the server.

[1663] Step 3:

[1664] Server: The server receives the reservation request and checks whether the reservation is available by connecting with the medical institution's system.

[1665] Step 4:

[1666] Server: The server returns the reservation approval result to the terminal.

[1667] Step 5:

[1668] Terminal: The terminal displays a confirmation message to the user.

[1669] Step 6:

[1670] Medical institution: The medical institution receives the reservation information and prepares for the examination based on the user's medical interview data.

[1671] Step 7:

[1672] Medical institution: After the examination, the medical institution sends the examination result information to the server.

[1673] Step 8:

[1674] Server: The server receives the medical result information and stores it in association with the user's account.

[1675] Step 9:

[1676] Device: The device will notify the user of the medical examination results and allow them to view them within the app.

[1677] The above is a detailed description of the operations performed in each processing step. In this way, the user can efficiently carry out the process from interviewing the patient to examining the patient and confirming the examination results.

[1678] Example 1

[1679] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1680] Modern medical care involves many processes, such as collecting and analyzing medical history information, scheduling appointments, and notifying patients of diagnosis results. Each process requires time and effort, creating a challenge. Furthermore, doctors' time spent comprehending detailed medical history information during consultations reduces the efficiency of the entire consultation. Conventional systems have struggled to manage these processes in an integrated manner and process them efficiently and accurately.

[1681] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1682] In this invention, the server includes a means for receiving medical interview information input by a user, a means for analyzing the received medical interview information, a means for providing electronic medical record information generated based on the analysis results to a medical institution, a means for receiving appointment information and making an appointment with the medical institution, and a means for notifying the user of the examination result information received from the medical institution. This enables efficient collection and analysis of medical interview information, prompt notification of diagnosis results, and efficient examinations through cooperation with medical institutions.

[1683] "User" refers to an individual or corporation that uses the system.

[1684] "Medical interview information" refers to information about health conditions and symptoms provided by the user.

[1685] "Electronic medical record information" refers to health records generated from medical interview information and analysis results.

[1686] "Medical institution" refers to an organization such as a hospital or clinic that provides medical care or treatment.

[1687] "Appointment for medical examination" refers to the act of reserving a desired date and time for medical examination at a medical institution by a user.

[1688] "Examination result information" refers to information regarding diagnoses and prescriptions provided by medical institutions after examinations.

[1689] A "server" is a computer device that is the core of a system and that processes and stores data and communicates with other devices.

[1690] A "machine learning model" refers to an algorithm that learns from large amounts of data and makes predictions and classifications.

[1691] "Notification" refers to a means by which the system notifies the user of information.

[1692] "Receive" refers to the act of obtaining data from another device or system.

[1693] "Analysis" refers to the act of examining information in detail based on received data and understanding its meaning and structure.

[1694] The present invention is a system that uses a medical application to collect medical interview information from users, analyzes the information, generates electronic medical records, and collaborates with medical institutions. This system consists of a server, terminals, and users.

[1695] Program processing explanation

[1696] User Registration and Login

[1697] When a user launches the app and taps the "New Registration" button, the device displays the new registration screen. When the user enters their name, email address, and password and taps the "Submit" button, the device sends the input information to the server. The server stores the received information in a database.

[1698] When a user enters their email address and password on the "Login" screen and taps the login button, the device sends the entered information to the server. The server performs authentication and returns an access token to the device, completing the login. The technology used is a smartphone or tablet as the device, and a cloud service (e.g., Amazon Web Services, Microsoft Azure) as the server.

[1699] Fill out the medical questionnaire form

[1700] When the user taps the "Start medical interview" button, the device sends a request to the server to start the medical interview. The server generates the first questions and returns them to the device. The device displays the questions, and the user answers each one. When the device sends the answers to the server, the server analyzes the answers and determines the next question. This process is repeated until all questions have been asked.

[1701] The technologies used include stream processing using Apache Kafka for real-time analysis on the server side, and natural language processing (NLP) libraries implemented in Python (e.g., SpaCy, NLTK).

[1702] Analysis of interview results and reflection in medical records

[1703] Once all interviews are completed, the device sends an end request to the server. The server inputs the interview data into a machine learning model for analysis. Specific technologies used include Scikit-Learn and TensorFlow. The server generates a diagnostic prediction and formats it as electronic medical record information. The server then creates and sends a request to send this electronic medical record information to the medical institution.

[1704] The medical institution's system receives electronic medical record information and prepares it so that it can be used for consultations. The technology used utilizes the standard FHIR (Fast Healthcare Interoperability Resources).

[1705] Appointments and doctor coordination

[1706] When a user makes a reservation for a desired consultation date within the app, the device sends the reservation information to the server. The server then connects with the medical institution to confirm whether the reservation is possible. If the reservation is approved, the device displays a success message to the user.

[1707] On the day of the consultation, the medical institution will prepare for the consultation efficiently based on the medical interview data received through the server. After the consultation, the medical institution will send the consultation results and prescription information to the server, which will notify the user. The user can then check the consultation results through the application.

[1708] As a specific example of how it works, for example, a user who complains of migraines uses the system and enters "I often get headaches" into the medical interview information, and provides detailed information about the frequency and other symptoms. Based on this information, the server uses a machine learning model to predict the possibility of a "migraine," generates electronic medical record information, and sends it to the medical institution. By receiving this information in advance, the medical institution can prepare for a prompt examination.

[1709] Prompt Sentence Examples

[1710] Here are some example prompts to input to a generative AI model:

[1711] "What API endpoints do you use to process user registration?"

[1712] "Please explain how to analyze medical interview data using a machine learning model."

[1713] "Please tell me about the process for sending medical record information according to the FHIR standard."

[1714] In this way, this system uses technically linked components to improve the efficiency of medical interviews and the accuracy of diagnoses.

[1715] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1716] System program processing steps

[1717] User Registration and Login

[1718] Step 1:

[1719] The user launches the app and taps the "New Registration" button. The device displays the new registration screen. The user enters their name, email address, and password, and taps the send button. The input is their name, email address, and password, and this information is sent from the device to the server as output.

[1720] Step 2:

[1721] The server stores the received information in a database. In this storage process, an INSERT query is executed using a database management system (e.g., MySQL) to record the user information. The input is the user information received from the terminal, and the output is the result stored in the database.

[1722] Step 3:

[1723] The user enters their email address and password on the "Login" screen and taps the login button. The device sends the authentication information to the server. The input is the email address and password, and the output is this information sent from the device to the server.

[1724] Step 4:

[1725] The server references the database and checks the authentication information. If authentication is successful, it generates a JWT (JSON Web Token) and returns it to the terminal as an access token. If authentication fails, it generates an error message and returns it to the terminal. The input is authentication information, and the output is an access token or an error message.

[1726] Fill out the medical questionnaire form

[1727] Step 5:

[1728] The user taps the "Start medical interview" button. The terminal sends a medical interview start request to the server. The input is the medical interview start request, and the output is that this request is sent from the terminal to the server.

[1729] Step 6:

[1730] The server generates the initial question and returns it to the terminal. At this time, the server executes an algorithm that randomly selects a question from a list of questions prepared in advance. The input is the request to start the medical interview, and the output is the initial question.

[1731] Step 7:

[1732] The terminal displays a question, the user answers the question, the input is the initial question, and the output is the user's answer.

[1733] Step 8:

[1734] The device sends the user's answer to the server. The server analyzes the received answer using an NLP algorithm (e.g., SpaCy, NLTK). Based on the analysis results, the next question is dynamically generated. The input is the user's answer, and the output is the next question.

[1735] Step 9:

[1736] The server generates the next question and sends it back to the terminal, which displays it to the user. This process is repeated until all questions have been asked. The input is the next question, and the output is that question will be displayed on the terminal.

[1737] Analysis of interview results and reflection in medical records

[1738] Step 10:

[1739] When all the inquiries are completed, the terminal sends an end request to the server. The input is the inquiry end request, and the output is that the request is sent from the terminal to the server.

[1740] Step 11:

[1741] The server inputs the received medical interview data into a machine learning model (e.g., Scikit-Learn, TensorFlow) for analysis. The machine learning model generates a diagnostic prediction. The input is the medical interview data, and the output is a diagnostic prediction.

[1742] Step 12:

[1743] The server formalizes the generated diagnosis prediction as electronic medical record information. Then, a request including this electronic medical record information is sent to the medical institution. The input is the diagnosis prediction, and the output is the electronic medical record information.

[1744] Step 13:

[1745] The medical institution's system receives the electronic medical record information and prepares it so that it can be used for consultations. The input is the electronic medical record information, and the output is the completion of consultation preparation.

[1746] Appointments and doctor coordination

[1747] Step 14:

[1748] The user makes a reservation for the desired consultation date within the app, and the device sends the reservation information to the server. The input is the reservation information, and the output is that information is sent from the device to the server.

[1749] Step 15:

[1750] The server receives the reservation information and sends a request to the medical institution to confirm whether the reservation is available. The input is the reservation information, and the output is confirmation of whether the reservation is available.

[1751] Step 16:

[1752] The server receives a response from the medical institution, and if the reservation is approved, it returns a success message to the terminal. The input is the response from the medical institution, and the output is the success message.

[1753] Step 17:

[1754] On the day of the consultation, the server retransmits the received medical interview data to the medical institution, allowing the medical institution to conduct the consultation efficiently. The input is the medical interview data and reservation information, and the output is the preparation for the consultation at the medical institution.

[1755] Step 18:

[1756] After the consultation, the medical institution sends the consultation results and prescription information to the server, and the server notifies the user of the information. The input is the consultation results and prescription information, and the output is a notification to the user. This notification allows the user to check the consultation results and prescription information through the app.

[1757] (Application example 1)

[1758] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1759] In today's medical environment, the process from medical interview to examination is complicated, resulting in time-consuming and laborious tasks for both users and medical facilities. Furthermore, there are limited ways for users to conduct medical interviews and receive diagnostic predictions without visiting a medical institution, making it difficult to provide prompt and efficient medical services. Furthermore, methods for conducting medical interviews and diagnoses in virtual spaces are not widespread, preventing an improved user experience and hindering the improvement of the quality of medical services.

[1760] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1761] In this invention, the server includes means for receiving medical interview information input by a user, means for analyzing the received medical interview information and making a diagnosis prediction, means for providing medical record information generated based on the analysis results to a medical facility, means for receiving consultation appointment information and making a consultation appointment with the medical facility, means for notifying the user of the examination result information received from the medical facility, and means for the user to input medical interview information in a virtual store and collaborate with a virtual doctor. This enables the user to efficiently conduct a medical interview in a virtual space, quickly receive a diagnosis prediction, and, if necessary, make a consultation appointment.

[1762] "User" refers to an individual who uses medical services.

[1763] "Medical interview information" refers to information regarding symptoms and health conditions entered by the user.

[1764] "Means for receiving" refers to the function by which the server receives medical interview information and reservation information from the user.

[1765] "Means of analysis" refers to the function of using machine learning models and algorithms to make diagnostic predictions based on the received medical interview information.

[1766] "Diagnosis prediction" refers to the results of predicting the user's symptoms and condition based on medical interview information.

[1767] "Medical record information" refers to digital information that summarizes the user's health status and diagnostic predictions.

[1768] "Means for providing" refers to the function for transmitting the generated medical record information to a medical facility.

[1769] "Reservation information" refers to information regarding the date and time of the consultation desired by the user.

[1770] "Means for making appointments" refers to the function of making appointments with medical facilities based on the user's desired date and time.

[1771] "Means for notifying" refers to the function for transmitting medical examination result information received from medical facilities to users.

[1772] A "virtual store" refers to a virtual space accessible via the Internet where users can receive medical services.

[1773] "Virtual doctor" refers to a digital doctor who conducts medical interviews with users within a virtual store.

[1774] A "machine learning model" refers to an artificial intelligence (AI) algorithm used for data analysis and prediction.

[1775] An "interactive experience" refers to an experience that is provided through two-way interaction with the user.

[1776] This invention provides a system that enables users to efficiently conduct medical interviews in a virtual store, quickly receive a diagnosis, and, if necessary, schedule an appointment. The system has a series of functions that receive and analyze the medical interview information entered by the user, perform a diagnosis prediction, and provide medical record information to medical facilities.

[1777] The system consists of an application installed on the user's smartphone and a backend service running on a server. When the user enters medical history information through the application, the information is sent to the server. The server analyzes the received information and makes a diagnosis prediction using a machine learning model. The specific software used for this analysis is the latest machine learning libraries such as TensorFlow and PyTorch.

[1778] The server then generates a medical record based on the diagnosis prediction and provides it to the medical facility. This medical record details the user's symptoms and diagnosis prediction. When the user enters the desired consultation date and time, the reservation information is also sent to the server, and the appointment is completed through collaboration with the medical facility.

[1779] After the user receives a medical examination, the medical facility sends the examination result information to the server, which then notifies the user, who can then check the information through the application.

[1780] As a concrete example, the following scenario shows a user named Tom using the app to complete a headache interview and ultimately receive a diagnosis of migraine.

[1781] User: Tom, please enter your name and email.

[1782] System: Name: Tom

[1783] System: Email: tom@example.com

[1784] System: Please answer the following questions about your symptoms.

[1785] System: Question: How often do you experience headaches?

[1786] User: Almost daily.

[1787] System: Question: How severe are your headaches on a scale of 1 to 10?

[1788] User: 7

[1789] System: Analyzing your answers...

[1790] System: Based on your answers, there is a high likelihood of migraine. Would you like to book an appointment with a doctor?

[1791] User: Yes.

[1792] System: Booking an appointment on November 1, 2023...

[1793] System: Appointment confirmed. You will receive further instructions via email.

[1794] In this way, users can use a system that integrates medical interviews, diagnosis predictions, and appointment bookings in a virtual space, allowing them to receive efficient and prompt medical services.

[1795] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1796] Step 1:

[1797] A user launches the app and registers or logs in.

[1798] Specific operation: The device displays a new registration screen or login screen, and the user enters their name, email address, and password. The input information is sent to the server, which stores it in a database or authenticates it, and if authentication is successful, returns an access token to the device.

[1799] Input: User's name, email address, and password

[1800] Output: Access token

[1801] Step 2:

[1802] The user taps the "Start medical interview" button.

[1803] Specific operation: The terminal sends a request to start an interview to the server, and the server generates the first question and sends it to the terminal. The terminal displays the question to the user.

[1804] Input: Access token, medical interview start request

[1805] Output: First question

[1806] Step 3:

[1807] The user answers each question.

[1808] Specific operation: The user enters an answer, and the device sends the answer to the server. The server analyzes the answer, generates the next question, and returns it to the device. This process is repeated until all questions have been answered.

[1809] Input: User's answer

[1810] Output: Next question

[1811] Step 4:

[1812] The interview information is analyzed to generate a diagnostic prediction.

[1813] Specific operation: The server receives the answers to all questions, analyzes the user's medical interview data using a machine learning model, makes a diagnosis prediction, and generates the results as medical record information.

[1814] Input: All response data

[1815] Output: Diagnosis prediction, medical record information

[1816] Step 5:

[1817] Provide medical record information to medical facilities.

[1818] Specific operation: The server creates a request to send the medical record information including the generated diagnosis prediction to the medical facility, and sends it to the medical facility. The medical facility receives the medical record information and prepares it for use in consultations.

[1819] Input: Medical record information

[1820] Output: Medical record information sent

[1821] Step 6:

[1822] A user makes an appointment.

[1823] Specific operation: The user inputs the desired appointment date and time, and the terminal sends the reservation information to the server. The server then connects with the medical facility to check whether the appointment is available. The result is returned to the terminal, which then displays a reservation confirmation message to the user.

[1824] Input: Consultation appointment information

[1825] Output: Reservation confirmation message

[1826] Step 7:

[1827] Notify the user of the medical examination result information.

[1828] Specific operation: After the examination, the medical facility sends the examination result information to the server, which then notifies the user's device, where the user can check the examination result through the app.

[1829] Input: Examination result information

[1830] Output: Examination result notification

[1831] Through these steps, users can conduct interviews in a virtual space, receive a diagnosis prediction, and schedule an appointment if necessary.

[1832] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1833] The present invention is a system that uses a medical app to collect medical interview information and emotional data from users, analyzes this data to generate medical record information, and then collaborates with medical institutions. The program processing of this system is explained in natural language below. Specific examples are also provided for further explanation.

[1834] User Registration and Login

[1835] When a user uses the app for the first time, they tap the "New Registration" button. The device displays the new registration screen, where the user enters their name, email address, and password and taps the submit button. The device sends the entered information to the server, which saves it in a database. Next, the user enters authentication information on the "Login" screen and taps the login button, which the device sends to the server. The server performs authentication and returns an access token to the device, completing the login.

[1836] Filling out a medical questionnaire and collecting emotional data

[1837] When the user taps the "Start medical interview" button, the device sends a request to start the medical interview to the server. The server generates the first question and returns it to the device, which then displays it to the user. The user then answers each question, which the device then sends to the server. The server analyzes the answers, determines the next question, and returns it to the device. During this process, the device also activates an emotion engine that analyzes the user's voice and facial expressions, collecting the user's emotional data.

[1838] Analysis of interview results and emotional data and reflection in medical records

[1839] After all interviews and emotional data have been collected, the server inputs this data into a machine learning model and begins analysis. The server obtains the analysis results and generates medical record information including a diagnosis prediction. The generated medical record information also includes information about the user's emotional state. The server creates a request to send the medical record information to the target medical institution and sends it to the target medical institution. The medical institution's system receives this medical record information and it is ready to be used for consultations.

[1840] Appointments and doctor coordination

[1841] When a user makes an appointment for a desired consultation date within the app, the device sends the appointment information to the server. The server then connects with the medical institution to confirm whether the appointment is possible and returns the result to the device. If the appointment is approved, the device displays a success message to the user. The medical institution prepares for an efficient consultation based on the medical interview data and emotion data received via the server. After the consultation, the medical institution sends the examination results and prescription information to the server, which then notifies the user. The user can then check the examination results through the app.

[1842] Specific examples

[1843] As an example, consider a case where a user complains of a migraine while feeling emotionally stressed. The user enters "I often get headaches" in the medical interview information and provides detailed responses about the frequency and other symptoms. The device uses an emotion engine to analyze the user's emotional state from their voice and facial expressions, detecting, for example, that the user is feeling stressed. The server receives the medical interview information and emotion data, and a machine learning model predicts that there is a high correlation between the possibility of "migraine" and "stress." The server generates a medical record including this diagnosis prediction and emotional state and sends it to the medical institution. By receiving this information in advance, the medical institution can quickly begin a consultation that takes into account not only the medical interview content but also the user's emotional state when they visit. After the consultation, the medical institution sends the examination results and prescription details to the server, allowing the user to check them through the app.

[1844] This will improve the efficiency of medical interviews and the accuracy of diagnoses, making the system beneficial for both users and medical institutions. Utilizing emotion data will enable deeper understanding and appropriate responses, improving the quality of patient care.

[1845] The processing flow will be explained below.

[1846] User Registration and Login

[1847] Step 1:

[1848] User: The user launches the app and taps the "Sign Up" button.

[1849] Step 2:

[1850] Device: The device will display the new registration screen.

[1851] Step 3:

[1852] User: The user enters their name, email address, and password, then taps the submit button.

[1853] Step 4:

[1854] Device: The device creates a request to send the input information to the server.

[1855] Step 5:

[1856] Server: The server receives the request and performs validation of the input information.

[1857] Step 6:

[1858] Server: If the server is happy with the information, it stores it in the database.

[1859] Step 7:

[1860] Server: The server returns a registration success response to the device.

[1861] Step 8:

[1862] Terminal: The terminal displays a successful registration message to the user.

[1863] Step 9:

[1864] User: The user enters their credentials on the "Login" screen and taps the login button.

[1865] Step 10:

[1866] Device: The device sends the entered authentication information to the server.

[1867] Step 11:

[1868] Server: The server verifies the authentication information, generates an access token, and returns a response to the device.

[1869] Step 12:

[1870] Device: The device stores the access token and uses it in subsequent requests.

[1871] Filling out a medical questionnaire and collecting emotional data

[1872] Step 1:

[1873] User: The user taps the "Start Interview" button.

[1874] Step 2:

[1875] Terminal: The terminal sends a request to start an interview to the server.

[1876] Step 3:

[1877] Server: The server generates the initial medical interview questions and returns them to the terminal.

[1878] Step 4:

[1879] Terminal: The terminal displays the initial question to the user.

[1880] Step 5:

[1881] User: The user answers the question and taps the Next button.

[1882] Step 6:

[1883] Device: The device makes a request that sends the answer to the server.

[1884] Step 7:

[1885] Server: The server stores the received answer in a database and generates the next question.

[1886] Step 8:

[1887] Device: At the same time, the device activates an emotion engine that analyzes the user's voice and facial expressions.

[1888] Step 9:

[1889] Emotion engine: The emotion engine collects the user's emotional data and returns the analysis results to the device.

[1890] Step 10:

[1891] Server: The server sends the following question to the device:

[1892] Step 11:

[1893] Terminal: The terminal displays the next question to the user. This process is repeated until all questions have been answered.

[1894] Analysis of interview results and emotional data and reflection in medical records

[1895] Step 1:

[1896] Server: Once all the questionnaire responses and emotional data have been collected, the server inputs the data into the machine learning model and begins analysis.

[1897] Step 2:

[1898] Server: The server obtains the analysis results and generates medical record information including diagnostic predictions.

[1899] Step 3:

[1900] Server: The generated medical record information also includes information about the user's emotional state.

[1901] Step 4:

[1902] Server: The server creates a request to send the medical record information to the medical institution.

[1903] Step 5:

[1904] Server: The server sends the medical record information to the target medical institution.

[1905] Step 6:

[1906] Medical institutions: Medical institutions receive the medical record information and use it for medical examinations.

[1907] Appointments and doctor coordination

[1908] Step 1:

[1909] User: The user opens the "Appointment" screen within the app and selects the desired appointment date and time.

[1910] Step 2:

[1911] Terminal: The terminal makes a request to send the reservation information to the server.

[1912] Step 3:

[1913] Server: The server receives the reservation request and checks whether the reservation is available by connecting with the medical institution's system.

[1914] Step 4:

[1915] Server: The server returns the reservation approval result to the terminal.

[1916] Step 5:

[1917] Terminal: The terminal displays a confirmation message to the user.

[1918] Step 6:

[1919] Medical institution: The medical institution receives the appointment information and prepares for the examination based on the user's medical interview data and emotion data.

[1920] Step 7:

[1921] Medical institution: After the examination, the medical institution sends the examination result information to the server.

[1922] Step 8:

[1923] Server: The server receives the medical result information and stores it in association with the user's account.

[1924] Step 9:

[1925] Device: The device will notify the user of the medical examination results and allow them to view them within the app.

[1926] The above are the specific operations in each processing step. In this way, the user can efficiently carry out the process from interviewing the patient to examining the patient, collecting emotional data, and checking the examination results.

[1927] Example 2

[1928] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1929] Conventional medical systems provide diagnostic predictions based solely on the user's medical interview information, and are unable to consider the user's emotional state. This makes it difficult to improve diagnostic accuracy and to properly reflect the impact of the user's emotions on the diagnosis. It is also difficult to provide a rapid examination and efficient appointment system that takes the user's emotional state into account.

[1930] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1931] In this invention, the server includes means for receiving medical interview information input by a user, means for analyzing the received medical interview information, means for providing medical institutions with medical record information generated based on the analysis results, means for receiving appointment information and making an appointment with the medical institution, means for notifying the user of medical examination result information received from the medical institution, means for collecting and analyzing emotional data of the user, and means for generating medical record information by integrating the medical interview information and emotional data. This enables highly accurate diagnosis prediction that integrates the user's medical interview information and emotional state, and realizes efficient and prompt examinations and appointments.

[1932] "User" refers to an individual or patient who uses the healthcare system.

[1933] A "terminal" is a hardware device used by a user, and refers to an electronic device such as a smartphone or tablet.

[1934] "Server" refers to a computer system that receives, processes, and stores data sent from users and terminals.

[1935] "Medical interview information" refers to data regarding health conditions and symptoms entered by the user.

[1936] "Emotional data" refers to data relating to the psychological state of a user that is analyzed from the user's voice, facial expressions, etc.

[1937] "Analysis" refers to the process of analyzing information and deriving certain results or predictions based on received data.

[1938] A "machine learning model" refers to an algorithm that learns patterns from large amounts of data and uses that learning to make predictions and judgments about new data.

[1939] "Medical record information" refers to information related to a medical diagnosis generated based on medical interview information and emotion data.

[1940] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1941] "Appointment for medical examination" refers to the act of a user reserving a date and time for a medical examination at a medical institution.

[1942] "Notification" refers to the act of sending information from a server to a user.

[1943] MODE FOR CARRYING OUT THE INVENTION

[1944] This invention relates to a system that collects and analyzes user interview information and emotional data to collaborate with medical institutions. The program processing of this system is described in detail below. The hardware and software used include smartphones and tablets as terminals, servers, databases, and machine learning models.

[1945] User Registration and Login

[1946] When a user uses the app for the first time, they tap the "Sign Up" button and the device displays the new registration screen. When the user enters their name, email address, and password and taps the "Submit" button, the device sends the entered data to the server. The server receives this data and stores it in a database (for example, MySQL). Next, when the user enters authentication information on the "Login" screen and taps the Login button, the device sends that information to the server. The server compares the authentication information with the database, and if it is correct, it generates an access token and returns it to the device, allowing the user to log in successfully.

[1947] Filling out a medical questionnaire and collecting emotional data

[1948] When the user taps the "Start Questionnaire" button, the device sends a request to start the questionnaire to the server. The server generates the first questions using Python-based logic and returns them to the device. The device displays the questions to the user, who answers each question. The device then sends the answers to the server, which analyzes the answers, generates the next question, and returns it to the device. During this process, the device launches an emotion analysis engine such as EmotionEngine or Affectiva to analyze the user's voice and facial expressions to collect emotional data.

[1949] Analysis of interview results and emotional data and reflection in medical records

[1950] Once all medical interview and emotion data collection is complete, the server receives this data. The server analyzes the data using machine learning models such as TensorFlow and PyTorch to obtain a diagnostic prediction. The server generates medical record information based on the diagnostic prediction. The generated medical record information includes the user's medical interview information and emotion data. The server provides the medical record information to medical institutions using the FHIR protocol, and the medical institutions prepare it so that it can be used for consultations.

[1951] Appointments and doctor coordination

[1952] The user selects the desired appointment date within the app and enters the appointment information. When the user taps the "Confirm appointment" button, the device sends that information to the server. The server communicates this information to the medical institution and checks whether the appointment is possible. The confirmation result is returned to the device, and if the appointment is approved, a success message is displayed to the user. The medical institution prepares for the appointment based on the interview data and emotion data received through the server. After the examination, the medical institution sends the examination results and prescription information to the server, which notifies the user. The user can check the examination results and prescription information through the app.

[1953] Specific examples

[1954] As an example, consider a case where a user complains of a migraine while feeling emotionally stressed. The user enters "I often get headaches" in the medical interview information and provides detailed information about the frequency and other symptoms. The device uses Emotion Engine to analyze the emotional state from the user's voice and facial expression and detects that the user is feeling stressed. The server receives the medical interview information and emotional data and uses a machine learning model to predict that there is a high correlation between "migraine" and "stress." The server generates medical record information based on this diagnosis prediction and emotional state and sends it to the medical institution. The medical institution receives this information in advance and makes preparations to respond quickly and appropriately when the user visits. After the examination, the medical institution sends the examination results and prescription information to the server, which the user can check through the app.

[1955] Examples of prompt statements

[1956] "The user complains of a headache, and you want to analyze their emotional state. Based on the interview information and emotional data, make a diagnosis prediction and generate a medical record."

[1957] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1958] User Registration and Login

[1959] Step 1:

[1960] When a user uses the app for the first time, they tap the "Sign Up" button. The inputs are the user's name, email address, and password. The output is a successful registration message.

[1961] Step 2:

[1962] The device displays the new registration screen, and the user enters their name, email address, and password and taps the "Submit" button. The input is the user's registration information. The output is the data to be sent to the server.

[1963] Step 3:

[1964] The terminal sends the entered data to the server. The input includes the user's registration information. The output is a message indicating successful data transmission to the server.

[1965] Step 4:

[1966] The server receives the data and stores it in a database. The input is the u...

Claims

1. means for receiving medical interview information input by a user; A means for analyzing the received medical interview information; A means for providing medical record information generated based on the analysis results to a medical institution; A means for receiving reservation information and making an appointment with a medical institution; means for notifying a user of medical examination result information received from a medical institution; A system including:

2. 2. The system according to claim 1, wherein the medical record information provided to the medical institution includes a diagnosis prediction based on the interview information.

3. 2. The system according to claim 1, wherein the means for analyzing the medical interview information utilizes a machine learning model.

Images